[ts-gen] OrderStatus inserts missing [Was: OrderStatus dups ...]

Bill Pippin pippin at owlriver.net
Fri Aug 21 18:03:52 EDT 2009


Evidently some OrderStatus records aren't making it into the database.
Your problem description indicates that log comments occur, the
database insert doesn't, and some kind of read --- likely command or
message, and almost certainly not database record --- fails at the
token level.  You also say that the problem changed to some degree
after you recreated the database. 

I believe you are seeing real problems, although it's not clear to me
yet what the chain of causation is.

I'll focus first on the syntax errors; after all, if the shim can't
make sense of input, it can't very well process it effectively.  So,
about the token-level syntax errors:

> For token text: [        ] and type target [Str]
> predicted token type:  str, scanner saw:  err
> Problem: 401 unexpected token text: syntax error
> Problem: 404 probe failed: messages not in sync?
> Drop: \n
> Del;    

What api level are you working with; more precisely, what api level
does the IB tws offer, and is the shim working at api level 23?
What build of the IB tws are you using?  By way of example, when I 
choose Help->About Trader Workstation, the panel that appears
includes the text "Build 897.6, Aug 4, 2009 9:23:37 AM"; is yours
newer or older than that, and what is it?

By the way, although the IB web site claims that you can download
a version of TWS 897 from Aug 18 2009, when I downloaded it a
moment ago, the jar file proved to be exactly the same as the
one that runs as built on Aug 4, so I'm probably running the newest
available for Unix.

Can you confirm what stream (downstream command or upstream message)
and what object (cmd name, msg name) is being read when the above
text occurs?  I'm ruling out database reads, since the
out_of_sync_probe_failed message doesn't apply to table reads, where
the expected table record format is already known.  I also suspect
that the problem is occurring with messages, since they are so much
more frequent than commands, and if there was a command syntax error,
I would expect that you probably would have diagnosed it by now.

Granted that the token level syntax problem occurs with IB tws api
messages, then what kinds of messages occur nearby, both previously
in the log, as well as after?

Have you included all of the error text produced by the shim?  The
first and last parts are particularly critical.  One problem here
is that since the shim is probing for the initial token at the
beginning of an event --- 404 probe failed tells us that much ---
and it's hard to tell what exactly has gone wrong, since there is
so little context.

With the file and save options in effect, so that the IB tws binary
input is copied to log/tws2shim.bin, and using the bin/msg.filter
script to look at the above binary log, can you tell where in the
text stream the problem is occurring?  Note, the bin/msg.filter
script is necessarily imprecise, and by inserting newlines, it may
well make it harder for you to be sure what's going wrong.  So,
if you have a text editor than can handle unlimited line lengths,
such as vi, just piping the tws2shim.bin file through tr to
replace ascii NULs with bars will convert the file to text, although
it will all be on one line:

    cat log/tws2shim.bin | tr '\000' \|

Then, if are using vi, and you so choose, you can incrementally
insert your own line breaks using the following greedy approach
(false positives will occur), e.g., for tick price messages:


where the above is typed as colon-percent s-slash
vertical bar-one-vertical bar-five-vertical bar-slash-CtrlVM ...,
and Ctrl indicates the control key being held down while you
type cap V-cap M .

Cryptic, I agree, but you're working with the raw, binary stream,
and the IB tws api protocol completely lacks record separators
or synchronization information, and we have to take what we can
get here.

In any case, examining the original binary input may be time
consuming, and I'm not sure how much you'll be able to understand
of what you see, but I would like to somehow know what the shim
was trying to read when it failed, and especially what it did with
the material that came afterwards.

In the below text, we see that the shim was starting to read a
message or command [probe failed ...], probably read spaces or
control chars instead, wanted something more, that is a command
verb or message index number, viewed what it saw as leaving the
scanner in an error state [scanner saw: err ...], dumped it
[drop, del], and wondered aloud whether message synchronization
had been lost, which is particularly a problem with api-level
conflicts for IB tws messages.

> For token text: [        ] and type target [Str]
> predicted token type:  str, scanner saw:  err
> Problem: 401 unexpected token text: syntax error
> Problem: 404 probe failed: messages not in sync?

Keep in mind here that the IB tws just kills the socket connection
when it has problems reading a request, so that the shim, which
can recover and resync for messages, is actually working to a higher
standard of lexical processing.

The token-level syntax error may not be the only problem, and for
that matter may not be related to the failed inserts, but I have
to start somewhere, and the syntax errors certainly are suspicious.

My first guess is an api level problem, perhaps you're using a
newer IB tws build and I need to fix one of the grammar rules.
At this point I can't be sure; as I noted above, I'm using the
newest one I can get.

I expect there are other problems as well, but the token-level syntax
errors are my natural starting point.



More information about the ts-general mailing list