[ts-gen] Blowup in Paper Account due to failed reception of OrderStatus 3/3/6 Filled

Ken Feng kfmfe04 at gmail.com
Thu Sep 24 20:06:24 EDT 2009

Hi Nils,

I did read your thread with interest last night before my mind turned  
into mush.  I think your problem may be related to what happened to  
me, but maybe not (more on this later).

Assuming that we are having the same problem, shouldn't an INSERT  
IGNORE just replace the old row with the new row, even if there were a  
unique key constraint?  I only care about (and generate reports on)  
the newest entries in OrderStatus.

Just to make things clear, in the 090904 code,
1. Is there an INSERT IGNORE for OrderStatus?
2. Is it true that in your case, you saw an entry in the Log, but not  
in OrderStatus?

If 1. is true, shouldn't we get the new row, while the old row gets  
destroyed?  Which would be ok in my case since I do a query to get the  
newest row according to the time field for a given perm/client_id.

If 2. is true, I am also very interested in the resolution of your  
problem, but it may be different from mine.
AFAIK, in my situation, the change in status didn't make it into the  
Log or the OrderStatus.
I've looked in my log files several times to confirm, but I will check  
again this morning to see if I can find any clues to what happened  


Here is where I think my problem may be different from yours.
3. My OrderResult table appears to hold accurate information, but...
4. My OrderStatus and OrderReport and listening for 3/3/6 Filled all  
failed to detect the fills

I am reviewing the logs this morning to see if I am missing anything  
or if I have misread something.

Meanwhile, I am seriously considering writing a kludgy GUI automator  
to download File/View Trades/File/Export Today's Reports (running  
every five minutes), as suggested in another thread, to ensure that I  
have another way of verifying fills.  The interesting thing is, if I  
go down this path, I've noticed that TWS doesn't give me any unique  
fields, so I may end up going full-circle, where you are - dealing  
with row-uniqueness issues.

- Ken

On Sep 25, 2009, at 3:10 AM, Nils Gebhardt wrote:

> Ken,
> On Thu, 2009-09-24 at 20:49 +0900, Ken Feng wrote:
> [...]
>> Today, while paper trading, my shim --risk failed to detect responses
>> in OrderStatus through a 3/3/6 after 15:00.  This caused a cascade of
>> problems as my system continued to submit orders thinking that  
>> nothing
>> was filled, when indeed, there were fills, according to TWS.
>> At first I thought it was a problem with my Ruby code so I looked at
>> the OrderStatus table.  However, OrderStatus did not detect any fills
>> after 12:30:55, either.  My third check in the .log file for 3/3/6
>> confirmed that "Filled" was received by the shim.  So I conclude that
>> the IB server did not send me a proper 3/3/6 Filled, for whatever
>> reason.  In other words, I can't seem to rely on 3/3/6 or the
>> OrderStatus table to detect my fills.
> to my current knowledge the most probable reason is a uniqueness
> constraint violation.
> You can follow Bill's hints on the list from Sep, 15 ("Debugging
> intermittent lost posts"). I found useful a
> fprintf(stderr, "PerQ: %s\n", q);
> in fill.c around line 82.
> This might give you an indication what shim tried to insert into
> OrderStatus, but maybe couldn't because of a duplicate key. You can
> verify this if you take corresponding insert statements from stderr,
> provided you can reproduce an error case - and try to insert it  
> without
> ignore clause.  The problem seems to be that the order_tag value is  
> not
> as unique as it seems at the first glance.
> I will try to rebuild the database without uniqueness constraint in
> OrderStatus and see what happens.
> I hope the order_tag is unique at least for a single day so that one
> still can make useful joins with other journal tables.
> If you want to verify key uniqueness for fills, which are in the log  
> but
> not in OrderStatus, you can pick up the order_tag value from the 7th
> field of log, e.g. if you have something like
> 29575|41427| 178589666|3| 3| 6|1245|Submitted|0|1|0.000000|60488164|0|
> 0.000000|8|
> then '1245' should correspond to order_tag, as I understand. From the
> remaining fields it should be possible to reconstruct the whole
> OrderStatus insert message.
>> However, I have noticed that clicking on the "ACCOUNT" button on TWS
>> does show the correct positions in the FX Portfolio section (eg aaa
>> QUESTION:  Does anyone know the command to submit to the shim to get
>> the same information that is on this screen?
> select acct;
> should give the corresponding information - and should always give  
> the correct information,
> even if other tables or even the GUI is out of sync.
> regards
> Nils
> _______________________________________________
> ts-general mailing list
> ts-general at trading-shim.org
> http://www.trading-shim.org/mailman/listinfo/ts-general

More information about the ts-general mailing list