[ts-gen] Keeping TWS/shim up for an entire week?

Richard Pruss boadie at gmail.com
Tue Aug 18 02:33:11 EDT 2009


Ken,
How are you getting the addition decimals on forex bars, your output
seems to imply 3 on YENAUD?  I only seem to get two decimal places in
the ShimText, regardless of currency.

49076|59268|1577398571|3|50| 3|    6|1250576860|   78.49|   78.49|
78.47|   78.47|    -1|   -1.00|    -1|CASH::AUD:JPY:
49076|59270|1579549294|3|50| 3|    4|1250576865|    0.83|    0.83|
0.83|    0.83|    -1|   -1.00|    -1|CASH::AUD:USD:
49076|59270|1579550410|3|50| 3|    5|1250576865|    0.83|    0.83|
0.83|    0.83|    -1|   -1.00|    -1|CASH::AUD:USD:

Clearly on something like AUDUSD one needs 4 decimal places, to do
something useful with the log.

-Ric

On Tue, Aug 18, 2009 at 3:58 PM, Ken Feng<kfmfe04 at gmail.com> wrote:
> Hi Bill & Russ,
>
> Regarding my daily 3:55UTC problem,
>
>> 1.  For example, I know for a fact that around 3:55UTC every single
>> day, I get an odd behavior in the system where the TWS stops
>> fulfilling the bar requests, even though TWS itself still receives
>> ticks and continues to display them in the GUI.  When I kill the shim
>> and restart it manually, the response has been sporadic - sometimes,
>> it works on the first try.  Sometimes on the second or third try.  My
>> guess is, the TWS has fallen into a state that I have not yet
>> anticipated, for whatever reasons (maybe daily maintenance by IB?),
>> and I need to to either detect the state at any time or patch around
>> the 3:55UTC gap.
>
> following Bill's suggestion that I cancel the bar requests first, I
> was able to continue receiving ticks without manual intervention.  The
> gap, however is kind of large:
>
> AUDJPY|2009-08-18 03:54:55|77.960|77.960|77.960|77.960
> AUDJPY|2009-08-18 03:55:00|77.960|77.960|77.960|77.960
> AUDJPY|2009-08-18 04:05:25|78.040|78.050|78.040|78.040
> AUDJPY|2009-08-18 04:05:30|78.040|78.040|78.040|78.040
>
> I dug into ShimText and found the following in summarized form (table
> lines up in a monospaced font:
>
> ----------------------------------------------------------------------
> secs  utime      utc      comment
> ----------------------------------------------------------------------
> 46505 1250567700 03:55:00 Last Good Tick at 03:55:00
> 46509                     MDS data farm connection is broken:ushmds2a
> 46512 ----- TRIGGER ----- MDS data farm connection is OK:ushmds2a
> 46512                     Ruby: begin sending 15 cancel bars to shim
> 46527                     Ruby: begin sending 15 select bars to shim
> 46861                     Market data farm connection is broken:cashfarm
> 46939                     Market data farm connection is OK:cashfarm
> 47111 ----- TRIGGER ----- MDS data farm connection is OK:ushmds2a
> 47111                     Ruby: begin sending 15 cancel bars to shim
> 47113 1250568305 04:05:05 EURCHF, EURJPY, USDCAD, EURUSD, USDJPY, EURGBP,
>                          EURCAD, GBPCHF, NZDUSD, USDCHF, GBPUSD, GBPJPY
> 47126                     Ruby: begin sending 15 select bars to shim
> 47130 1250568325 04:05:25 AUDJPY, AUDUSD, and others come in
> ----------------------------------------------------------------------
>
> I am TRIGGERing 15 cancel bars and 15 select bars to be sent to the
> shim on "MDS data farm connection is OK:ushmds2a", and even though
> this works, the behavior appears to be sub-optimal.
>
> 1. Observe 47113 - the fact that I am already receiving ticks here
> implies that 46512 and 46527 are sufficient, while 47111 and 47126
> appear to be redundant.
> 2. If I use no TRIGGERs at all, I will fail to receive any ticks at
> all after 46505.
>
> So given 1. and 2., we now have some state-based ugliness.  It almost
> seems that instead of TRIGGERing on "MDS data farm connection is
> OK:ushmds2a", I must TRIGGER if and only if "Market data farm
> connection is broken:cashfarm" and "Market data farm connection is
> OK:cashfarm" do not immediately proceed it.
>
> I think this kind of state-based code should work to prevent 47111 and
> 47126 redundancies, but it is extremely brittle and prone to breakage,
> and difficult to debug when the server changes.  I can go for
> "tight-coding" where the state-based code is expected to work only for
> this particular case, or I can go for "general-coding" where it will
> apply to similar events throughout the day.  Without any
> specifications from IB, I can only guess which type of coding may be
> more "robust".  At this point, I am towards going with the
> "tight-coding", perhaps even limiting the TRIGGER to a certain range
> of UTC time.
>
> If anyone has any better, simple solutions, please do let me know!
>
> BTW, Russ' suggestion of modifying .shimrc values like ShimText before
> shim startup also works very well.  I now make my Ruby script modify
> the .shimrc to output to yyyymmdd.log before starting up the shim.
>
> Thank you.
>
> - Ken
> _______________________________________________
> ts-general mailing list
> ts-general at trading-shim.org
> http://www.trading-shim.org/mailman/listinfo/ts-general
>



More information about the ts-general mailing list