[ts-gen] Do I really need to tail -f ShimText?

Nils Gebhardt mail at ngebhardt.de
Mon Sep 14 17:22:35 EDT 2009



Hi, 


On Mon, 2009-09-14 at 15:26 +0100, Paul C wrote:
> On Mon, Sep 14, 2009 at 09:25:31PM +0900, Ken Feng wrote:
> > 
> . . .
> > Question:  If I need to know the moment that a response to an order is
> > coming from upstream, is the right approach to open a pipe to "tail -f
> > ShimText" and select() on that pipe?
> 
> Hi Ken,
> 
> A few notes on what I do, in case it helps:
> 
> I'm not sure I'm using the 'right' approach, but I also look for the 3/3/6 'Filled'
> response in the shim output.  However, I don't tail -f ShimText but
> instead run the shim with the 'cout' option (shim --risk cout) so
> the response is on the stdout, which in turn is connected to a
> pipe/handle (popen in ruby?). (I'm not actually using ruby, hence the
> lack of an example/code).
> 

No that I can provide answers, rather add some more questions on the
'right way' -.
I also match shim output for 3|3|6 patterns. 


> 
> My approach may be too simplistic - I don't actually utilise any of
> the MySql info, or deal with multiple orders.  I run two instances of
> the shim from within the downstream program: one in --data mode for
> reading prices (select tick...) and the other in --risk mode (as
> above) for sending trades.  With both shim processes I parse the
> output looking for the appropriate |src|tag|_| lines.

I would call my approach also very simplistic. I like awk the most for
pattern based state manipulation upon the shim streaming data pipe end.

One reason is that I am missing from times to times records in the
OrderStatus table which are present in the shim log. However, having
100% reliable order status information seems to be crucial as soon one
feeds it back to some kind of order logic. On a side note: I am not so
much familiar with the code as I would like to be - I just wonder, If
run under gdb, is there a canonical break point in the sources from
where on one might be able to trace OrderStatus messages (and spot at
which they might get lost)?

Currently I also separate between data stream and order data stream on
FIFO pipes. Is there a simple way to concatenate to pipes to a single
stream? Doing a simple redirection I get problems with buffering,
resulting in missing lines as well. And in awk there is no select() I
think - . Maybe it is not necessary to separate these streams at all.
One hope is to get better data quality; in the tick stream I am missing 
some percentage of data. 




regards

Nils



More information about the ts-general mailing list