[ts-gen] Quality of market data via api [Was: Keeping TWS/shim up ...]

Ken Feng kfmfe04 at gmail.com
Tue Aug 18 15:14:53 EDT 2009


Hi Bill,

This >is< a tough question, indeed.  At the very core, you must ask yourself:

- At what level do I trust live TWS data?

On one hand, you must trust it at some level, or you might as well not
trade at all.  On the other hand, trusting every bit that comes down
the pipe blindly could be very costly.

Your solution implies that tick data is more accurate than bar data.
This could be true if there are bugs in the code that translates from
tick data to bar data - there could be bugs due to handling of
real-time.  For exchange-traded securities, like futures, if you do
enough due diligence, you can verify the accuracy of data.  It is
definitely doable historically and it may be doable in real-time.

Forex, on the other hand, isn't traded on an exchange.  Which means
even if you had 2 or 3 or 4 different data sources, you cannot be sure
which one is right.  In fact, they may be all right but different, but
they should not be vastly different.  So, theoretically, it is best to
use historical data from the broker that you are going to trade from.
But even this expectation has a ton of assumptions behind it.  BTW, at
this point, I am not trying to collect ticks to build a database.  I
am collecting ticks so I can feed it into a ATS to generate orders.  I
do have to make sure I don"t get hosed by zeros or some crazy
ticks/bars.

Thank you, Bill, for your detailed description of the problems with
TWS bars - I may have to look into ticks later on as a cross-check.
The algorithm converting ticks to bars is rather simple, so I suspect
when bars are bad, maybe the underlying ticks are also bad.  But this
may not be accurate as I have seen cases where highs and lows were set
to zero, but I cannot tell if this is due to IB being wrong or if I am
using the shim incorrectly, as I am still learning how to use the
shim.  Over time, things will sort themselves out, and if I find that
bars are really causing trouble, I will dig deeper.

I guess my attitude is, be aware of problems, but don"t be overly
obsessed by them (except to fix them, or at the very least, not to be
hurt by them), or you will never leave the starting gate...  As
programmers, we tend to want to get it as right as possible before
jumping in, but in trading, at some point, you have to decide "this is
good enough" and go for it.

As always, I am learning a great deal from your posts and from Russ' - TY.

- Ken


More information about the ts-general mailing list