[ts-gen] Unable to retrieve historical data for german stocks

Bill Pippin pippin at owlriver.net
Tue Oct 7 14:09:53 EDT 2008


Glad you've gotten the shim running:

> running TWS version 886.5 + shim-081003
> I'm able to connect to the TWS 

As for determining contract ids, which you do indeed currently
need to submit history queries, market data subscriptions, and

> ... in search for the desired "contract record unique id numbers".

There are two kinds of contract id involved here: the IB-defined
values you can obtain via contract info requests, and the shim
database record id from the left column of sql/syms.txt.  In this
case you need the second kind of contract index.  The first domain
is sparse over (apparently) 2^32; the second, sequential starting
from 1.  When you see the product count in the banner:

          The trading shim has connected to the database server
          and loaded 51649 products; connected to the IB tws as
          client id  1; and queried for the account code, which
          seems to be DU-----.  Program initialization has been

you have a close approximation to the number of contracts in the
shim database, since as a rule, each product has a contract.

To abbreviate a long story to a not very short one:

    1.  for now, you need one of the shim database record ids,
        just the number, e.g., for YM as 142 in the exs/past example;
    2.  in the near future, you'll be able to use either shim record
        ids or IB contracts ids, disambiguating via the notation
        cid:142 or ibc:37225386, both of which refer to
    3.  eventually, you'll be able to use product/route
        descriptions directly, e.g., FUT:ECBOT:YM:20081218;
    4.  at that time shim database contracts will be deprecated,
        since there are many possible routes, and so an
        unmanageable number of possible concrete contracts;
    5.  the IB "contract" ids, which are actually abstract products,
        since the route may vary, will be the preferred way
        to numerically specify contract symbols;
    6.  you'll also be free to use contract descriptions, as long
        as they are unambiguous; and, finally:
    7.  for now you can look up the shim database contract record
        uid, e.g., 142 for YM above, as follows:
        a. in the file syms.txt, using grep or your favorite editor; or
        b. by running the query bin/get_ids.sql, which was used to
           produce the file syms.txt; or
        c. adapting the select statements in bin/get_ids.sql to your
           own purposes.

If the above is confusing, just focus on (7); look up the record id,
which is the first column in syms.txt, and make sure you regenerate
that file if you have changed your local symbol set in any way, say
by recreating the database after updates to mod/LocalSet.sql or

Now, about the syntax, that is token-level, errors.  The shim looks
up most command argument values in tables, and if the lookup
fails, refuses to proceed furthur with that command, and drops it.
So, for commands such as:

    select past     14121 d1   1  2w Ymd_T(20081001  9:00:00);
    select past     62139 d1   1  2w Ymd_T(20081001  9:00:00);
    select past 125071870 d1   1  2w Ymd_T(20081001  9:00:00);

when you see text such as:
>   select past     62139 d1   1  2w Ymd_T(20081001  9:00:00);
>   ---------------------^------------------------------------
>   Terminal tag: name:ConI index:1

it means the contract id (internally to the shim, the ConI type
class) lookup failed for the value 62139.  This is not too
surprising since syms.txt has 45264 lines, and the largest contract
uid there is 45241.  As for contract index 14121, the commands:

    select past     14121 m15  1  2d now; wait 11;

provide, via the log:

    1|12| 0|select past     14121 m15  1  2d ;			|
    1| 2| 0|wait 11;|
    2|20| 3|20|3|1|A7A|STK||0.00||1|SMART||EUR|||8|2 d|0|TRADES|1|
    3| 4| 2|      -1|2106|HMDS data farm connection is OK:euhmds2|
    3| 4| 2|         1| 165|Historical Market Data Service query ...
    3| 4| 2|         1| 162|Historical Market Data Service error ...
    1| 1| 0|exit;|

Where the 165 and 162 error messages from the IB tws include, in the
text following the ellipsis:

    165|query message:HMDS server connection was successful.|
    162|error message: No market data permissions for IBIS STK|

So, I don't get any history data, but that's due to my account
setup, and outside the control of the shim.

Looking at the contents of syms.txt, for cid:14121:

14121   14112   42911678        STK     SMART   A7A     EUR     DE

I see that its description is STK:SMART:A7A, which symbol I'm
not familiar with, though the IB tws seems to think its home is
at IBIS.  Looking at the original load files used to populate the
database, in sql/sym:

    sql$ grep A7A sym/*
    sym/A0Stk07.sql:(42911678, 'STK', 'A7A'       , 'EUR', 'DE' ,
    'ALTIRA', 'FWB,IBIS,SMART,SWB', ''        , 'SMART'   ),

it appears to be a stock with the description ALTIRA, and IBIS is
indeed one of the possible routes.

Please let us know if you can get history queries running for
your setup.  You may want to consider starting from the examples
in directory exs.



More information about the ts-general mailing list