[ts-gen] Re: ts-gen] new user

R P Herrold herrold at owlriver.com
Sun Apr 15 01:18:45 EDT 2007

On Fri, 13 Apr 2007, Andrew McClure wrote:

Hi Andrew, and welcome -- I got an email from the mailman, 
which had held you post as you were not posting from an 
address it recognized as one subscribed to the list; it is 
prefectly fine to register and set to 'nomail' all but one 
account, so that hte list processor will pass email from you 
from multiple addresses, and I would encourage this.  I have 
manually CC'd you directly in on this post so you are sure to 
see it

> I recently downloaded, and installed shim on a debian box.
> Everything connects and works, and I get some output to a 
> file when I run it. (See below).
> Apr 13 01:16:07 shim : shim|data|0.28| 3944|04566|   1039290|4|100| 5|# 
> |4|100|5|*******************|
> Apr 13 01:16:07 shim : shim|data|0.28| 3944|04566|   1039301|4|100| 5|# 
> |4|100|5|version|0.28|070228|
> Apr 13 01:16:07 shim : shim|data|0.28| 3944|04566|   1039306|4|100| 5|# 
> |4|100|5|*******************|

This is the startup banner that the shim has gotten through 
the initiailzation phases, connected to the database, and 
connect to the TWS

> Apr 13 01:16:07 shim : shim|data|0.28| 3944|04566|   1043444|3| 9| 1|1|
> Apr 13 01:16:07 shim : shim|data|0.28| 3944|04567|   1346184|3| 4| 2| 
> -1|2104|Market data farm connection is OK:usfuture|

And this is the handshake from the TWS, and the passthrough 
message that a connection to the datafarm in question has been 

> Anyway, it is a mystery to me which symbol this is for, and where I can 
> change it.

No symbol yet has been specified, nor any symbol specific data 

> I have to say, the documentation for the product with 
> respect to actually getting data, is a little thin,

Yes indeed, but largely we seemed to not have attracted any 
co-developers, nor indeed any who responded or commented 
further after talking them through setup and compile issues; 
notwithstanding that, I have personal notes which I'll look 
over and pound on a bit to make them something I'd be proud to 
release, and I know Bill has been working on manual matters as 
well.  As such we focussed on pushing some features further 
along, and on some deferred matters in the code we had wished 
to address.

> and seems to boil down to a couple of lines about how 
> important it is to understand the schema, and two tables in 
> particular - I have looked at these, but failed to find an 
> obvious correlation with the above data.

Try running a couple more of the scripts during US market 
hours, and the logfile will show you much. (See paragraph 
after the next)

Our test scripts are US centric, because that is where we 
develop and it is convenient to our local work schedule.  The 
code works for the SPI.SNFE.FUT just fine as well.  Z.LIFFE 
and the DAX are just too early in the day for us.  Currencies 
should work, but seem not to have ticks as we are accustomed 
to seeing.  Someone who knows what they are seeing being 
available to comment would help us there. ;)

The bin/regress script (which is run with 'make test' ), and 
the bin/unsafe are the next things to run at this point.  We 
run them all day, every day as changes are made to the code to 
expose any intorduced regressions at once.  I am substantially 
certain this is documented but the hour is late here, so I'll 
defer a specific pointer until Monday

> Sorry to request the obvious, but could someone please walk 
> me through the steps of figuring out where this data is 
> from, and how I can configure shim to make changes.
> In simple terms, I want to be able get both historical and 
> real time data, and ideally, to populate a database table 
> with this.  Would love to hear from someone who is 
> successfully up and running with this too.

We do this and have for months and months.  Tick by tick (at 
the IB 300 mSec interval is not stuffed into the database -- I 
aggregate out a OHLC watching the data stream presently, and 
we are in process making this a bit friendlier in our work the 
last few weeks)

History returns will be inserted into the shim's database 
presently with no further interversion -- if one is wiping and 
reinstalling the database, one has to 'harvest' that data into 
a permanent store, but that is easy enough to do.

We'll walk through it on Monday.  If you have a preferred set 
of tuples, please send them along as we'll craft an example 

A well formed, full tuple looks like:
and so forth, in the shorthand we have adopted.  Obviously 
with futures, we infer the front month when we omit it.

Again welcome, and remember to subscribe all addresses from 
which you intend to post, so they sail through after the 
anti-spam milter concludes you are using a real MTA at the far 
end -- less than 30 minutes tops.

-- Russ Herrold

More information about the ts-general mailing list