[ts-gen] Order states and db version ++ [Was: Risk-related events ...]

Bill Pippin pippin at owlriver.net
Mon Aug 24 15:12:02 EDT 2009


To summarize, most of this message, 1: lists the IB-defined order
states from their documentation, 2: notifies the list that
the database version has been incremented, since the OrderStatus
create table definition has been extended to comprehensively follow
the IB-documented order states, with the result that the database
version is now 1.76; and 3: speculates that this change may fix
the problem Nils has observed for OrderStatus inserts failing. 

About the journal:

> Great job on these journaling tables - will save me from having to
> implement my own!

You're welcome.  By the way, your question below probably suggests
a fix for the problem Nils has observed, so thanks much for asking

> ... I do see the status=="filled" in ... OrderStatus ...
> ... Any other ... states I am unaware of, but should check for?

You ask about order states, as did I back in May on the Yahoo list.
EikiMera was kind enough to point out that the answer lies in the
APIguide.pdf document.

Following this paragraph I give a [reordered, in two parts] quote
from that manual.  The first quotation lists the states that you might
see in actual order status messages, and so in the journal, in the
OrderStatus table.  Since we use a mysql enum for this attribute, the
values in the database domain have a dual representation as either
numbers or strings, although you should note that in the original
messages, the states are indicated by strings.  So, the database
numbering for the observed states from 3 to 7 is ours.  From IB's

    * PreSubmitted - indicates that a simulated order type has been
      accepted by the IB system and that this order has yet to be
      elected. The order is held in the IB system until the election
      criteria are met. At that time the order is transmitted to the
      order destination as specified.

    * Submitted - indicates that your order has been accepted at the
      order destination and is working.

    * Cancelled - indicates that the balance of your order has been
      confirmed canceled by the IB system. This could occur
      unexpectedly when IB or the destination has rejected your

    * Filled - indicates that the order has been completely filled.

    * Inactive - indicates that the order has been accepted by the
      system (simulated orders) or an exchange (native orders) but that
      currently the order is inactive due to system, exchange or other

The IB docs also list two start states they indicate will not appear
in api messages, which is natural, since messages are events, and the
states they include are next states given the event itself.  If these
states occurred in the journal, which as noted they do not, they would
have our indices 1 and 2, respectively:

    * PendingSubmit - indicates that you have transmitted the order,
      but have not yet received confirmation that it has been accepted
      by the order destination. NOTE: This order status is not sent by
      TWS and should be explicitly set by the API developer when an
      order is submitted.

    * PendingCancel - indicates that you have sent a request to cancel
      the order but have not yet received cancel confirmation from the
      order destination.  At this point, your order is not confirmed
      canceled.  You may still receive an execution while your
      cancellation request is pending. NOTE: This order status is not
      sent by TWS and should be explicitly set by the API developer
      when an order is canceled.

Since our code is not derived from the sample client sources, we are
free to ignore their unwanted advice about how start states should be
defined.  Instead of using the above start states in the command side
of the journal, we indicate events explicitly in ChangeOrder records,
as one of the following:

    act     enum('create', 'modify',
                 'submit', 'cancel',
                 'modsub', 'urgent') not null ...

Our create and cancel correspond to the IB PendingSubmit and
PendingCancel until a reply message shows up from the upstream.

You can find mention of the IB-defined order states in the
create table definition for OrderStatus.  I've just pushed a
new release that includes a relatively rare state, Inactive,
that was not defined in the database previously, so the
database version increments to 1.76 .  The following is a two-part
excerpt from the create table statement for OrderStatus:

--  status  enum('Pending', 'Submitted', 'Filled',		-- thru db
--               'Partial', 'Cancelled') not null,		-- ver 1.75

    status  enum('PendingSubmit',
                 'PreSubmitted', 'Submitted',
                 'Cancelled', 'Filled', 'Inactive') not null,	--  2

The old state list remains as comments, and the number of IB-defined
states increases from 5 to 7, of which they claim to use only the last
five.  Note that the character string descriptions do not completely
describe order state; partial fills are indicated by a non-zero
remaining quantity, so any comprehensive description of order
state must also consider the next two attributes in the
OrderStatus table: 

    filled      int unsigned           default 0,		--  3
    remaining   int unsigned           default 0,		--  4

Now, about failed OrderStatus inserts:

The shim uses strings to represent the IB-defined order state symbols
when constructing the related order status message term, so even if
IB adds new states, the shim's message parser will keep working.  The
database inserts, however, do check those symbols against the enum
list, and will fail if the order state symbol is not in the enum list;
this is an example of the kind of typechecking used widely in the
trading-shim architecture.  Since the IB order state PreSubmitted
did not have an explicit entry before this, when and if it occurred,
order status inserts for orders with that state symbol would have
failed.  I don't know if this explains the problem Nils has observed,
but it seems likely. 



More information about the ts-general mailing list