Every live market is subscribed on two connections chosen at random, and its book takes each change from whichever connection delivers it first: the book hash says whether a change is a duplicate, the next one, or one to hold until the change before it arrives. A change one connection misses costs nothing, each Delta carries the earliest receive time, and a connection that drops is covered by the other. When a change fits on no connection in time, the book follows one connection unchecked and asks the other for a fresh book. Connections now decode and route events to 16 book threads, independent of them; a move leaves its old connection only once the new one delivers, so it costs no change; a connection closed with 1013 twice within ten minutes is split. reconcile reads the collector's log and fails only on a mismatch no cause line explains. SIGTERM and SIGHUP stop the collector cleanly. Breaking: lanes are gone (connections and book threads are separate), and reconcile takes the log as a second argument.
Read before getting started!!
First, read src/main.rs. Then, some background on this repo:
This repo uses the FIX SBE standard
for writing binary messages. The rust code abstracts this away for you, and you can simply use the helpers, as demonstrated in main.rs.
More background, in case you are curious!
FIX SBE is defined in XML, which I find extremely cumbersome to edit, especially when updating schemas frequently. To solve this, I have also introduced some helper code that takes our own custom YAML format, and turns it into FIX SBE XML, which in turn is used to generate rust code to encode and decode the messages. You don't need to edit any of these files. This explanation is simply to help you understand the origins of the various files in this repo.
If you're curious to see the YAML schema, take a look at challenge_orderbook.yaml.
We also use a custom timestamp field, so that the Rust struct can use
chrono::DateTime in the generated struct, and in SBE schema, it is a uint64
representing the timestamp as the number of microseconds since unix epoch.