Archived
The orchestrator, the connection loop and the book thread's shell move to src/core/, generic over an Exchange and its loss rule (ADR 0004). Polymarket runs through it unchanged. Kalshi joins: a signed handshake (KALSHI_KEY_ID_FILE, KALSHI_PRIVATE_KEY_FILE), a reconnect on a skipped sequence number, first one wins by clock and rank, and its market listing with markets tracked from three hours before they open. Breaking: the exchange is now the first argument, `engineering-challenge <polymarket|kalshi|kalshi-demo> [OUTPUT]`.
142 lines
5.1 KiB
Markdown
142 lines
5.1 KiB
Markdown
# Order-book collector
|
|
|
|
The collector's language: what it tracks on each exchange and what it writes
|
|
for the backtests that read its output.
|
|
|
|
## Exchange side
|
|
|
|
**Market**:
|
|
A tradable question on an exchange. On Polymarket it has two outcomes (Yes,
|
|
No), each traded as its own symbol, and their books mirror each other. On
|
|
Kalshi it has one book, holding bids for Yes and bids for No.
|
|
_Avoid_: Event, contract
|
|
|
|
**Symbol**:
|
|
The unit that has one order book: on Polymarket a token (one outcome of a
|
|
market), named by its token id; on Kalshi the market itself, named by its
|
|
ticker, a bid for No at a price being an ask at one minus that price.
|
|
_Avoid_: Asset, instrument, ticker (for Polymarket)
|
|
|
|
**Live**:
|
|
Said of a market (and its symbols) the exchange lists as open for trading on
|
|
its order book. A market stops being live when it closes or stops accepting
|
|
orders, not at its end date. A Kalshi market about to open is tracked ahead
|
|
of being live, its book empty.
|
|
_Avoid_: Active (Gamma's flag, only one input of being live)
|
|
|
|
**Book hash**:
|
|
Polymarket's fingerprint of a symbol's full book state, sent with every
|
|
full book and every change.
|
|
|
|
**Sequence number**:
|
|
Kalshi's count of the messages it sent on one connection, all its markets
|
|
together: one skipped means the connection lost a message, for a market it
|
|
doesn't name.
|
|
|
|
**Clock**:
|
|
Kalshi's counter for one market, the same on every connection, carried by
|
|
every book and change. Several changes can share one clock, always in the
|
|
same order.
|
|
|
|
## Collector side
|
|
|
|
**Book**:
|
|
The collector's in-memory order book for one symbol: its bid and ask price
|
|
levels, each with its absolute quantity.
|
|
_Avoid_: Order book state, depth
|
|
|
|
**Connection**:
|
|
One websocket to the exchange. Every live symbol is subscribed on two
|
|
connections, chosen independently of each other.
|
|
_Avoid_: Socket (for the concept; fine for the code), stream, lane (the
|
|
retired pairing of one connection with one book thread)
|
|
|
|
**Load**:
|
|
How much a market weighs on a connection: the bytes per second of its frames
|
|
there. A connection's load is the sum of its markets'.
|
|
_Avoid_: Activity, volume (the exchange's traded amount, unrelated)
|
|
|
|
**First one wins**:
|
|
The rule for a symbol's two connections: a change is taken from whichever
|
|
delivers it first, with that receive time, and the other's copy is dropped.
|
|
_Avoid_: Source, primary, standby, failover (no connection has a rank)
|
|
|
|
**Duplicate**:
|
|
A change another connection already delivered.
|
|
|
|
**Verified**:
|
|
Said of a symbol whose book gives the exchange's book hash after every
|
|
change. It stops being verified when a change fits on no connection in time.
|
|
|
|
**Held**:
|
|
Said of a change kept unwritten because it doesn't fit the book yet: a change
|
|
before it is still to come, from this connection or the other.
|
|
_Avoid_: Buffered, pending, queued
|
|
|
|
**Rescued**:
|
|
Said of a change one connection missed and the other delivered: the output
|
|
has it, where a single connection would have had a loss.
|
|
|
|
**Follow**:
|
|
To apply one connection's changes unchecked, while a symbol can't be
|
|
verified and until a fresh book says where it stands.
|
|
|
|
**Split**:
|
|
Moving half the markets off a connection the exchange keeps closing.
|
|
|
|
**Quiet**:
|
|
Said of a symbol's subscription on a connection that is up, while the
|
|
exchange has stopped sending that symbol's changes there. Nothing tells us:
|
|
it has to be looked for.
|
|
_Avoid_: Stale, dead (a connection can be fine while one of its
|
|
subscriptions is quiet)
|
|
|
|
**Resubscribe**:
|
|
To unsubscribe a symbol on one connection and subscribe it again there, which
|
|
makes the exchange start its subscription afresh. A resync is a resubscribe
|
|
whose book we asked for.
|
|
|
|
**Silent**:
|
|
Said of a connection that is open but delivers nothing at all, not even the
|
|
answer to its keepalive.
|
|
|
|
**Recycle**:
|
|
To close a working connection on purpose and reconnect it with the same
|
|
symbols, to shed what has gone wrong on the exchange's side of it.
|
|
_Avoid_: Restart, rotate
|
|
|
|
**Receive time**:
|
|
When a message came off the socket, by the local clock.
|
|
_Avoid_: Arrival time, local timestamp
|
|
|
|
**Loss**:
|
|
A change for a symbol that no connection delivered in time, which leaves its
|
|
book silently wrong. A change only one connection missed is rescued, not
|
|
lost.
|
|
_Avoid_: Gap (a skipped sequence number is one connection's, and may be no
|
|
loss at all)
|
|
|
|
**Resync**:
|
|
Asking the exchange for a fresh full book of a symbol and replacing ours with
|
|
it, whenever our book can't be verified. Our book was wrong only if the fresh
|
|
one differs: a loss, or a subscription gap.
|
|
_Avoid_: Re-snapshot, rebuild, refresh
|
|
|
|
**Subscription gap**:
|
|
The changes our own subscribe keeps from us, whether it opens or resyncs a
|
|
symbol: the exchange holds them in the book it sends but never sends them as
|
|
changes. A connection that stays subscribed meanwhile has no such gap.
|
|
_Avoid_: Resync gap (the older, narrower name); calling it a loss (the state
|
|
stays right; only those updates are missing)
|
|
|
|
## Output
|
|
|
|
**Snapshot**:
|
|
The output message holding one symbol's whole book at a point in time.
|
|
_Avoid_: using it for the action of fetching a fresh book (that is a resync)
|
|
|
|
**Delta**:
|
|
The output message changing one price level of one symbol's book to a new
|
|
absolute quantity (`PriceLevelUpdate` in the schema).
|
|
_Avoid_: Increment, diff
|