This repository has been archived on 2026-11-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
pascal 1a5bbcd602 feat!: a generic core, and Kalshi through it
The orchestrator, the connection loop and the book thread's shell move to
src/core/, generic over an Exchange and its loss rule (ADR 0004).
Polymarket runs through it unchanged. Kalshi joins: a signed handshake
(KALSHI_KEY_ID_FILE, KALSHI_PRIVATE_KEY_FILE), a reconnect on a skipped
sequence number, first one wins by clock and rank, and its market listing
with markets tracked from three hours before they open.

Breaking: the exchange is now the first argument,
`engineering-challenge <polymarket|kalshi|kalshi-demo> [OUTPUT]`.
2026-11-11 20:54:17 +00:00

49 lines
2.6 KiB
Markdown

# 0003. Two connections per symbol, first one wins
## Context
Extension 1 asks for redundant websockets: connections drop, go quiet, lag,
and get closed by the server (code 1013) when they carry too much. Sprint 03
gave each lane a connection and a book thread of its own
([0002](0002-lanes-and-orchestrator.md)). The obvious next step was a second,
identical connection per lane, one of them the trusted source, with a
Snapshot at each switch, as the brief sketches. Measured on 2026-11-11
([sprint 04](../../roadmap/04.%20redundant%20connections.md)): two connections
on the same symbols deliver the same changes in the same order, 0.5 ms apart
at the median and 100 ms or more for 1% of changes; and two connections
carrying the same symbols are closed together.
## Decision
Every symbol is subscribed on **two connections chosen independently and at
random**, and its one book takes each change from **whichever connection
delivers it first**. Polymarket's book hash, sent with every change, says
where a change belongs: already applied (a duplicate), next (it fits and is
written), or early (held until the change before it arrives). Only when a
change fits on no connection in time does the book thread fall back to
following one connection unchecked and resyncing on the other.
Connections therefore no longer belong to a book thread. A connection thread
stamps, parses and routes each event to the book thread of its market, one of
16, derived from the market's id. The orchestrator of 0002 stays; its lanes
are superseded.
## Why
First one wins gives each Delta the earliest receive time, turns a change one
connection missed into no loss at all, and needs no second book and no
Snapshot per switch. A source and a standby would have kept the source's
receive time and lost the change the source missed. Independent connections
were measured against twins on the same 48,038 symbols for 20 minutes: 272
symbol-seconds with both subscriptions down, against 3,787 for twins dealt at
random and 33,345 for twins in creation order. A move becomes a subscribe and
an unsubscribe with the book left where it is, so it opens no subscription
gap.
Trade-off: the routing step 0002 avoided is back, parsing moves into the
connection thread, and the book thread's rules get subtler (held changes, a
ceiling, a chain declared broken). The design leans on an exchange that lets
the same change be recognised across connections: Polymarket's hash does;
on Kalshi a change's `clock` and its rank among the changes of that clock do
(recorded in M6, [sprint 06](../../roadmap/06.%20generic%20core%20and%20kalshi.md)).