feat!: a generic core, and Kalshi through it

The orchestrator, the connection loop and the book thread's shell move to
src/core/, generic over an Exchange and its loss rule (ADR 0004).
Polymarket runs through it unchanged. Kalshi joins: a signed handshake
(KALSHI_KEY_ID_FILE, KALSHI_PRIVATE_KEY_FILE), a reconnect on a skipped
sequence number, first one wins by clock and rank, and its market listing
with markets tracked from three hours before they open.

Breaking: the exchange is now the first argument,
`engineering-challenge <polymarket|kalshi|kalshi-demo> [OUTPUT]`.
This commit is contained in:
pascal committed 2026-11-11 20:54:17 +00:00
1 parent 480ee3ad2d
commit 1a5bbcd602
33 files changed
+8150 -1846

No files matched your search

+95 -4
View File
@@ -4,15 +4,17 @@
A delta carries the **new total quantity** at a price level, not a change to it.
`quantity: 400` means "this level is now 400," not "+400". Overwrite the level;
never add. `quantity: 0` means delete the level. (Schema confirms: "absolute
available quantity".)
available quantity".) That is the output, and Polymarket's wire. Kalshi's wire
is relative (`delta_fp`: "+400"): the book adds it, and the Delta written
still carries the level's new total.
## A missed message means a wrong book: resync it
If a message is lost, the in-memory book is silently wrong and cannot be
patched — throw it away and resync that symbol from a fresh full book. This is the
"completeness" requirement made mechanical. *How* a loss is detected depends on
the exchange: a sequence-number gap where one exists (Kalshi, to verify in M6),
something else where it doesn't — Polymarket has no sequence number, but its
book hash does the job (below).
the exchange: a sequence-number gap where one exists (Kalshi: per connection,
below), something else where it doesn't — Polymarket has no sequence number,
but its book hash does the job (below).
The challenge brief prescribes no mechanism; it only asks for complete data.
## Polymarket market channel (observed 2026-11-04)
@@ -222,6 +224,95 @@ connection recycled every 60 s (sprint 05).
20 s): a check reaching back 5 minutes misses some. 195,130 markets were
live that evening.
## Kalshi's websocket (demo, observed 2026-11-11)
From Kalshi's docs and four recordings of the demo with a throwaway recorder
(`../kalshi-spike/`, outside the repo). Production is untried where a number
is the demo's.
- **Hosts:** production `wss://external-api-ws.kalshi.com/trade-api/ws/v2`
and `https://external-api.kalshi.com/trade-api/v2`; demo
`wss://external-api-ws.demo.kalshi.co/trade-api/ws/v2` and
`https://external-api.demo.kalshi.co/trade-api/v2`. The older
`api.elections.kalshi.com` and `demo-api.kalshi.co` still answer.
- **Auth:** the websocket handshake carries `KALSHI-ACCESS-KEY` (the key id),
`KALSHI-ACCESS-TIMESTAMP` (ms) and `KALSHI-ACCESS-SIGNATURE`: base64 of the
Ed25519 signature of `<timestamp>GET/trade-api/ws/v2`. Without them: 401.
The market listing needs no key. Keys are created per environment
(Account → Profile → API Keys); the private key is a PKCS#8 PEM shown once.
- **Subscribe:** `{"id":1,"cmd":"subscribe","params":{"channels":
["orderbook_delta"],"market_tickers":[...]}}` is answered
`{"type":"subscribed","id":1,"msg":{"channel":"orderbook_delta","sid":1}}`,
then one `orderbook_snapshot` per market. A connection has one such
subscription: a second `subscribe`, or `{"cmd":"update_subscription",
"params":{"sids":[1],"market_tickers":[...],"action":"add_markets"}}`, adds
markets and sends their books; `delete_markets` stops a market's changes;
`get_snapshot` sends books again (carrying the command's `id`). Each `ok`
answer lists every market of the subscription unless `skip_ticker_ack:
true` is sent. An empty ticker list is error 14.
- **`orderbook_snapshot`:** `{"type","sid","seq","sending_ts_ms","msg":
{"market_ticker","market_id","clock","yes_dollars_fp":[["0.4800","80.00"],
...],"no_dollars_fp":[...]}}`. A side with no level is omitted. Bids only:
YES bids and NO bids. No time of its own.
- **`orderbook_delta`:** `msg` is `{"market_ticker","market_id","clock",
"price_dollars":"0.9600","delta_fp":"-54.00","side":"yes"|"no","ts",
"ts_ms"}`: a signed change of the level's quantity. Folding them gave the
exchange's book every time (1,352 `get_snapshot` books equal to ours, no
level below zero).
- **`use_yes_price: true`** in the subscribe gives NO levels at `1 − q`, the
YES ask they are (books and changes alike); the default, `false`, gives
`q` and is announced to flip.
- **`seq`** is contiguous per connection across all its markets, counting
books, changes, `ok` and `error` (not `subscribed`): no gap in 343,000
messages.
- **`clock`** is per market, the same on every connection, never going back,
and shared by 2 to 4 changes for 12% of its values (some identical in
content). A book's clock is its last change's: none of 1,378 books was
followed by a change of its own clock or an older one. Two connections
deliver a market's changes in the same order (2,643 changes on 26 markets),
the second copy 2 ms later at the median, 344 ms at worst.
- **No limit met per connection:** 30,000 markets in one subscribe message,
or in 30 messages 20 ms apart; every book within 1.7 s; ~1,560 changes and
566 KB a second. The docs give 200 connections per user and unnumbered
limits (errors 26 and 27).
- **Keepalive:** Kalshi sends a Ping frame (`heartbeat`) to a connection it
has sent nothing else for 10 s; a busy connection gets none. tungstenite
answers it by itself.
- **An unopened market can be subscribed:** its book is empty with `clock: 0`
and `market_id: ""`, and its changes flow from its first order after the
opening, with no new subscribe.
- **Time:** changes arrived 88 to 174 ms after their `ts_ms` (median 90),
from the server.
- **Listing:** `GET /markets?status=open&mve_filter=exclude&limit=1000`,
paged by `cursor`: 177,966 open markets on production, about 4 minutes
unauthenticated. `status=open` returns `active` markets, `unopened`
`initialized` ones, `closed` also `determined` ones. `min_created_ts`
(with no status: open and unopened alike), `status=closed&min_close_ts`
and `tickers=a,b` all work without a key. Markets open at their
`open_time` (none of the demo's 20,504 unopened ones was overdue), often
long after creation: of 15,000 open markets, 82% opened more than a minute
after, 19% more than 15 minutes.
- **Commands on an open subscription** (tried 2026-11-11): `skip_ticker_ack`
works on `update_subscription` too (the `ok` comes with an empty `msg`);
a subscription emptied by `delete_markets` stays and takes markets again,
with their books; subscribing a market already subscribed answers `ok` and
sends no book, so a fresh book takes `delete_markets` first; deleting an
unknown ticker answers `ok`.
- **A market's close sends its subscribers a fresh book**, unasked and
without the changes that lead to it (20-minute demo run, 2026-11-11: 54
markets closing at 20:00 UTC, each a newer, different book at 20:00:01 to
20:00:04). Taken as a book past ours that differs, it is logged as a loss;
the Snapshot is right.
- **A closed market answers like an unopened one**: an empty book, `clock:
0`, `market_id: ""`.
- **Handshakes are rate limited**: opening 110 connections at once, 54
handshakes were refused with 429; each got in on its retry a second later.
- **Production lists unopened markets past their `open_time`** (seen
2026-11-11: `initialized` 16 minutes after it); the demo had none.
- **The demo's size:** 246,365 markets open or opening within two hours
(2026-11-11, 19:29 UTC).
- **Maintenance (docs):** Thursdays 3:00 to 5:00 AM ET; connections drop.
## Clock accuracy bounds time accuracy
`received_at` comes from the local clock, so it carries the clock's error. On
2026-11-04 the dev machine ran ~30 ms slow (`sntp`: +30.6 ms ± 13 ms), which