Archived
feat!: a generic core, and Kalshi through it
The orchestrator, the connection loop and the book thread's shell move to src/core/, generic over an Exchange and its loss rule (ADR 0004). Polymarket runs through it unchanged. Kalshi joins: a signed handshake (KALSHI_KEY_ID_FILE, KALSHI_PRIVATE_KEY_FILE), a reconnect on a skipped sequence number, first one wins by clock and rank, and its market listing with markets tracked from three hours before they open. Breaking: the exchange is now the first argument, `engineering-challenge <polymarket|kalshi|kalshi-demo> [OUTPUT]`.
This commit is contained in:
33 files changed
+8150
-1846
No files matched your search
+95
-4
@@ -4,15 +4,17 @@
|
||||
A delta carries the **new total quantity** at a price level, not a change to it.
|
||||
`quantity: 400` means "this level is now 400," not "+400". Overwrite the level;
|
||||
never add. `quantity: 0` means delete the level. (Schema confirms: "absolute
|
||||
available quantity".)
|
||||
available quantity".) That is the output, and Polymarket's wire. Kalshi's wire
|
||||
is relative (`delta_fp`: "+400"): the book adds it, and the Delta written
|
||||
still carries the level's new total.
|
||||
|
||||
## A missed message means a wrong book: resync it
|
||||
If a message is lost, the in-memory book is silently wrong and cannot be
|
||||
patched — throw it away and resync that symbol from a fresh full book. This is the
|
||||
"completeness" requirement made mechanical. *How* a loss is detected depends on
|
||||
the exchange: a sequence-number gap where one exists (Kalshi, to verify in M6),
|
||||
something else where it doesn't — Polymarket has no sequence number, but its
|
||||
book hash does the job (below).
|
||||
the exchange: a sequence-number gap where one exists (Kalshi: per connection,
|
||||
below), something else where it doesn't — Polymarket has no sequence number,
|
||||
but its book hash does the job (below).
|
||||
The challenge brief prescribes no mechanism; it only asks for complete data.
|
||||
|
||||
## Polymarket market channel (observed 2026-11-04)
|
||||
@@ -222,6 +224,95 @@ connection recycled every 60 s (sprint 05).
|
||||
20 s): a check reaching back 5 minutes misses some. 195,130 markets were
|
||||
live that evening.
|
||||
|
||||
## Kalshi's websocket (demo, observed 2026-11-11)
|
||||
|
||||
From Kalshi's docs and four recordings of the demo with a throwaway recorder
|
||||
(`../kalshi-spike/`, outside the repo). Production is untried where a number
|
||||
is the demo's.
|
||||
|
||||
- **Hosts:** production `wss://external-api-ws.kalshi.com/trade-api/ws/v2`
|
||||
and `https://external-api.kalshi.com/trade-api/v2`; demo
|
||||
`wss://external-api-ws.demo.kalshi.co/trade-api/ws/v2` and
|
||||
`https://external-api.demo.kalshi.co/trade-api/v2`. The older
|
||||
`api.elections.kalshi.com` and `demo-api.kalshi.co` still answer.
|
||||
- **Auth:** the websocket handshake carries `KALSHI-ACCESS-KEY` (the key id),
|
||||
`KALSHI-ACCESS-TIMESTAMP` (ms) and `KALSHI-ACCESS-SIGNATURE`: base64 of the
|
||||
Ed25519 signature of `<timestamp>GET/trade-api/ws/v2`. Without them: 401.
|
||||
The market listing needs no key. Keys are created per environment
|
||||
(Account → Profile → API Keys); the private key is a PKCS#8 PEM shown once.
|
||||
- **Subscribe:** `{"id":1,"cmd":"subscribe","params":{"channels":
|
||||
["orderbook_delta"],"market_tickers":[...]}}` is answered
|
||||
`{"type":"subscribed","id":1,"msg":{"channel":"orderbook_delta","sid":1}}`,
|
||||
then one `orderbook_snapshot` per market. A connection has one such
|
||||
subscription: a second `subscribe`, or `{"cmd":"update_subscription",
|
||||
"params":{"sids":[1],"market_tickers":[...],"action":"add_markets"}}`, adds
|
||||
markets and sends their books; `delete_markets` stops a market's changes;
|
||||
`get_snapshot` sends books again (carrying the command's `id`). Each `ok`
|
||||
answer lists every market of the subscription unless `skip_ticker_ack:
|
||||
true` is sent. An empty ticker list is error 14.
|
||||
- **`orderbook_snapshot`:** `{"type","sid","seq","sending_ts_ms","msg":
|
||||
{"market_ticker","market_id","clock","yes_dollars_fp":[["0.4800","80.00"],
|
||||
...],"no_dollars_fp":[...]}}`. A side with no level is omitted. Bids only:
|
||||
YES bids and NO bids. No time of its own.
|
||||
- **`orderbook_delta`:** `msg` is `{"market_ticker","market_id","clock",
|
||||
"price_dollars":"0.9600","delta_fp":"-54.00","side":"yes"|"no","ts",
|
||||
"ts_ms"}`: a signed change of the level's quantity. Folding them gave the
|
||||
exchange's book every time (1,352 `get_snapshot` books equal to ours, no
|
||||
level below zero).
|
||||
- **`use_yes_price: true`** in the subscribe gives NO levels at `1 − q`, the
|
||||
YES ask they are (books and changes alike); the default, `false`, gives
|
||||
`q` and is announced to flip.
|
||||
- **`seq`** is contiguous per connection across all its markets, counting
|
||||
books, changes, `ok` and `error` (not `subscribed`): no gap in 343,000
|
||||
messages.
|
||||
- **`clock`** is per market, the same on every connection, never going back,
|
||||
and shared by 2 to 4 changes for 12% of its values (some identical in
|
||||
content). A book's clock is its last change's: none of 1,378 books was
|
||||
followed by a change of its own clock or an older one. Two connections
|
||||
deliver a market's changes in the same order (2,643 changes on 26 markets),
|
||||
the second copy 2 ms later at the median, 344 ms at worst.
|
||||
- **No limit met per connection:** 30,000 markets in one subscribe message,
|
||||
or in 30 messages 20 ms apart; every book within 1.7 s; ~1,560 changes and
|
||||
566 KB a second. The docs give 200 connections per user and unnumbered
|
||||
limits (errors 26 and 27).
|
||||
- **Keepalive:** Kalshi sends a Ping frame (`heartbeat`) to a connection it
|
||||
has sent nothing else for 10 s; a busy connection gets none. tungstenite
|
||||
answers it by itself.
|
||||
- **An unopened market can be subscribed:** its book is empty with `clock: 0`
|
||||
and `market_id: ""`, and its changes flow from its first order after the
|
||||
opening, with no new subscribe.
|
||||
- **Time:** changes arrived 88 to 174 ms after their `ts_ms` (median 90),
|
||||
from the server.
|
||||
- **Listing:** `GET /markets?status=open&mve_filter=exclude&limit=1000`,
|
||||
paged by `cursor`: 177,966 open markets on production, about 4 minutes
|
||||
unauthenticated. `status=open` returns `active` markets, `unopened`
|
||||
`initialized` ones, `closed` also `determined` ones. `min_created_ts`
|
||||
(with no status: open and unopened alike), `status=closed&min_close_ts`
|
||||
and `tickers=a,b` all work without a key. Markets open at their
|
||||
`open_time` (none of the demo's 20,504 unopened ones was overdue), often
|
||||
long after creation: of 15,000 open markets, 82% opened more than a minute
|
||||
after, 19% more than 15 minutes.
|
||||
- **Commands on an open subscription** (tried 2026-11-11): `skip_ticker_ack`
|
||||
works on `update_subscription` too (the `ok` comes with an empty `msg`);
|
||||
a subscription emptied by `delete_markets` stays and takes markets again,
|
||||
with their books; subscribing a market already subscribed answers `ok` and
|
||||
sends no book, so a fresh book takes `delete_markets` first; deleting an
|
||||
unknown ticker answers `ok`.
|
||||
- **A market's close sends its subscribers a fresh book**, unasked and
|
||||
without the changes that lead to it (20-minute demo run, 2026-11-11: 54
|
||||
markets closing at 20:00 UTC, each a newer, different book at 20:00:01 to
|
||||
20:00:04). Taken as a book past ours that differs, it is logged as a loss;
|
||||
the Snapshot is right.
|
||||
- **A closed market answers like an unopened one**: an empty book, `clock:
|
||||
0`, `market_id: ""`.
|
||||
- **Handshakes are rate limited**: opening 110 connections at once, 54
|
||||
handshakes were refused with 429; each got in on its retry a second later.
|
||||
- **Production lists unopened markets past their `open_time`** (seen
|
||||
2026-11-11: `initialized` 16 minutes after it); the demo had none.
|
||||
- **The demo's size:** 246,365 markets open or opening within two hours
|
||||
(2026-11-11, 19:29 UTC).
|
||||
- **Maintenance (docs):** Thursdays 3:00 to 5:00 AM ET; connections drop.
|
||||
|
||||
## Clock accuracy bounds time accuracy
|
||||
`received_at` comes from the local clock, so it carries the clock's error. On
|
||||
2026-11-04 the dev machine ran ~30 ms slow (`sntp`: +30.6 ms ± 13 ms), which
|
||||
|
||||
Reference in new issue
Block a user