This repository has been archived on 2026-11-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
orderbook-collector/docs/notes.md
T
pascal 0e687e7eba feat(polymarket): skeleton and market-channel connectivity spike
Replaces the template's demo main.rs with a spike that connects to
Polymarket's public market channel, subscribes to one market, and logs every
frame with its receive time. Adds scripts/ci.sh (fmt, clippy, tests), the
project CLAUDE.md, ADR 0001 (OS threads, shared-nothing), and the market
channel's message shapes in docs/notes.md: no sequence number, a book hash on
every change, mirrored Yes/No books.
2026-11-04 14:47:14 +01:00

4.5 KiB
Raw Blame History

Challenge notes

Deltas are absolute, not incremental

A delta carries the new total quantity at a price level, not a change to it. quantity: 400 means "this level is now 400," not "+400". Overwrite the level; never add. quantity: 0 means delete the level. (Schema confirms: "absolute available quantity".)

A missed message means a wrong book: rebuild it

If a message is lost, the in-memory book is silently wrong and cannot be patched — throw it away and re-snapshot that symbol. This is the "completeness" requirement made mechanical. How a loss is detected depends on the exchange: a sequence-number gap where one exists (Kalshi, to verify in M6), something else where it doesn't — Polymarket has no sequence number (below). The challenge brief prescribes no mechanism; it only asks for complete data.

Polymarket market channel (observed 2026-11-04)

From a 60 s run of the M1 spike on one market, both tokens subscribed, and the market channel docs (read 2026-11-04).

  • Endpoint: wss://ws-subscriptions-clob.polymarket.com/ws/market, public, no auth. Subscribe: {"type":"market","assets_ids":["<token id>", ...]}. On an open connection, {"assets_ids":[...],"operation":"subscribe"} (or "unsubscribe") changes the token set without reconnecting.
  • custom_feature_enabled: true in the subscribe message adds best_bid_ask, new_market and market_resolved events — candidate signals for symbol discovery (M3), possibly replacing REST polling.
  • Symbol = token. A market (market, a 0x… condition id) has two tokens (asset_id: Yes, No), each with its own book. The two books are exact mirrors: Yes had only bids (best 0.999), No only asks (best 0.001), and every change arrives as a pair in one message — Yes BUY p with No SELL 1−p, same size.
  • Frames: on subscribe, one frame holding a JSON array of book messages (one per token); afterwards, one object per frame. A frame's text can end with a newline.
  • Full books also arrive mid-stream. After a trade the server sent, all with one timestamp: the price_change, then a full book object for each token, then a last_trade_price — a natural resync point.
  • book (snapshot): event_type, market, asset_id, timestamp, hash, bids, asks, tick_size, last_trade_price. bids/asks are lists of {price, size} sorted ascending by price, so the best bid is the last bid. Its timestamp is the time of the book's last change, not the send time: 19 s old on receipt.
  • price_change (delta): event_type, market, timestamp, price_changes[], each entry {asset_id, price, size, side, hash, best_bid, best_ask}. side is BUY (bid) or SELL (ask); size is the absolute level size (docs: "0" removes the level).
  • Numbers are decimal strings ("0.77", "26350.28"): parse them exactly into fixed point (1_000_000 = 1), never through a float.
  • Timestamps are milliseconds since the epoch (the schema stores microseconds).
  • No sequence number, confirmed by the docs: none, and no option enables one, for the market channel (the seq in Polymarket's docs belongs to PolyBolt, a separate price feed). The docs describe no gap detection or resync either. Candidates for loss detection, to settle in M2:
    • the hash (40 hex chars, SHA-1 length) on every book and every change. A change's hash equals that of the book sent right after it for the same token, so it identifies the full book state after the change: a full check if we can reproduce it from our own book. The docs don't say how it is computed (M2's opening question);
    • best_bid/best_ask on every change — a cheap top-of-book check after applying it.
  • Keepalive (docs): send the text frame PING every 10 s; the server answers PONG. The spike survived 60 s without it, but it is required.
  • last_trade_price: asset_id, price, size, side, fee_rate_bps, timestamp, transaction_hash (one per trade, for one token).
  • Not observed: tick_size_change and the custom-feature events.

Clock accuracy bounds time accuracy

received_at comes from the local clock, so it carries the clock's error. On 2026-11-04 the dev machine ran ~30 ms slow (sntp: +30.6 ms ± 13 ms), which made raw received_at − exchange timestamp read −14 to +41 ms; corrected, the real lag was ~+16 to +71 ms. Latency figures need a synced clock, or the offset measured and reported alongside them.