This repository has been archived on 2026-11-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
orderbook-collector/docs/notes.md
T
pascal 0e687e7eba feat(polymarket): skeleton and market-channel connectivity spike
Replaces the template's demo main.rs with a spike that connects to
Polymarket's public market channel, subscribes to one market, and logs every
frame with its receive time. Adds scripts/ci.sh (fmt, clippy, tests), the
project CLAUDE.md, ADR 0001 (OS threads, shared-nothing), and the market
channel's message shapes in docs/notes.md: no sequence number, a book hash on
every change, mirrored Yes/No books.
2026-11-04 14:47:14 +01:00

77 lines
4.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Challenge notes
## Deltas are absolute, not incremental
A delta carries the **new total quantity** at a price level, not a change to it.
`quantity: 400` means "this level is now 400," not "+400". Overwrite the level;
never add. `quantity: 0` means delete the level. (Schema confirms: "absolute
available quantity".)
## A missed message means a wrong book: rebuild it
If a message is lost, the in-memory book is silently wrong and cannot be
patched — throw it away and re-snapshot that symbol. This is the
"completeness" requirement made mechanical. *How* a loss is detected depends on
the exchange: a sequence-number gap where one exists (Kalshi, to verify in M6),
something else where it doesn't — Polymarket has no sequence number (below).
The challenge brief prescribes no mechanism; it only asks for complete data.
## Polymarket market channel (observed 2026-11-04)
From a 60 s run of the M1 spike on one market, both tokens subscribed, and the
[market channel docs](https://docs.polymarket.com/market-data/websocket/market-channel.md)
(read 2026-11-04).
- **Endpoint:** `wss://ws-subscriptions-clob.polymarket.com/ws/market`, public,
no auth. Subscribe: `{"type":"market","assets_ids":["<token id>", ...]}`.
On an open connection, `{"assets_ids":[...],"operation":"subscribe"}` (or
`"unsubscribe"`) changes the token set without reconnecting.
- **`custom_feature_enabled: true`** in the subscribe message adds
`best_bid_ask`, `new_market` and `market_resolved` events — candidate signals
for symbol discovery (M3), possibly replacing REST polling.
- **Symbol = token.** A market (`market`, a `0x…` condition id) has two tokens
(`asset_id`: Yes, No), each with its own book. The two books are **exact
mirrors**: Yes had only bids (best 0.999), No only asks (best 0.001), and
every change arrives as a pair in one message — Yes `BUY p` with No `SELL 1−p`,
same size.
- **Frames:** on subscribe, one frame holding a JSON **array** of `book`
messages (one per token); afterwards, one **object** per frame. A frame's text
can end with a newline.
- **Full books also arrive mid-stream.** After a trade the server sent, all with
one timestamp: the `price_change`, then a full `book` object for each token,
then a `last_trade_price` — a natural resync point.
- **`book`** (snapshot): `event_type`, `market`, `asset_id`, `timestamp`,
`hash`, `bids`, `asks`, `tick_size`, `last_trade_price`. `bids`/`asks` are lists
of `{price, size}` sorted **ascending by price**, so the best bid is the
*last* bid. Its `timestamp` is the time of the book's **last change**, not the
send time: 19 s old on receipt.
- **`price_change`** (delta): `event_type`, `market`, `timestamp`,
`price_changes[]`, each entry `{asset_id, price, size, side, hash, best_bid,
best_ask}`. `side` is `BUY` (bid) or `SELL` (ask); `size` is the absolute
level size (docs: `"0"` removes the level).
- **Numbers are decimal strings** (`"0.77"`, `"26350.28"`): parse them exactly
into fixed point (`1_000_000` = 1), never through a float.
- **Timestamps are milliseconds** since the epoch (the schema stores
microseconds).
- **No sequence number**, confirmed by the docs: none, and no option enables
one, for the market channel (the `seq` in Polymarket's docs belongs to
PolyBolt, a separate price feed). The docs describe no gap detection or
resync either. Candidates for loss detection, to settle in M2:
- the `hash` (40 hex chars, SHA-1 length) on every book and every change. A
change's `hash` equals that of the `book` sent right after it for the same
token, so it identifies the **full book state after the change**: a full
check if we can reproduce it from our own book. The docs don't say how it is
computed (M2's opening question);
- `best_bid`/`best_ask` on every change — a cheap top-of-book check after
applying it.
- **Keepalive (docs):** send the text frame `PING` every 10 s; the server
answers `PONG`. The spike survived 60 s without it, but it is required.
- **`last_trade_price`:** `asset_id`, `price`, `size`, `side`, `fee_rate_bps`,
`timestamp`, `transaction_hash` (one per trade, for one token).
- **Not observed:** `tick_size_change` and the custom-feature events.
## Clock accuracy bounds time accuracy
`received_at` comes from the local clock, so it carries the clock's error. On
2026-11-04 the dev machine ran ~30 ms slow (`sntp`: +30.6 ms ± 13 ms), which
made raw `received_at − exchange timestamp` read −14 to +41 ms; corrected, the
real lag was ~+16 to +71 ms. Latency figures need a synced clock, or the offset
measured and reported alongside them.