Archived
feat(polymarket): track every live market over load-balanced lanes
An orchestrator keeps the tracked set equal to Gamma's live markets (a scan at startup and every hour, new and closed markets every minute, reaching back 15 minutes for Gamma's indexing delay) and spreads them over lanes, each a connection and a book thread, all writing through the one writer. Lanes report each market's load in bytes per second; the orchestrator puts new markets on the least-loaded lane and moves the busiest markets off lanes over a 300 KB/s budget, busiest lane first. Differing books our own subscribe explains are subscription gaps, not losses, and every first book after a move is logged, so reconcile's mismatches are all accounted for in the log. A 75-minute live run tracked ~195k markets (~390k symbols) in 89 lanes.
This commit is contained in:
18 files changed
+2826
-320
No files matched your search
Vendored
+8
-2
@@ -1,10 +1,16 @@
|
||||
# Test fixtures
|
||||
|
||||
A recording of Polymarket's market channel, replayed by the tests in
|
||||
`src/polymarket/feed.rs` (no network in tests).
|
||||
`src/polymarket/feed.rs`, and Gamma pages read by `src/polymarket/gamma.rs`
|
||||
(no network in tests).
|
||||
|
||||
- `gamma-markets.json`: Gamma's top 20 markets by 24 h volume on 2026-11-04,
|
||||
trimmed to the fields the collector reads.
|
||||
trimmed to the fields the collector reads. `conditionId`, `active`, `closed`
|
||||
and `acceptingOrders` were added in sprint 03 (the ids from the recorded
|
||||
books' `market`).
|
||||
- `gamma-created.json`, `gamma-closed.json`: one `/markets/keyset` page each
|
||||
(`limit=10`) of the newest live markets by `createdAt` and the latest closed
|
||||
by `closedTime`, 2026-11-04 ~17:00 UTC, trimmed the same way.
|
||||
- `polymarket-frames.jsonl`: the first 90 s (14:24:44 to 14:26:14 UTC, so two
|
||||
minute boundaries) of the market channel for those markets' 40 tokens, one
|
||||
line per frame: `{"received_at_us": <receive time, µs>, "text": <frame>}`.
|
||||
|
||||
Reference in new issue
Block a user