Commit Graph
2 Commits
Author SHA1 Message Date
pascal 8114a51dc2 feat(polymarket): track every live market over load-balanced lanes
An orchestrator keeps the tracked set equal to Gamma's live markets (a scan at
startup and every hour, new and closed markets every minute, reaching back 15
minutes for Gamma's indexing delay) and spreads them over lanes, each a
connection and a book thread, all writing through the one writer. Lanes report
each market's load in bytes per second; the orchestrator puts new markets on
the least-loaded lane and moves the busiest markets off lanes over a 300 KB/s
budget, busiest lane first. Differing books our own subscribe explains are
subscription gaps, not losses, and every first book after a move is logged,
so reconcile's mismatches are all accounted for in the log. A 75-minute live
run tracked ~195k markets (~390k symbols) in 89 lanes.
2026-11-11 06:28:25 +00:00
pascal adb7fee8d8 feat(polymarket): hash-verified order books for a subset of symbols
Collects the order books of Polymarket's 20 busiest markets (40 symbols)
into the template's SBE format: a Delta per level change, plus Snapshots on
subscribe, every minute, and whenever a fresh book corrects ours. Every
change is checked against Polymarket's own book hash; a failed check
resyncs the symbol on the open socket, and the log counts losses apart from
the few ms our own resync hides. Runs until SIGINT, which closes the socket
and flushes the output; reconnects when the server drops it. A `reconcile`
binary checks an output file against itself.
2026-11-04 17:10:17 +01:00