feat(polymarket)!: two connections per market, first one wins

Every live market is subscribed on two connections chosen at random, and its
book takes each change from whichever connection delivers it first: the book
hash says whether a change is a duplicate, the next one, or one to hold until
the change before it arrives. A change one connection misses costs nothing,
each Delta carries the earliest receive time, and a connection that drops is
covered by the other. When a change fits on no connection in time, the book
follows one connection unchecked and asks the other for a fresh book.
Connections now decode and route events to 16 book threads, independent of
them; a move leaves its old connection only once the new one delivers, so it
costs no change; a connection closed with 1013 twice within ten minutes is
split. reconcile reads the collector's log and fails only on a mismatch no
cause line explains. SIGTERM and SIGHUP stop the collector cleanly.

Breaking: lanes are gone (connections and book threads are separate), and
reconcile takes the log as a second argument.
This commit is contained in:
pascal committed 2026-11-11 15:18:48 +00:00
1 parent 8114a51dc2
commit 02747b882f
24 files changed
+8984 -1214

No files matched your search

+37 -2
View File
@@ -1,10 +1,25 @@
//! Polymarket: which markets are live (Gamma), the orchestrator that spreads
//! them over lanes, and each lane's connection and feed (roadmap/03).
//! Polymarket: which markets are live (Gamma), the orchestrator that places
//! them on connections, the connections, and the book threads they feed
//! (docs/adr/0003-first-one-wins.md).
pub mod feed;
pub mod gamma;
pub mod orchestrator;
pub mod socket;
pub mod wire;
/// How many book threads own the books. One handled ~30,000 frames a second
/// in sprint 02.
pub const BOOK_THREADS: usize = 16;
/// The book thread that owns a market's books, from its `conditionId` (`0x`
/// and 64 hex digits, a hash: any 16 of them spread evenly). Computed, so the
/// orchestrator and every connection agree without a shared table.
pub fn book_thread(condition_id: &str) -> usize {
let digits = condition_id.get(2..18).unwrap_or_default();
// An id of another shape still gets a thread, always the same one.
u64::from_str_radix(digits, 16).unwrap_or(0) as usize % BOOK_THREADS
}
/// A live market: its symbols and the two inputs of their book hash that the
/// websocket doesn't carry (docs/notes.md).
@@ -23,3 +38,23 @@ pub struct Market {
pub(crate) fn fixture_markets() -> Vec<Market> {
gamma::parse_markets(include_str!("../../tests/fixtures/gamma-markets.json"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_market_always_gets_the_same_book_thread_and_markets_spread() {
let markets = fixture_markets();
let threads: std::collections::BTreeSet<usize> = markets
.iter()
.map(|market| book_thread(&market.condition_id))
.collect();
assert!(threads.iter().all(|&thread| thread < BOOK_THREADS));
// 20 markets over 16 threads: several threads, not one.
assert!(threads.len() > 5, "{threads:?}");
let id = &markets[0].condition_id;
assert_eq!(book_thread(id), book_thread(&id.clone()));
assert_eq!(book_thread("not an id"), 0);
}
}