feat(polymarket)!: two connections per market, first one wins

Every live market is subscribed on two connections chosen at random, and its
book takes each change from whichever connection delivers it first: the book
hash says whether a change is a duplicate, the next one, or one to hold until
the change before it arrives. A change one connection misses costs nothing,
each Delta carries the earliest receive time, and a connection that drops is
covered by the other. When a change fits on no connection in time, the book
follows one connection unchecked and asks the other for a fresh book.
Connections now decode and route events to 16 book threads, independent of
them; a move leaves its old connection only once the new one delivers, so it
costs no change; a connection closed with 1013 twice within ten minutes is
split. reconcile reads the collector's log and fails only on a mismatch no
cause line explains. SIGTERM and SIGHUP stop the collector cleanly.

Breaking: lanes are gone (connections and book threads are separate), and
reconcile takes the log as a second argument.
This commit is contained in:
pascal committed 2026-11-11 15:18:48 +00:00
1 parent 8114a51dc2
commit 02747b882f
24 files changed
+8984 -1214

No files matched your search

+80
View File
@@ -0,0 +1,80 @@
//! Why a Snapshot outside the minute was written: one log line per Snapshot,
//! written by the collector and read back by `reconcile`, which pairs it with
//! a mismatch by symbol and receive time.
use std::fmt;
use chrono::{DateTime, Utc};
/// What a Snapshot that may differ from the book before it stands for.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Cause {
/// A fresh book differed from ours: changes no connection delivered.
Loss,
/// A fresh book differed from ours before any change of the symbol was
/// verified: changes our own subscribe kept from us.
SubscriptionGap,
/// The connection followed changed: the new one's book, which may hold
/// changes the old one never delivered.
Switch,
}
impl Cause {
const ALL: [Cause; 3] = [Cause::Loss, Cause::SubscriptionGap, Cause::Switch];
fn name(self) -> &'static str {
match self {
Cause::Loss => "loss",
Cause::SubscriptionGap => "subscription gap",
Cause::Switch => "switch",
}
}
}
impl fmt::Display for Cause {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.write_str(self.name())
}
}
const MARK: &str = "Snapshot of ";
/// The line's text: `Snapshot of <symbol> at <receive time, µs>: <cause>`,
/// then any detail for a reader.
pub fn line(symbol: &str, received_at: DateTime<Utc>, cause: Cause, detail: &str) -> String {
let at = received_at.timestamp_micros();
format!("{MARK}{symbol} at {at}: {cause}{detail}")
}
/// The symbol, receive time and cause of a log line written from [`line`],
/// wherever it sits in the line.
pub fn parse(log_line: &str) -> Option<(String, DateTime<Utc>, Cause)> {
let rest = &log_line[log_line.find(MARK)? + MARK.len()..];
let (symbol, rest) = rest.split_once(" at ")?;
let (at, rest) = rest.split_once(": ")?;
let received_at = DateTime::from_timestamp_micros(at.parse().ok()?)?;
let cause = Cause::ALL
.into_iter()
.find(|cause| rest.starts_with(cause.name()))?;
Some((symbol.to_string(), received_at, cause))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_line_reads_back() {
let at = DateTime::from_timestamp_micros(1_794_390_726_365_123).unwrap();
for cause in Cause::ALL {
let text = line("123", at, cause, " (its time 12:00:00.000)");
let logged = format!("12:00:00.001 polymarket: {text}");
assert_eq!(parse(&logged), Some(("123".to_string(), at, cause)));
}
assert_eq!(
parse("12:00:00.001 polymarket: connection 3: connected"),
None
);
assert_eq!(parse("Snapshot of 1 at soon: loss"), None);
}
}