Archived
feat(polymarket)!: two connections per market, first one wins
Every live market is subscribed on two connections chosen at random, and its book takes each change from whichever connection delivers it first: the book hash says whether a change is a duplicate, the next one, or one to hold until the change before it arrives. A change one connection misses costs nothing, each Delta carries the earliest receive time, and a connection that drops is covered by the other. When a change fits on no connection in time, the book follows one connection unchecked and asks the other for a fresh book. Connections now decode and route events to 16 book threads, independent of them; a move leaves its old connection only once the new one delivers, so it costs no change; a connection closed with 1013 twice within ten minutes is split. reconcile reads the collector's log and fails only on a mismatch no cause line explains. SIGTERM and SIGHUP stop the collector cleanly. Breaking: lanes are gone (connections and book threads are separate), and reconcile takes the log as a second argument.
This commit is contained in:
24 files changed
+8984
-1214
No files matched your search
@@ -0,0 +1,80 @@
|
||||
//! Why a Snapshot outside the minute was written: one log line per Snapshot,
|
||||
//! written by the collector and read back by `reconcile`, which pairs it with
|
||||
//! a mismatch by symbol and receive time.
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
/// What a Snapshot that may differ from the book before it stands for.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum Cause {
|
||||
/// A fresh book differed from ours: changes no connection delivered.
|
||||
Loss,
|
||||
/// A fresh book differed from ours before any change of the symbol was
|
||||
/// verified: changes our own subscribe kept from us.
|
||||
SubscriptionGap,
|
||||
/// The connection followed changed: the new one's book, which may hold
|
||||
/// changes the old one never delivered.
|
||||
Switch,
|
||||
}
|
||||
|
||||
impl Cause {
|
||||
const ALL: [Cause; 3] = [Cause::Loss, Cause::SubscriptionGap, Cause::Switch];
|
||||
|
||||
fn name(self) -> &'static str {
|
||||
match self {
|
||||
Cause::Loss => "loss",
|
||||
Cause::SubscriptionGap => "subscription gap",
|
||||
Cause::Switch => "switch",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Display for Cause {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
f.write_str(self.name())
|
||||
}
|
||||
}
|
||||
|
||||
const MARK: &str = "Snapshot of ";
|
||||
|
||||
/// The line's text: `Snapshot of <symbol> at <receive time, µs>: <cause>`,
|
||||
/// then any detail for a reader.
|
||||
pub fn line(symbol: &str, received_at: DateTime<Utc>, cause: Cause, detail: &str) -> String {
|
||||
let at = received_at.timestamp_micros();
|
||||
format!("{MARK}{symbol} at {at}: {cause}{detail}")
|
||||
}
|
||||
|
||||
/// The symbol, receive time and cause of a log line written from [`line`],
|
||||
/// wherever it sits in the line.
|
||||
pub fn parse(log_line: &str) -> Option<(String, DateTime<Utc>, Cause)> {
|
||||
let rest = &log_line[log_line.find(MARK)? + MARK.len()..];
|
||||
let (symbol, rest) = rest.split_once(" at ")?;
|
||||
let (at, rest) = rest.split_once(": ")?;
|
||||
let received_at = DateTime::from_timestamp_micros(at.parse().ok()?)?;
|
||||
let cause = Cause::ALL
|
||||
.into_iter()
|
||||
.find(|cause| rest.starts_with(cause.name()))?;
|
||||
Some((symbol.to_string(), received_at, cause))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn a_line_reads_back() {
|
||||
let at = DateTime::from_timestamp_micros(1_794_390_726_365_123).unwrap();
|
||||
for cause in Cause::ALL {
|
||||
let text = line("123", at, cause, " (its time 12:00:00.000)");
|
||||
let logged = format!("12:00:00.001 polymarket: {text}");
|
||||
assert_eq!(parse(&logged), Some(("123".to_string(), at, cause)));
|
||||
}
|
||||
assert_eq!(
|
||||
parse("12:00:00.001 polymarket: connection 3: connected"),
|
||||
None
|
||||
);
|
||||
assert_eq!(parse("Snapshot of 1 at soon: loss"), None);
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user