feat(polymarket): subscription monitoring and connection recycling

The exchange can fail without telling us, and first one wins hides a
subscription gone quiet on one of a symbol's two connections.

- A connection that receives nothing for 30 s, not even the keepalive's
  answer, reconnects.
- Each book thread counts a symbol's changes per connection every minute:
  a subscription with none against three or more on the other is
  resubscribed, and moved to another connection if it stays quiet.
- The orchestrator recycles one connection at a time, each about hourly,
  and never while another connection is down or just back.
- A reconnect's opening book of a symbol unchanged for 10 s is judged like
  a resync's answer: one newer than ours that differs shows a symbol quiet
  on both connections, written as a Snapshot and logged as a loss.
This commit is contained in:
pascal committed 2026-11-11 17:42:27 +00:00
1 parent 02747b882f
commit 480ee3ad2d
10 files changed
+1346 -102

No files matched your search

+10 -8
View File
@@ -135,14 +135,15 @@ and drop from the global workflow for a job-challenge repo.
## Pipeline
The primary thing we debate sprint by sprint (this project's equivalent of the
schema debate). As settled in sprint 04:
schema debate). As settled in sprints 04 and 05:
```
rescan fetcher ──findings──▶ ┌──────────────┐ ◀──findings── recent fetcher
│ orchestrator │ ◀── 1013 closes ── connections
└──────┬───▲───┘
│ orchestrator │ ◀── up, down, 1013 ─── connections
└──────┬───▲───┘ ──── recycle ────────▶ connections
Add / Move / Remove (a market and │ │ load reports, each minute
its connections), to its thread │ │ (bytes/s per market)
its connections), to its thread │ │ (bytes/s per market); markets
│ │ quiet twice on a connection
┌──────▼───┴───┐
connection 0..N ──events───▶ │ book threads │ ──Snapshot | Delta──▶ writer
(read, stamp, parse, route) │ 0..15 │
@@ -151,15 +152,16 @@ schema debate). As settled in sprint 04:
```
- The **orchestrator** owns what is tracked (each market's two connections,
the timers) and decides: placing, moving, splitting. It never blocks on
HTTP.
the timers) and decides: placing, moving, splitting, and recycling one
connection at a time, about hourly. It never blocks on HTTP.
- The two **fetchers** only page Gamma on request and stream what they read.
- A **connection** reads, stamps the receive time, parses, and routes each
event to the book thread of its market. Every symbol is on two connections,
chosen independently.
chosen independently. One silent for 30 s reconnects.
- A **book thread** owns its markets' books and applies first one wins: a
change is a duplicate, the next one, or held until it fits (fold, check,
resync, Snapshots as before).
resync, Snapshots as before). It resubscribes a subscription gone quiet on
one connection, and judges a reconnect's books of unchanged symbols.
- One **writer** owns the output.
Settled: OS threads, shared-nothing, no mutexes