feat: ready to submit, a run to watch, the README, the review's fixes

Sprint 08 (roadmap/08. ready to submit.md), the last milestone.

- scripts/watch.sh runs the collector with a live view of its log, stops
  cleanly on Ctrl-C, then checks the output (reconcile) and shows one
  symbol's book as text (show, a new binary).
- The README for the solution, with the final runs: Polymarket through an
  outage it recovered from on its own, Kalshi's demo with two slow
  connections recycled.
- From a review of the whole codebase: Kalshi takes RSA keys too
  (RSA-PSS-SHA256); a slow Kalshi connection takes the hourly recycle's
  next turn, which it nearly never got before; a market tracked again no
  longer fails reconcile; Book::add checks for overflow; usage and --help
  name the key variables; the first scan says how far it got.
- Tests for every gap the reviews found, an offline run of the whole
  orchestrator among them; ShellCheck in CI.
This commit is contained in:
pascal committed 2026-11-12 22:32:39 +01:00
1 parent 515e85941d
commit cf6c16c9cf
35 files changed
+2791 -207

No files matched your search

+13 -5
View File
@@ -186,8 +186,9 @@ as the median lag), but at the median it wins 9 ms, a tenth of the lag.
- **On Kalshi's demo it is worth tens of milliseconds**, in the tail too
(p99 28% and 42% lower with the fastest pair kept). This met the
threshold agreed before the runs, so the collector now recycles a Kalshi
connection that stays slow for 3 minutes running, ahead of its hourly
turn: its replacement's speed is drawn afresh. Each duplicate tells how
connection that stays slow for 3 minutes running at the hourly
recycle's next turn (one every 30 s or so), in place of the oldest: its
replacement's speed is drawn afresh. Each duplicate tells how
late its connection was against the other's copy (`src/kalshi/feed.rs`);
once a minute the orchestrator ranks the connections by their mean
lateness, and one is slow if it is in the slowest quarter and more than
@@ -197,9 +198,16 @@ as the median lag), but at the median it wins 9 ms, a tenth of the lag.
quarter always exists: a first version, by median alone, recycled 6
connections in 20 minutes, one a minute, connections no slower than the
rest. **In 20 minutes of the collector on the demo (270,000 markets on
120 connections, load 5.7 at the start), no connection was slow by that
rule**, so none was recycled for it; the hourly recycle went on (33), and
the output reconciled (25 mismatches, each a logged loss). The probe's 16
120 connections, load 5.7 at the start), none was recycled for it**; the
hourly recycle went on (33), and the output reconciled (25 mismatches,
each a logged loss). That run doesn't say whether one was judged slow: a
later review found the slow recycle waited for the same moment as the
hourly one, which nearly always took it first, and only the recycle was
logged. Now the slow one takes the hourly recycle's turn, and the
judgment is logged on its own. **In 20 minutes on the demo after that
fix** (260,865 markets on 116 connections, 2026-11-12), two connections
were judged slow and recycled: one at a mean lateness of 246 ms, one at
80 ms. The probe's 16
connections, on the 100 busiest markets, differed far more than the
collector's; assumed to be because each of the collector's shares its
markets with many others and is judged against all their copies, which