This repository has been archived on 2026-11-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
orderbook-collector/roadmap/08. ready to submit.md
T
pascal cf6c16c9cf feat: ready to submit, a run to watch, the README, the review's fixes
Sprint 08 (roadmap/08. ready to submit.md), the last milestone.

- scripts/watch.sh runs the collector with a live view of its log, stops
  cleanly on Ctrl-C, then checks the output (reconcile) and shows one
  symbol's book as text (show, a new binary).
- The README for the solution, with the final runs: Polymarket through an
  outage it recovered from on its own, Kalshi's demo with two slow
  connections recycled.
- From a review of the whole codebase: Kalshi takes RSA keys too
  (RSA-PSS-SHA256); a slow Kalshi connection takes the hourly recycle's
  next turn, which it nearly never got before; a market tracked again no
  longer fails reconcile; Book::add checks for overflow; usage and --help
  name the key variables; the first scan says how far it got.
- Tests for every gap the reviews found, an offline run of the whole
  orchestrator among them; ShellCheck in CI.
2026-11-12 22:32:39 +01:00

22 KiB

08. Ready to submit

Status: done (2026-11-12)

M8, the last milestone: the collector handed to Voluntas in a state a reviewer can read, run and check on their own. A review of the whole codebase (not one sprint's diff) and its fixes; one command that runs the collector and shows what it does; a README for the solution in place of the template's; a final live run of each exchange, reconciled, whose figures the README quotes; and the handoff as a public repo.

The watched run

scripts/watch.sh [EXCHANGE] [MINUTES], defaults polymarket (no key) and 3 minutes (two minute summaries; Polymarket tracks about 200,000 markets within its first minute, ../runs/poly10.log). It builds release, runs the collector under timeout -s INT, and writes the output and the full log to a gitignored watch/. On the terminal, a curated view of the log:

  • each minute, the orchestrator's line as is, and the 16 book threads' lines summed into one (changes written, duplicates, rescued, resyncs, losses, subscription gaps; their counts are cumulative, "so far");
  • between them, the notable events only: a connection lost, recycled (hourly or slow), split, silent; the startup's connections as one count, not a line each. The Snapshot cause lines (3,300 of 3,952 lines in a 10-minute Polymarket run) stay in the file.

Ctrl-C stops the run early and cleanly: the script goes on to the checks (how Ctrl-C reaches timeout and the collector from a script is assumed; a quick try without a terminal didn't deliver the signal, so the first task tries it in one). Then it runs reconcile on the output and the log, and show on the output.

On the workspace: rc shell voluntas, then cd voluntas/repo && scripts/watch.sh (rc shell's interactive use is assumed from rc shell exec). Kalshi needs KALSHI_KEY_ID_FILE and KALSHI_PRIVATE_KEY_FILE exported; without them the collector stops before connecting, naming the variable (src/kalshi/sign.rs:26). The script adds nothing for keys.

show

src/bin/show.rs: show OUTPUT [SYMBOL], reading through the template's AlignedSbeReader as src/reconcile.rs does. With no symbol, the one with the most Deltas in the file. It prints that symbol's first Snapshot (top 5 levels a side, its receive time), its next 20 Deltas (receive time, side, price, new quantity; 0 shown as deleted), then the top 5 levels a side of the book folded from them, in exchange units through book::format_fixed (src/book.rs:46). A Polymarket symbol is its token id: the output holds no market name. An unknown symbol is an error.

The review

The whole codebase, not a diff: src/, scripts/, the tests, by five reviewer subagents:

  • brief, in place of a spec reviewer: the brief checked line by line against the code (every live symbol, MVEs excluded on Kalshi, new and gone symbols, Deltas to stdout and a Snapshot each minute, SIGINT closes the websockets and the file, each extension);
  • smells, read as a stranger: leftovers of superseded designs (lanes, the template's demo), comments that need sprint history (19 citations of roadmap/0N or "sprint 0N" in src/), the two exchanges inconsistent with each other;
  • bugs, tests, security, as usual; sprint 07's 11 post-review fixes, never reviewed a second time, included.

Findings are sorted with the user: fix now (a task below, with its test), or a limit or a "next" in the README.

The README

Replaces the template's, for the Voluntas reader, in this order: what the collector does, with the line on how it was built (below); quick start (scripts/watch.sh); build and run per exchange, Kalshi's two export lines, and that Kalshi ran only against the demo; checking the output (reconcile with the log); the final runs' figures; the design in one diagram (docs/project.md's pipeline); each extension and where it is met (ADRs, LATENCY.md); the limits (the crash ceiling, the subscription gap when a symbol opens, Kalshi production never run); next (a short pick of the roadmap's possible features); what to read first (docs/project.md, docs/adr/, LATENCY.md, docs/notes.md, and GLOSSARY.md, CLAUDE.md) and that roadmap/ holds each sprint's plan and outcome; an "Output format" section (the template's AlignedSbeWriter unchanged, 64 MiB chunks, the schema's path). The order as built supersedes the first one (the runs after the extensions, the line on how it was built last): the runs read as evidence for the design that follows, and the line belongs with what the project is. The line:

Built with Claude Code. I set the architecture (the threading model, redundancy, the exchange seam) and settled every design decision; Claude proposed the options, wrote the code and ran the experiments. Each decision and its reasons are in roadmap/ and docs/adr/.

Decisions

  • The watched run is a script over the existing log, not pipeline code. Rejected: a status line in the collector (counts across threads, new pipeline code); a TUI.
  • The data is shown, not only counted (show), after the run: the writer emits 64 MiB chunks, so a live stream of the output would come in bursts.
  • One script for everything: the final runs and the README's quick start are scripts/watch.sh, so the grader runs what made the README's figures. The workspace's ../runs/run.sh is retired.
  • Everything ships unchanged (docs/, roadmap/, CLAUDE.md, GLOSSARY.md, LATENCY.md); the README says what to read first. Rejected: stripping the process files; the work being AI-assisted is said openly instead.
  • Sprint citations in comments: a file path stays (roadmap/ ships); a reference that leans on history is rewritten to say what the code does, or to cite the ADR or docs/notes.md.
  • The final runs pass reconcile (every mismatch explained, no orphan Delta), load and peak memory recorded (VmHWM from /proc); no comparison with earlier sprints' runs (done in their sprints; the load differs from day to day).
  • The template's README text: its stale lines go ("read main.rs", "as demonstrated in main.rs"); the rest becomes "Output format".
  • Delivery: a public repo on the user's Gitea (git.pband.ch), not on GitHub. The brief says nothing on how (it links only the template, public, 25 forks). Public, so the link opens with no account; on GitHub it would publish a solution to the challenge where other candidates search, which the user doesn't want. Rejected: a GitHub fork (listed under the template), a GitHub repo of our own (found by search), a private repo (an invite to a handle not known), a PR to the template (public). Full history, the template's 3 commits first, so a diff against it shows what was added. Pushed over SSH (which reaches Gitea from here), with the user's go. Supersedes "a public repo of the user's own on GitHub", at the user's request during the build.
  • Kalshi's RSA keys are taken (from the review): RSA-PSS-SHA256 with ring, PKCS#1 or PKCS#8, told from Ed25519 by the parsed key; tested on RFC 9500's published test key (no private key of our own in a public repo; *.pem is gitignored globally); checked against openssl. Supersedes sprint 06's "Ed25519 only", because a grader's key from Kalshi may well be RSA.
  • A slow connection takes the hourly recycle's next turn (from the review), in place of being recycled at once: the slow recycle waited for the same moment as the hourly one, which nearly always took it first. One path; the mark is logged when made. Supersedes sprint 07's "recycled at once, ahead of the hourly order". LATENCY.md's "no connection was slow" in 20 live minutes rested on the starved count: reworded.
  • The final runs, as they went. Polymarket's 10 minutes met an outage of about two minutes (177 of 178 connections found silent at 16:50:17, so since about 16:49:47; back from 16:51:38 to 16:51:59): kept in the README, as it shows the recovery; 138,950 of its 140,129 losses (the log's and reconcile's) are those two minutes. Kalshi's 10-minute run spent 7.5 of them on the first scan (the demo's listing at 1.7 s a page) and saw 74 normal closes by the demo, never seen before; an A/B of main's build against the branch's, 10 minutes each, saw none on either, so the closes were the demo's. The README's Kalshi run is 20 minutes, long enough for the slow judgment (from a connection's 10th minute): it recycled two. Supersedes "kalshi-demo 10". Logs in ../runs/.
  • ShellCheck in CI: uvx --from shellcheck-py==0.10.0.1 shellcheck over scripts/*.sh (tried: 0.10.0, scripts/ci.sh clean).

Tasks

  • Ctrl-C through the script, tried in a terminal: timeout -s INT and the collector stop cleanly and the script goes on to the checks; the script's handling settled from what happens.
  • scripts/watch.sh: build, bounded run, watch/ (gitignored), the curated view, then reconcile and show.
  • The view's summed book-thread line checked once by hand against the 16 raw lines of the same minute.
  • ShellCheck in scripts/ci.sh.
  • src/bin/show.rs.
  • Test (show): on an output written by AlignedSbeWriter in the test, the symbol with the most Deltas is picked; a symbol named is used; its Deltas come in order and the folded book is right; an unknown symbol is an error.
  • The review: five reviewers over the whole codebase; findings sorted with the user, the fix-now ones added here as tasks.
  • The citations in comments the review lists, rewritten.
  • A short run, stopped with Ctrl-C partway: the output flushes and reconciles.
  • The final runs, scripts/watch.sh polymarket 10 and scripts/watch.sh kalshi-demo 20, from the branch with every fix in, load checked before each; both reconcile; load and peak memory recorded.
  • README.md, its figures from those runs.
  • docs/roadmap.md (M8 done) and CLAUDE.md (watch, show) brought up to date.
  • The stale branches feat/full-coverage and feat/redundant-connections deleted.
  • After the squash into main, with the user's go: the public repo created on git.pband.ch and pushed, and its link opened logged out.

From the review (2026-11-12, at 73a1551):

  • Kalshi's RSA keys (PKCS#1 PEM, as Kalshi generates them), signed RSA-PSS-SHA256 with ring (the scheme assumed until checked in Kalshi's docs); Ed25519 kept. Test: an RSA signature verifies (with ring; checked once by hand with openssl); a key of neither kind is refused.
  • Usage names the two KALSHI_* variables; --help and -h print it; the missing-key error names both variables.
  • The first scan is logged: when it starts, and every 10 s the markets listed so far; watch.awk shows them.
  • A slow connection takes the hourly recycle's next turn, in place of the oldest: one recycle path. Test: with the hourly recycle running, a connection slow 3 minutes is the next recycled.
  • A symbol tracked again after its market was removed: its first Snapshot logs a cause, tracked again, which reconcile reads. Test: remove, add, the Snapshot reconciles with the log.
  • watch.sh exits non-zero when the collector or reconcile fails.
  • book::add checks for overflow; an overflow counts as a loss and resyncs. Test.
  • Test: orchestrator::run offline against the local listing and websocket servers; on stop, each socket gets a Close and the output reconciles.
  • Tests: a command sent on an open connection reaches the server; a 1013 close becomes a split.
  • Tests for the review's surviving mutants: Kalshi's rule (src/kalshi/feed.rs 94, 114, 215, 285), the shell (src/core/feed.rs 483, 539, 589, 683, 687), show (several asks, the 20-Delta cap, the first ladder), reconcile (orphan Deltas, one cause line explains one Snapshot), the socket (a dead book thread stops it, the PING).
  • Test: a sample log through watch.awk, from scripts/ci.sh.
  • a_silent_connection_reconnects_with_its_tokens fails instead of hanging (a timeout on the server's accept).
  • "ponytail:" becomes "Ceiling:" (4 places); "lag" for lateness and "stream" reworded (or "stream" in GLOSSARY.md); the 4 history citations rewritten; Cargo.toml's comments, the CHUNK constants' names, impl RecentChange's place, "the server" in watch.sh.

From the review of 26b4d67:

  • watch.sh records the peak thread count, not the last sample.
  • README: Polymarket's threads (one per connection plus about 20, derived from the Kalshi run: 137 = 116 + 21, labelled so); losses 140,129 (the log's and reconcile's), the same in this file; the first reconnect at 16:51:38; 177 of the 178 connections; the outage "about two minutes" (silence is noticed after 30 s), the roadmap too; Kalshi's scan at about 1.2 s a page; "every sign of a loss resyncs, and the count includes some that weren't"; "system load"; each term defined at its first use; reconcile's Delta count only; the unwrapped line.
  • This file: the RSA task says the openssl check was by hand; the README section's order follows the README; "Done when" names the 20-minute Kalshi run.
  • Test: a "tracked again" line explains a mismatch in reconcile.
  • Test: an overflowing Kalshi change asks a resync and removes the level.
  • The view prints a minute's sum on the first line 2 s or more after its first book line, too (a book thread with nothing logs none). Test: a minute missing one thread is printed before the orchestrator's line.
  • "Next to recycle" is logged when the mark is new only; the slow log text in one place. Test.
  • Test: the reconcile binary fails on an unexplained mismatch.
  • Test: a mark is cleared when a later judgment finds the connection no longer slow.
  • Test: --help prints the usage and exits 0.
  • The two timing-dependent tests poll until their condition, under a timeout, in place of a fixed sleep.
  • gamma.rs's "stream what they read"; orchestrator.rs:60's unwrapped doc comment; the Kalshi test's hand-built Market uses market().

From the review of d94f683:

  • The view's window is 10 s, to the millisecond and across midnight: a minute's book-thread lines spread 1.6 s in the final Polymarket run, and the 2 s rule, in whole seconds, could split its totals. Tests: a thread 1.6 s late, live and final; across midnight.
  • "Next to recycle" said once per connection while it stays slow, even when two marked connections swap ranks. Test.
  • README: silent between 16:50:17 and 16:50:25; back from 16:51:37; recycling explained before the slow recycle; a line wrapped.
  • The reconcile binary's test matches unexplained: 1 at , not the symbol alone.

Each test is seen to fail once (the code it protects removed, then put back).

Review of d94f683 (2026-11-12)

The 7 commits after the second review's tasks, one reviewer on the five questions. Fix now: the four tasks above. No: the test of an unknown exchange (unasked, kept: cheap and useful); the 1013 test's copy of client (changing client touches every socket test); output(orphan, mismatched)'s two flags (a test helper). Assumed, written so in watch.awk before: "they come within milliseconds", which the final run's log contradicted.

Review of 26b4d67 (2026-11-12)

The diff since 73a1551, five reviewers; none of the findings dropped. Fix now: the tasks above. Otherwise:

  • Spec: no: the Polymarket run predating the view's fix (src/ unchanged since; the sums recomputed on the log); "sprint 08's Outcome" (written at the close); the README's longer reading list.
  • Tests: no: watch.sh's exit status (needs the live exchange or a fake collector; tried by hand); "neither kind" with an EC key (garbage covers the refusal); the first-scan lines typed in test_watch.sh (CLAUDE.md says change both).
  • Smells: no: Symbol::again carrying the exchange's name (documented); 64 << 20 in the library's tests (the binary's constant isn't visible there); run_tests serving Kalshi locally (real sockets).
  • Security: no vulnerability.

Review of 73a1551 (2026-11-12)

The whole codebase, five reviewers (brief, bugs, tests, smells, security); the tests reviewer ran 40 mutants, 19 survived. 2 findings dropped (files since committed; a 429 that recovered). Fix now: the tasks above. Otherwise:

  • Brief: quiet judged against the twin connection, a Rule beyond encode/decode, first one wins in place of a source switch: the README explains each with its ADR. No: /dev/stdout truncating a >>.
  • Bugs: later (docs/roadmap.md): a Kalshi resync left asked on a connection the symbol left; only the first resolved address tried. No: Polymarket changes held at stop, at most 1 s (a README limit); the probe's exit 0 when every connection drops.
  • Smells: no: the two fetchers, Rule::Symbol, the socket's data clumps, on_book's length, the exchanges' decoding and construction.
  • Tests, as built: every new test seen red on its mutant, but one: removing drop(outputs) at the top of orchestrator::run changes nothing a test can see (outputs drops when run returns, which is after threads.join()); kept, as it says when the writer may end.
  • Security: no secret in the tree or history. No: unbounded channels (never dropping is the design; a README limit), raw exchange text in the log, the template's unsafe-libyaml. Kept, the user's call: the commit email and CLAUDE.md's DNS lines in the public history.

Not in this sprint

The trading simulator of docs/evaluation.md (its fill model is a project of its own; reconcile, lateness and rescued counts already measure the claims); the roadmap's possible features (the README's "next"); containerized packaging; a Kalshi production run (no key).

Done when

  • The review's findings are sorted; the fix-now ones are fixed and tested.
  • scripts/watch.sh and show work; Ctrl-C was tried.
  • Both final runs (Polymarket 10 minutes, Kalshi's demo 20) reconcile, load and memory recorded.
  • README.md is written, its figures from those runs.
  • The stale branches are deleted.
  • scripts/ci.sh is green, ShellCheck included.

After the squash:

  • The public repo is created on git.pband.ch and pushed, with the user's go, and its link works logged out.

Outcome

The collector is ready to hand over. scripts/watch.sh runs it with a live view, then reconcile and show (one symbol's book as text); the README replaces the template's; three reviews (the whole codebase, then each round of fixes) and their fixes are in; both final runs reconciled.

  • Fixed, from the whole-codebase review: Kalshi takes RSA keys too (RSA-PSS-SHA256, PKCS#1 or PKCS#8, told by the parsed key); the slow recycle, which the hourly one nearly always took first, now marks the connection and takes the next turn, and acted twice in the 20-minute Kalshi run; a market tracked again says so, so reconcile doesn't fail on it; Book::add checks for overflow; usage, --help and the missing-key error name both key variables; the first scan says how far it got. 19 surviving mutants got their tests, and an offline run of the whole orchestrator proves SIGINT closes every socket and the output reconciles.
  • Final runs: Polymarket, 10 minutes, 199,531 markets on 178 connections, 14.4 million Deltas, through an outage of about two minutes that it recovered from on its own (138,950 of its 140,129 losses). Kalshi's demo, 20 minutes, 260,865 markets on 116 connections, 4.1 million Deltas, 12 losses, two slow connections recycled.

Diverged from the plan, each written into this file when it happened: delivery on the user's Gitea, not GitHub (the user's call, so as not to publish a solution where candidates search); RSA keys taken (sprint 06 had Ed25519 only), tested on RFC 9500's published key since *.pem is ignored globally; the slow recycle takes the hourly turn, not recycled at once; the Kalshi final run is 20 minutes, not 10 (the demo's listing took 5 to 7.5 minutes to page); the README's sections in another order; LATENCY.md's "no connection was slow" withdrawn, since it rested on the starved count.

Left out, and where it went: a Kalshi resync left asked on a connection the symbol left; connecting to every resolved address; paging Kalshi's listing in parallel (docs/roadmap.md, possible features). Unexplained, recorded: the Polymarket outage's cause, and the 74 normal closes by Kalshi's demo in one run (an A/B ruled out our code).

Assumed, not shown: that the outage was on the network's side ("no route to host"); that RSA keys work against Kalshi itself (tried offline only, against openssl); rc shell's interactive use; Polymarket's thread count, about 200, derived from the Kalshi run's.

Reviewed at 73a1551 (the whole codebase), 26b4d67 and d94f683; the 3 commits after d94f683 are its fixes, each with a test seen to fail, and had no further review.

Retrospective

  • Went well: reviewing the whole codebase, not the sprint's diff, found a starved mechanism that had made a sprint 07 claim false; a sprint diff never would have. Rerunning one key mutant of each subagent's tests, and an A/B against main's build, kept their claims and the demo's closes honest. The user's watchable run turned out to be the best test of the view: each live run found something.
  • Went wrong: my own instruments misled me four times: head cut the mutation output so a red test looked green, $? after a pipe reported the pipe's last command, the view summed part of a minute, and a count grouped by 10 s gave a reconnect time 8 s early. The README's first figures came from the view and from memory, not from reconcile and the log, and two reviews corrected them. The view's 2 s window rested on "they come within milliseconds", never measured. Three review rounds where one careful round of fixes would have done.
  • Tested enough? The fixes are, by removal. Not: watch.sh's exit status and its sampling (live only), RSA against Kalshi, the "tracked again" log line itself (stderr), the first-scan lines' wording against the view (typed in the test).
  • Next time: take every quoted figure from the tool that checks it (reconcile, the log), never from a summary of it; check an exit status without a pipe; measure a timing before building a rule on it.