This repository has been archived on 2026-11-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
orderbook-collector/roadmap/08. ready to submit.md
T
pascal cf6c16c9cf feat: ready to submit, a run to watch, the README, the review's fixes
Sprint 08 (roadmap/08. ready to submit.md), the last milestone.

- scripts/watch.sh runs the collector with a live view of its log, stops
  cleanly on Ctrl-C, then checks the output (reconcile) and shows one
  symbol's book as text (show, a new binary).
- The README for the solution, with the final runs: Polymarket through an
  outage it recovered from on its own, Kalshi's demo with two slow
  connections recycled.
- From a review of the whole codebase: Kalshi takes RSA keys too
  (RSA-PSS-SHA256); a slow Kalshi connection takes the hourly recycle's
  next turn, which it nearly never got before; a market tracked again no
  longer fails reconcile; Book::add checks for overflow; usage and --help
  name the key variables; the first scan says how far it got.
- Tests for every gap the reviews found, an offline run of the whole
  orchestrator among them; ShellCheck in CI.
2026-11-12 22:32:39 +01:00

426 lines
22 KiB
Markdown

# 08. Ready to submit
**Status: done (2026-11-12)**
M8, the last milestone: the collector handed to Voluntas in a state a
reviewer can read, run and check on their own. A review of the whole
codebase (not one sprint's diff) and its fixes; one command that runs the
collector and shows what it does; a README for the solution in place of
the template's; a final live run of each exchange, reconciled, whose
figures the README quotes; and the handoff as a public repo.
## The watched run
`scripts/watch.sh [EXCHANGE] [MINUTES]`, defaults `polymarket` (no key) and
3 minutes (two minute summaries; Polymarket tracks about 200,000 markets
within its first minute, `../runs/poly10.log`). It builds release, runs the
collector under `timeout -s INT`, and writes the output and the full log
to a gitignored `watch/`. On the terminal, a curated view of the log:
- each minute, the orchestrator's line as is, and the 16 book threads'
lines summed into one (changes written, duplicates, rescued, resyncs,
losses, subscription gaps; their counts are cumulative, "so far");
- between them, the notable events only: a connection lost, recycled
(hourly or slow), split, silent; the startup's connections as one count,
not a line each. The Snapshot cause lines (3,300 of 3,952 lines in a
10-minute Polymarket run) stay in the file.
Ctrl-C stops the run early and cleanly: the script goes on to the checks
(how Ctrl-C reaches `timeout` and the collector from a script is
**assumed**; a quick try without a terminal didn't deliver the signal, so
the first task tries it in one). Then it runs `reconcile` on the output
and the log, and `show` on the output.
On the workspace: `rc shell voluntas`, then `cd voluntas/repo &&
scripts/watch.sh` (`rc shell`'s interactive use is assumed from
`rc shell exec`). Kalshi needs `KALSHI_KEY_ID_FILE` and
`KALSHI_PRIVATE_KEY_FILE` exported; without them the collector stops
before connecting, naming the variable (`src/kalshi/sign.rs:26`). The
script adds nothing for keys.
## show
`src/bin/show.rs`: `show OUTPUT [SYMBOL]`, reading through the template's
`AlignedSbeReader` as `src/reconcile.rs` does. With no symbol, the one
with the most Deltas in the file. It prints that symbol's first Snapshot
(top 5 levels a side, its receive time), its next 20 Deltas (receive time,
side, price, new quantity; 0 shown as deleted), then the top 5 levels a
side of the book folded from them, in exchange units through
`book::format_fixed` (`src/book.rs:46`). A Polymarket symbol is its token
id: the output holds no market name. An unknown symbol is an error.
## The review
The whole codebase, not a diff: `src/`, `scripts/`, the tests, by five
`reviewer` subagents:
- **brief**, in place of a spec reviewer: the brief checked line by line
against the code (every live symbol, MVEs excluded on Kalshi, new and
gone symbols, Deltas to stdout and a Snapshot each minute, SIGINT closes
the websockets and the file, each extension);
- **smells**, read as a stranger: leftovers of superseded designs (lanes,
the template's demo), comments that need sprint history (19 citations
of `roadmap/0N` or "sprint 0N" in `src/`), the two exchanges
inconsistent with each other;
- **bugs**, **tests**, **security**, as usual; sprint 07's 11 post-review
fixes, never reviewed a second time, included.
Findings are sorted with the user: fix now (a task below, with its test),
or a limit or a "next" in the README.
## The README
Replaces the template's, for the Voluntas reader, in this order: what the
collector does, with the line on how it was built (below); quick start
(`scripts/watch.sh`); build and run per exchange, Kalshi's two `export`
lines, and that Kalshi ran only against the demo; checking the output
(`reconcile` with the log); the final runs' figures; the design in one
diagram (`docs/project.md`'s pipeline); each extension and where it is
met (ADRs, `LATENCY.md`); the limits (the crash ceiling, the subscription
gap when a symbol opens, Kalshi production never run); next (a short
pick of the roadmap's possible features); what to read first
(`docs/project.md`, `docs/adr/`, `LATENCY.md`, `docs/notes.md`, and
`GLOSSARY.md`, `CLAUDE.md`) and that `roadmap/` holds each sprint's plan
and outcome; an "Output format" section (the template's
`AlignedSbeWriter` unchanged, 64 MiB chunks, the schema's path). The
order as built supersedes the first one (the runs after the extensions,
the line on how it was built last): the runs read as evidence for the
design that follows, and the line belongs with what the project is. The
line:
> Built with Claude Code. I set the architecture (the threading model,
> redundancy, the exchange seam) and settled every design decision;
> Claude proposed the options, wrote the code and ran the experiments.
> Each decision and its reasons are in `roadmap/` and `docs/adr/`.
## Decisions
- **The watched run is a script over the existing log**, not pipeline
code. Rejected: a status line in the collector (counts across threads,
new pipeline code); a TUI.
- **The data is shown, not only counted** (`show`), after the run: the
writer emits 64 MiB chunks, so a live stream of the output would come
in bursts.
- **One script for everything**: the final runs and the README's quick
start are `scripts/watch.sh`, so the grader runs what made the
README's figures. The workspace's `../runs/run.sh` is retired.
- **Everything ships unchanged** (`docs/`, `roadmap/`, `CLAUDE.md`,
`GLOSSARY.md`, `LATENCY.md`); the README says what to read first.
Rejected: stripping the process files; the work being AI-assisted is
said openly instead.
- **Sprint citations in comments**: a file path stays (`roadmap/` ships);
a reference that leans on history is rewritten to say what the code
does, or to cite the ADR or `docs/notes.md`.
- **The final runs** pass `reconcile` (every mismatch explained, no
orphan Delta), load and peak memory recorded (`VmHWM` from `/proc`);
no comparison with earlier sprints' runs (done in their sprints; the
load differs from day to day).
- **The template's README text**: its stale lines go ("read `main.rs`",
"as demonstrated in `main.rs`"); the rest becomes "Output format".
- **Delivery: a public repo on the user's Gitea (git.pband.ch), not on
GitHub.** The brief says nothing on how (it links only the template,
public, 25 forks). Public, so the link opens with no account; on
GitHub it would publish a solution to the challenge where other
candidates search, which the user doesn't want. Rejected: a GitHub
fork (listed under the template), a GitHub repo of our own (found by
search), a private repo (an invite to a handle not known), a PR to the
template (public). Full history, the template's 3 commits first, so a
diff against it shows what was added. Pushed over SSH (which reaches
Gitea from here), with the user's go. Supersedes "a public repo of the
user's own on GitHub", at the user's request during the build.
- **Kalshi's RSA keys are taken** (from the review): RSA-PSS-SHA256 with
`ring`, PKCS#1 or PKCS#8, told from Ed25519 by the parsed key; tested
on RFC 9500's published test key (no private key of our own in a public
repo; `*.pem` is gitignored globally); checked
against `openssl`. Supersedes sprint 06's "Ed25519 only", because a
grader's key from Kalshi may well be RSA.
- **A slow connection takes the hourly recycle's next turn** (from the
review), in place of being recycled at once: the slow recycle waited for
the same moment as the hourly one, which nearly always took it first.
One path; the mark is logged when made. Supersedes sprint 07's "recycled
at once, ahead of the hourly order". `LATENCY.md`'s "no connection was
slow" in 20 live minutes rested on the starved count: reworded.
- **The final runs, as they went.** Polymarket's 10 minutes met an outage
of about two minutes (177 of 178 connections found silent at 16:50:17,
so since about 16:49:47; back from 16:51:38 to 16:51:59): kept in the
README, as it shows the recovery; 138,950 of its 140,129 losses (the
log's and `reconcile`'s) are those two minutes. Kalshi's 10-minute run spent 7.5
of them on the first scan (the demo's listing at 1.7 s a page) and saw
74 normal closes by the demo, never seen before; an A/B of `main`'s
build against the branch's, 10 minutes each, saw none on either, so
the closes were the demo's. The README's Kalshi run is 20 minutes,
long enough for the slow judgment (from a connection's 10th minute):
it recycled two. Supersedes "kalshi-demo 10". Logs in `../runs/`.
- **ShellCheck in CI**: `uvx --from shellcheck-py==0.10.0.1 shellcheck`
over `scripts/*.sh` (tried: 0.10.0, `scripts/ci.sh` clean).
## Tasks
- [x] Ctrl-C through the script, tried in a terminal: `timeout -s INT`
and the collector stop cleanly and the script goes on to the
checks; the script's handling settled from what happens.
- [x] `scripts/watch.sh`: build, bounded run, `watch/` (gitignored), the
curated view, then `reconcile` and `show`.
- [x] The view's summed book-thread line checked once by hand against
the 16 raw lines of the same minute.
- [x] ShellCheck in `scripts/ci.sh`.
- [x] `src/bin/show.rs`.
- [x] Test (`show`): on an output written by `AlignedSbeWriter` in the
test, the symbol with the most Deltas is picked; a symbol named is
used; its Deltas come in order and the folded book is right; an
unknown symbol is an error.
- [x] The review: five reviewers over the whole codebase; findings
sorted with the user, the fix-now ones added here as tasks.
- [x] The citations in comments the review lists, rewritten.
- [x] A short run, stopped with Ctrl-C partway: the output flushes and
reconciles.
- [x] The final runs, `scripts/watch.sh polymarket 10` and
`scripts/watch.sh kalshi-demo 20`, from the branch with every fix
in, load checked before each; both reconcile; load and peak
memory recorded.
- [x] `README.md`, its figures from those runs.
- [x] `docs/roadmap.md` (M8 done) and `CLAUDE.md` (watch, show) brought
up to date.
- [x] The stale branches `feat/full-coverage` and
`feat/redundant-connections` deleted.
- [ ] After the squash into `main`, with the user's go: the public repo
created on git.pband.ch and pushed, and its link opened logged
out.
From the review (2026-11-12, at `73a1551`):
- [x] Kalshi's RSA keys (PKCS#1 PEM, as Kalshi generates them), signed
RSA-PSS-SHA256 with `ring` (the scheme assumed until checked in
Kalshi's docs); Ed25519 kept. Test: an RSA signature verifies (with
`ring`; checked once by hand with `openssl`); a key of neither kind
is refused.
- [x] Usage names the two `KALSHI_*` variables; `--help` and `-h` print
it; the missing-key error names both variables.
- [x] The first scan is logged: when it starts, and every 10 s the
markets listed so far; `watch.awk` shows them.
- [x] A slow connection takes the hourly recycle's next turn, in place of
the oldest: one recycle path. Test: with the hourly recycle running,
a connection slow 3 minutes is the next recycled.
- [x] A symbol tracked again after its market was removed: its first
Snapshot logs a cause, tracked again, which `reconcile` reads. Test:
remove, add, the Snapshot reconciles with the log.
- [x] `watch.sh` exits non-zero when the collector or `reconcile` fails.
- [x] `book::add` checks for overflow; an overflow counts as a loss and
resyncs. Test.
- [x] Test: `orchestrator::run` offline against the local listing and
websocket servers; on stop, each socket gets a Close and the output
reconciles.
- [x] Tests: a command sent on an open connection reaches the server; a
1013 close becomes a split.
- [x] Tests for the review's surviving mutants: Kalshi's rule
(`src/kalshi/feed.rs` 94, 114, 215, 285), the shell
(`src/core/feed.rs` 483, 539, 589, 683, 687), `show` (several asks,
the 20-Delta cap, the first ladder), `reconcile` (orphan Deltas, one
cause line explains one Snapshot), the socket (a dead book thread
stops it, the PING).
- [x] Test: a sample log through `watch.awk`, from `scripts/ci.sh`.
- [x] `a_silent_connection_reconnects_with_its_tokens` fails instead of
hanging (a timeout on the server's accept).
- [x] "ponytail:" becomes "Ceiling:" (4 places); "lag" for lateness and
"stream" reworded (or "stream" in `GLOSSARY.md`); the 4 history
citations rewritten; `Cargo.toml`'s comments, the `CHUNK`
constants' names, `impl RecentChange`'s place, "the server" in
`watch.sh`.
From the review of `26b4d67`:
- [x] `watch.sh` records the peak thread count, not the last sample.
- [x] README: Polymarket's threads (one per connection plus about 20,
derived from the Kalshi run: 137 = 116 + 21, labelled so); losses
140,129 (the log's and `reconcile`'s), the same in this file; the
first reconnect at 16:51:38; 177 of the 178 connections; the outage
"about two minutes" (silence is noticed after 30 s), the roadmap
too; Kalshi's scan at about 1.2 s a page; "every sign of a loss
resyncs, and the count includes some that weren't"; "system load";
each term defined at its first use; `reconcile`'s Delta count only;
the unwrapped line.
- [x] This file: the RSA task says the `openssl` check was by hand; the
README section's order follows the README; "Done when" names the
20-minute Kalshi run.
- [x] Test: a "tracked again" line explains a mismatch in `reconcile`.
- [x] Test: an overflowing Kalshi change asks a resync and removes the
level.
- [x] The view prints a minute's sum on the first line 2 s or more after
its first book line, too (a book thread with nothing logs none).
Test: a minute missing one thread is printed before the
orchestrator's line.
- [x] "Next to recycle" is logged when the mark is new only; the slow log
text in one place. Test.
- [x] Test: the `reconcile` binary fails on an unexplained mismatch.
- [x] Test: a mark is cleared when a later judgment finds the connection
no longer slow.
- [x] Test: `--help` prints the usage and exits 0.
- [x] The two timing-dependent tests poll until their condition, under a
timeout, in place of a fixed sleep.
- [x] `gamma.rs`'s "stream what they read"; `orchestrator.rs:60`'s
unwrapped doc comment; the Kalshi test's hand-built `Market` uses
`market()`.
From the review of `d94f683`:
- [x] The view's window is 10 s, to the millisecond and across midnight:
a minute's book-thread lines spread 1.6 s in the final Polymarket
run, and the 2 s rule, in whole seconds, could split its totals.
Tests: a thread 1.6 s late, live and final; across midnight.
- [x] "Next to recycle" said once per connection while it stays slow,
even when two marked connections swap ranks. Test.
- [x] README: silent between 16:50:17 and 16:50:25; back from 16:51:37;
recycling explained before the slow recycle; a line wrapped.
- [x] The `reconcile` binary's test matches `unexplained: 1 at `, not the
symbol alone.
Each test is seen to fail once (the code it protects removed, then put
back).
## Review of `d94f683` (2026-11-12)
The 7 commits after the second review's tasks, one reviewer on the five
questions. Fix now: the four tasks above. No: the test of an unknown
exchange (unasked, kept: cheap and useful); the 1013 test's copy of
`client` (changing `client` touches every socket test); `output(orphan,
mismatched)`'s two flags (a test helper). Assumed, written so in
`watch.awk` before: "they come within milliseconds", which the final
run's log contradicted.
## Review of `26b4d67` (2026-11-12)
The diff since `73a1551`, five reviewers; none of the findings dropped.
Fix now: the tasks above. Otherwise:
- **Spec:** no: the Polymarket run predating the view's fix (`src/`
unchanged since; the sums recomputed on the log); "sprint 08's Outcome"
(written at the close); the README's longer reading list.
- **Tests:** no: `watch.sh`'s exit status (needs the live exchange or a
fake collector; tried by hand); "neither kind" with an EC key (garbage
covers the refusal); the first-scan lines typed in `test_watch.sh`
(`CLAUDE.md` says change both).
- **Smells:** no: `Symbol::again` carrying the exchange's name
(documented); `64 << 20` in the library's tests (the binary's constant
isn't visible there); `run_tests` serving Kalshi locally (real sockets).
- **Security:** no vulnerability.
## Review of `73a1551` (2026-11-12)
The whole codebase, five reviewers (brief, bugs, tests, smells,
security); the tests reviewer ran 40 mutants, 19 survived. 2 findings
dropped (files since committed; a 429 that recovered). Fix now: the
tasks above. Otherwise:
- **Brief:** quiet judged against the twin connection, a `Rule` beyond
encode/decode, first one wins in place of a source switch: the README
explains each with its ADR. No: `/dev/stdout` truncating a `>>`.
- **Bugs:** later (`docs/roadmap.md`): a Kalshi resync left asked on a
connection the symbol left; only the first resolved address tried. No:
Polymarket changes held at stop, at most 1 s (a README limit); the
probe's exit 0 when every connection drops.
- **Smells:** no: the two fetchers, `Rule::Symbol`, the socket's data
clumps, `on_book`'s length, the exchanges' decoding and construction.
- **Tests, as built:** every new test seen red on its mutant, but one:
removing `drop(outputs)` at the top of `orchestrator::run` changes
nothing a test can see (`outputs` drops when `run` returns, which is
after `threads.join()`); kept, as it says when the writer may end.
- **Security:** no secret in the tree or history. No: unbounded channels
(never dropping is the design; a README limit), raw exchange text in
the log, the template's `unsafe-libyaml`. Kept, the user's call: the
commit email and `CLAUDE.md`'s DNS lines in the public history.
## Not in this sprint
The trading simulator of `docs/evaluation.md` (its fill model is a project
of its own; reconcile, lateness and rescued counts already measure the
claims); the roadmap's possible features (the README's "next");
containerized packaging; a Kalshi production run (no key).
## Done when
- The review's findings are sorted; the fix-now ones are fixed and
tested.
- `scripts/watch.sh` and `show` work; Ctrl-C was tried.
- Both final runs (Polymarket 10 minutes, Kalshi's demo 20) reconcile,
load and memory recorded.
- `README.md` is written, its figures from those runs.
- The stale branches are deleted.
- `scripts/ci.sh` is green, ShellCheck included.
After the squash:
- The public repo is created on git.pband.ch and pushed, with the user's
go, and its link works logged out.
## Outcome
The collector is ready to hand over. `scripts/watch.sh` runs it with a live
view, then `reconcile` and `show` (one symbol's book as text); the README
replaces the template's; three reviews (the whole codebase, then each round
of fixes) and their fixes are in; both final runs reconciled.
- **Fixed, from the whole-codebase review:** Kalshi takes RSA keys too
(RSA-PSS-SHA256, PKCS#1 or PKCS#8, told by the parsed key); the slow
recycle, which the hourly one nearly always took first, now marks the
connection and takes the next turn, and acted twice in the 20-minute
Kalshi run; a market tracked again says so, so `reconcile` doesn't fail
on it; `Book::add` checks for overflow; usage, `--help` and the
missing-key error name both key variables; the first scan says how far
it got. 19 surviving mutants got their tests, and an offline run of the
whole orchestrator proves SIGINT closes every socket and the output
reconciles.
- **Final runs:** Polymarket, 10 minutes, 199,531 markets on 178
connections, 14.4 million Deltas, through an outage of about two minutes
that it recovered from on its own (138,950 of its 140,129 losses).
Kalshi's demo, 20 minutes, 260,865 markets on 116 connections, 4.1
million Deltas, 12 losses, two slow connections recycled.
**Diverged from the plan**, each written into this file when it
happened: delivery on the user's Gitea, not GitHub (the user's call, so as
not to publish a solution where candidates search); RSA keys taken
(sprint 06 had Ed25519 only), tested on RFC 9500's published key since
`*.pem` is ignored globally; the slow recycle takes the hourly turn, not
recycled at once; the Kalshi final run is 20 minutes, not 10 (the demo's
listing took 5 to 7.5 minutes to page); the README's sections in another
order; `LATENCY.md`'s "no connection was slow" withdrawn, since it rested
on the starved count.
**Left out**, and where it went: a Kalshi resync left asked on a
connection the symbol left; connecting to every resolved address;
paging Kalshi's listing in parallel (`docs/roadmap.md`, possible
features). Unexplained, recorded: the Polymarket outage's cause, and the
74 normal closes by Kalshi's demo in one run (an A/B ruled out our code).
**Assumed**, not shown: that the outage was on the network's side ("no
route to host"); that RSA keys work against Kalshi itself (tried offline
only, against `openssl`); `rc shell`'s interactive use; Polymarket's
thread count, about 200, derived from the Kalshi run's.
**Reviewed** at `73a1551` (the whole codebase), `26b4d67` and `d94f683`;
the 3 commits after `d94f683` are its fixes, each with a test seen to
fail, and had no further review.
### Retrospective
- **Went well:** reviewing the whole codebase, not the sprint's diff, found
a starved mechanism that had made a sprint 07 claim false; a sprint diff
never would have. Rerunning one key mutant of each subagent's tests, and
an A/B against `main`'s build, kept their claims and the demo's closes
honest. The user's watchable run turned out to be the best test of the
view: each live run found something.
- **Went wrong:** my own instruments misled me four times: `head` cut the
mutation output so a red test looked green, `$?` after a pipe reported
the pipe's last command, the view summed part of a minute, and a count
grouped by 10 s gave a reconnect time 8 s early. The README's first
figures came from the view and from memory, not from `reconcile` and the
log, and two reviews corrected them. The view's 2 s window rested on
"they come within milliseconds", never measured. Three review rounds
where one careful round of fixes would have done.
- **Tested enough?** The fixes are, by removal. Not: `watch.sh`'s exit
status and its sampling (live only), RSA against Kalshi, the "tracked
again" log line itself (stderr), the first-scan lines' wording against
the view (typed in the test).
- **Next time:** take every quoted figure from the tool that checks it
(`reconcile`, the log), never from a summary of it; check an exit status
without a pipe; measure a timing before building a rule on it.