Archived
feat: ready to submit, a run to watch, the README, the review's fixes
Sprint 08 (roadmap/08. ready to submit.md), the last milestone. - scripts/watch.sh runs the collector with a live view of its log, stops cleanly on Ctrl-C, then checks the output (reconcile) and shows one symbol's book as text (show, a new binary). - The README for the solution, with the final runs: Polymarket through an outage it recovered from on its own, Kalshi's demo with two slow connections recycled. - From a review of the whole codebase: Kalshi takes RSA keys too (RSA-PSS-SHA256); a slow Kalshi connection takes the hourly recycle's next turn, which it nearly never got before; a market tracked again no longer fails reconcile; Book::add checks for overflow; usage and --help name the key variables; the first scan says how far it got. - Tests for every gap the reviews found, an offline run of the whole orchestrator among them; ShellCheck in CI.
This commit is contained in:
35 files changed
+2791
-207
No files matched your search
@@ -23,8 +23,8 @@ outside this repo.
|
||||
Delta, lateness, lag, speed).
|
||||
- `src/core/`: what is the same for every exchange (ADR 0004).
|
||||
`orchestrator.rs`: what is tracked, which connections carry each market,
|
||||
their balancing and their recycling (hourly, and sooner for a Kalshi
|
||||
connection that stays among the slowest); `socket.rs`: a connection (read,
|
||||
their balancing and their recycling (hourly, a Kalshi connection that
|
||||
stays among the slowest taking the next turn); `socket.rs`: a connection (read,
|
||||
stamp, decode, route; reconnects when dropped, silent, lost a message or
|
||||
recycled); `feed.rs`: a book thread's shell (Snapshots, moves, the quiet
|
||||
watch) and the `Rule` trait; `mod.rs`: the `Exchange`, `Session`, `Market`
|
||||
@@ -37,8 +37,14 @@ outside this repo.
|
||||
rank rule, `mod.rs` the exchange and its session (the `seq` check).
|
||||
- `src/book.rs`: the book and fixed point; `src/reconcile.rs` +
|
||||
`src/bin/reconcile.rs`: the output check, reading the log's cause lines
|
||||
(`src/cause.rs`). `tests/fixtures/`: recordings of the live feeds that the
|
||||
(`src/cause.rs`); `src/show.rs` + `src/bin/show.rs`: one symbol of the
|
||||
output as text. `tests/fixtures/`: recordings of the live feeds that the
|
||||
tests replay.
|
||||
- `scripts/watch.sh [EXCHANGE] [MINUTES]`: a run to watch, also the final
|
||||
runs' and the README's quick start (roadmap/08): the collector under
|
||||
`timeout -s INT`, a live view of its log (`scripts/watch.awk`, which
|
||||
parses the minute lines' wording: change one, check the other), then
|
||||
`reconcile` and `show`. Output and log in `watch/`.
|
||||
- `src/latency.rs` + `scripts/latency/analyse.py`: the latency probe
|
||||
(`engineering-challenge latency EXCHANGE OUT.csv`, until SIGINT: one CSV
|
||||
line per copy of a change on each of 16 connections) and its analysis
|
||||
@@ -67,6 +73,10 @@ and let the user settle it. Claude owns the data model.
|
||||
- Use `GLOSSARY.md`'s terms: a **Snapshot** is only the output message; getting
|
||||
a fresh book is a **resync**.
|
||||
- A corner cut for speed carries a comment naming its ceiling and upgrade path.
|
||||
- A figure quoted in a doc (README, `LATENCY.md`, a sprint's Outcome) comes
|
||||
from the tool that checks it, `reconcile`'s output or the log itself,
|
||||
never from `watch.sh`'s view or a count grouped by hand: sprint 08's
|
||||
first README figures had four errors from those.
|
||||
|
||||
## Environment traps
|
||||
|
||||
@@ -91,7 +101,20 @@ and let the user settle it. Claude owns the data model.
|
||||
(`pgrep -f '^./target/release/...'`, anchored) and loop on `kill -0 $pid`.
|
||||
- **No `/usr/bin/time` on the server**: for a run's peak memory and thread
|
||||
count, read `VmHWM` and `Threads` from `/proc/<pid>/status` while it runs
|
||||
(sprint 05's run script sampled them every 30 s).
|
||||
(`scripts/watch.sh` samples them every 10 s).
|
||||
- **mawk buffers a pipe's input** unless `-W interactive`: a live view
|
||||
through `awk` shows nothing until the end (`scripts/watch.sh` passes it
|
||||
when the awk is mawk, as on the server).
|
||||
- **A job started with `&` in `sh` ignores Ctrl-C**: a script that runs the
|
||||
collector in the background must pass SIGINT on itself (`watch.sh`'s
|
||||
trap). To try Ctrl-C without a terminal: `(sleep N; printf '\003') |
|
||||
script -qfec "CMD" /dev/null`.
|
||||
- **Polymarket's Gamma answers 429 to runs started close together**: two
|
||||
collectors' first scans at once (or a few runs within minutes) got every
|
||||
page refused; the scan retries, but leave a few minutes between runs.
|
||||
- **`*.pem` is gitignored globally** (`~/.gitignore`): a test key goes in
|
||||
the source, as `src/kalshi/sign.rs` does with RFC 8032's and RFC 9500's
|
||||
published test keys.
|
||||
- **The server is shared and gets overloaded** (load up to 92 on 12 cores,
|
||||
2026-11-11): a starved collector falls behind and gets closed with 1013.
|
||||
Check `cat /proc/loadavg` before a live run whose numbers are compared, and
|
||||
@@ -99,7 +122,8 @@ and let the user settle it. Claude owns the data model.
|
||||
- **Kalshi needs a key; nothing else does**: `KALSHI_KEY_ID_FILE` and
|
||||
`KALSHI_PRIVATE_KEY_FILE` name its two files (on the server
|
||||
`/run/secrets/kalshi-demo-key-id` and `kalshi-demo-private-key`; never read
|
||||
or print them). Opening ~100 connections at once gets some handshakes
|
||||
or print them). The key is Ed25519 or RSA (RSA-PSS-SHA256), told apart
|
||||
by the parsed key. Opening ~100 connections at once gets some handshakes
|
||||
refused with 429; each retries after 1 s and gets in.
|
||||
- **cargo keeps a build whose source got an older mtime**: a script that
|
||||
restores a file from a copy (`shutil.move`, `cp -p`) must `touch` it, or
|
||||
|
||||
Reference in new issue
Block a user