feat: ready to submit, a run to watch, the README, the review's fixes

Sprint 08 (roadmap/08. ready to submit.md), the last milestone.

- scripts/watch.sh runs the collector with a live view of its log, stops
  cleanly on Ctrl-C, then checks the output (reconcile) and shows one
  symbol's book as text (show, a new binary).
- The README for the solution, with the final runs: Polymarket through an
  outage it recovered from on its own, Kalshi's demo with two slow
  connections recycled.
- From a review of the whole codebase: Kalshi takes RSA keys too
  (RSA-PSS-SHA256); a slow Kalshi connection takes the hourly recycle's
  next turn, which it nearly never got before; a market tracked again no
  longer fails reconcile; Book::add checks for overflow; usage and --help
  name the key variables; the first scan says how far it got.
- Tests for every gap the reviews found, an offline run of the whole
  orchestrator among them; ShellCheck in CI.
This commit is contained in:
pascal committed 2026-11-12 22:32:39 +01:00
1 parent 515e85941d
commit cf6c16c9cf
35 files changed
+2791 -207

No files matched your search

+157
View File
@@ -0,0 +1,157 @@
#!/bin/sh
# Pipes a sample log of each exchange through scripts/watch.awk, live and
# with -v final=1, and compares what it prints with what the view must show;
# exits non-zero on a difference. The lines are worded as the collector
# writes them: the book threads' minute lines (log_minute in
# src/polymarket/feed.rs and src/kalshi/feed.rs), the orchestrator's
# (src/core/orchestrator.rs), the connections' (src/core/socket.rs) and a
# Snapshot's cause (src/cause.rs).
set -eu
cd "$(dirname "$0")/.."
failed=0
# check NAME LOG EXPECTED [AWK OPTION...]
check() {
name=$1 log=$2 expected=$3
shift 3
actual=$(printf '%s\n' "$log" | awk -v book_threads=2 "$@" -f scripts/watch.awk)
if [ "$actual" != "$expected" ]; then
printf 'watch.awk, %s: printed\n%s\nexpected\n%s\n' "$name" "$actual" "$expected"
failed=1
fi
}
polymarket=$(
cat <<'EOF'
12:00:00.100 polymarket: orchestrator: first scan, listing the exchange's markets
12:00:30.200 polymarket: orchestrator: first scan done, 40 live markets, subscribing
12:00:30.300 polymarket: connection 0: connected, 40 symbols
12:00:30.310 polymarket: connection 1: connected, 40 symbols
12:01:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:00.002 polymarket: book thread 1: 30 symbols; so far 300 changes written (290 verified), 150 duplicates, 4 rescued, 0 held dropped, 0 chains broken, 1 resyncs, 0 full books behind ours, 0 subscription gaps, 1 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 0 reconnect books judged (0 losses); duplicates' lateness 2.0 ms at the median, 8.0 ms at p99; this minute 450 changes received, at most 9 events waited
12:01:00.050 polymarket: Snapshot of 123 at 1794390060050000: loss
12:01:30.400 polymarket: orchestrator: 40 markets on 2 connections; 40 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 1200 bytes/s
12:02:00.001 polymarket: book thread 0: 10 symbols; so far 160 changes written (150 verified), 80 duplicates, 3 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 1 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 90 changes received, at most 2 events waited
12:02:00.002 polymarket: book thread 1: 30 symbols; so far 500 changes written (480 verified), 250 duplicates, 6 rescued, 0 held dropped, 0 chains broken, 2 resyncs, 0 full books behind ours, 0 subscription gaps, 1 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 0 reconnect books judged (0 losses); duplicates' lateness 2.0 ms at the median, 8.0 ms at p99; this minute 300 changes received, at most 5 events waited
12:02:10.000 polymarket: connection 1: recycled
12:02:11.000 polymarket: connection 1: connected, 40 symbols
12:02:30.400 polymarket: orchestrator: 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1300 bytes/s
EOF
)
check "Polymarket, live" "$polymarket" "$(
cat <<'EOF'
12:00:00 polymarket: orchestrator: first scan, listing the exchange's markets
12:00:30 polymarket: orchestrator: first scan done, 40 live markets, subscribing
12:00:30 connections opening (counted in the minute line)
12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses
12:01:30 40 markets on 2 connections; 40 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 1200 bytes/s; 2 connected (0 refused, retried), 1 Snapshots with a cause
12:02 books (2 threads): 40 symbols, 660 changes written, 330 duplicates, 9 rescued, 5 resyncs, 1 subscription gaps, 3 losses
12:02:10 polymarket: connection 1: recycled
12:02:30 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1300 bytes/s; 1 connected (0 refused, retried), 0 Snapshots with a cause
EOF
)"
check "Polymarket, final" "$polymarket" \
"12:02 books (2 threads): 40 symbols, 660 changes written, 330 duplicates, 9 rescued, 5 resyncs, 1 subscription gaps, 3 losses" \
-v final=1
# A minute with one of its two book threads' lines: printed with the first
# line 10 s after it, before the orchestrator's.
missing=$(
cat <<'EOF'
12:01:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:00.500 polymarket: connection 1: recycled
12:01:30.400 polymarket: orchestrator: 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1200 bytes/s
EOF
)
check "a book thread missing" "$missing" "$(
cat <<'EOF'
12:01:00 polymarket: connection 1: recycled
12:01 books (1 threads): 10 symbols, 100 changes written, 50 duplicates, 2 rescued, 3 resyncs, 1 subscription gaps, 2 losses
12:01:30 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1200 bytes/s; 0 connected (0 refused, retried), 0 Snapshots with a cause
EOF
)"
# A thread's line 1.6 s after the first, past a line between them: still
# the same minute's, summed whole.
late=$(
cat <<'EOF'
12:01:00.075 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:01.600 polymarket: connection 1: recycled
12:01:01.672 polymarket: book thread 1: 30 symbols; so far 300 changes written (290 verified), 150 duplicates, 4 rescued, 0 held dropped, 0 chains broken, 1 resyncs, 0 full books behind ours, 0 subscription gaps, 1 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 0 reconnect books judged (0 losses); duplicates' lateness 2.0 ms at the median, 8.0 ms at p99; this minute 450 changes received, at most 9 events waited
EOF
)
check "a thread 1.6 s late" "$late" "$(
cat <<'EOF'
12:01:01 polymarket: connection 1: recycled
12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses
EOF
)"
check "a thread 1.6 s late, final" "$late" \
"12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses" \
-v final=1
# Across midnight: a minute begun at 23:59 still ends 10 s on.
midnight=$(
cat <<'EOF'
23:59:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
00:00:30.000 polymarket: connection 1: recycled
EOF
)
check "across midnight" "$midnight" "$(
cat <<'EOF'
23:59 books (1 threads): 10 symbols, 100 changes written, 50 duplicates, 2 rescued, 3 resyncs, 1 subscription gaps, 2 losses
00:00:30 polymarket: connection 1: recycled
EOF
)"
# A line between a minute's book threads: the minute is still summed whole.
interleaved=$(
cat <<'EOF'
12:01:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:00.002 polymarket: connection 1: recycled
12:01:00.003 polymarket: book thread 1: 30 symbols; so far 300 changes written (290 verified), 150 duplicates, 4 rescued, 0 held dropped, 0 chains broken, 1 resyncs, 0 full books behind ours, 0 subscription gaps, 1 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 0 reconnect books judged (0 losses); duplicates' lateness 2.0 ms at the median, 8.0 ms at p99; this minute 450 changes received, at most 9 events waited
EOF
)
check "a line between book threads" "$interleaved" "$(
cat <<'EOF'
12:01:00 polymarket: connection 1: recycled
12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses
EOF
)"
kalshi=$(
cat <<'EOF'
12:00:00.100 kalshi: orchestrator: first scan, listing the exchange's markets
12:00:10.100 kalshi: orchestrator: first scan, 1000 live markets listed so far
12:00:20.200 kalshi: orchestrator: first scan done, 1500 live markets, subscribing
12:00:20.300 kalshi: connection 0: connected, 1500 symbols
12:00:20.310 kalshi: connection 1: connect failed: handshake refused: 429 Too Many Requests
12:00:21.320 kalshi: connection 1: connected, 1500 symbols
12:01:00.001 kalshi: book thread 0: 700 symbols; so far 100 changes written, 90 duplicates, 1 before their connection's book, 0 ahead of the book, 3 fresh books (1 losses), 0 below zero, 2 resyncs, 0 quiet subscriptions resubscribed, 0 markets reported quiet again; this minute 190 changes received, at most 3 events waited
12:01:00.002 kalshi: book thread 1: 800 symbols; so far 200 changes written, 190 duplicates, 0 before their connection's book, 1 ahead of the book, 2 fresh books (0 losses), 1 below zero, 1 resyncs, 0 quiet subscriptions resubscribed, 0 markets reported quiet again; this minute 390 changes received, at most 6 events waited
12:01:00.030 kalshi: Snapshot of KXA at 1794390060030000: loss (its clock 12, ours 11)
12:01:20.400 kalshi: orchestrator: 1500 markets on 2 connections; 1500 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 9000 bytes/s
12:02:00.001 kalshi: book thread 0: 700 symbols; so far 150 changes written, 140 duplicates, 1 before their connection's book, 0 ahead of the book, 3 fresh books (1 losses), 0 below zero, 2 resyncs, 0 quiet subscriptions resubscribed, 0 markets reported quiet again; this minute 100 changes received, at most 2 events waited
12:02:00.002 kalshi: book thread 1: 800 symbols; so far 260 changes written, 250 duplicates, 0 before their connection's book, 1 ahead of the book, 2 fresh books (0 losses), 1 below zero, 1 resyncs, 0 quiet subscriptions resubscribed, 0 markets reported quiet again; this minute 120 changes received, at most 4 events waited
12:02:20.400 kalshi: orchestrator: 1500 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 8000 bytes/s
EOF
)
check "Kalshi, live" "$kalshi" "$(
cat <<'EOF'
12:00:00 kalshi: orchestrator: first scan, listing the exchange's markets
12:00:10 kalshi: orchestrator: first scan, 1000 live markets listed so far
12:00:20 kalshi: orchestrator: first scan done, 1500 live markets, subscribing
12:00:20 connections opening (counted in the minute line)
12:01 books (2 threads): 1500 symbols, 300 changes written, 280 duplicates, 3 resyncs, 1 losses
12:01:20 1500 markets on 2 connections; 1500 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 9000 bytes/s; 2 connected (1 refused, retried), 1 Snapshots with a cause
12:02 books (2 threads): 1500 symbols, 410 changes written, 390 duplicates, 3 resyncs, 1 losses
12:02:20 1500 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 8000 bytes/s; 0 connected (0 refused, retried), 0 Snapshots with a cause
EOF
)"
check "Kalshi, final" "$kalshi" \
"12:02 books (2 threads): 1500 symbols, 410 changes written, 390 duplicates, 3 resyncs, 1 losses" \
-v final=1
exit "$failed"