docs: correct the README's claims, state the limits, and CI on macOS

The README said the quiet rule holds for the quietest markets (it can't
judge one under 3 changes a minute) and that all four extensions were
done (two of the brief's latency ideas were not tried); LATENCY.md said
the slow recycle was not seen to fire live, two lines after the run where
it did. Limits gains the process going down, the quiet rule's blind
spots and Kalshi's skipped-change case.

CLAUDE.md loses the home network's details and the server's key paths,
of no use to a reader of the public repo.

scripts/test_watch.sh no longer puts a here-document inside `$( )`:
bash 3.2, which is sh on macOS, stopped at an apostrophe there and
scripts/ci.sh ended on a syntax error.
This commit is contained in:
pascal committed 2026-11-12 23:17:10 +01:00
1 parent 6b3b2c1ca4
commit 00202bbc51
4 files changed
+71 -77

No files matched your search

+5 -10
View File
@@ -2,8 +2,7 @@
A collector for prediction-market order books (Polymarket, then Kalshi) that
emits SBE-encoded updates and per-minute snapshots, built inside the challenge
template. The brief is `../website/Engineering Challenge _ Voluntas.html`,
outside this repo.
template. The brief is https://voluntas.io/engineering-challenge.
## Where things are
@@ -80,11 +79,8 @@ and let the user settle it. Claude owns the data model.
## Environment traps
- **DNS: the exchanges resolve normally on the server** (a remote-claude
workspace, since 2026-11-09). The Mac's dev network blocks them (its resolver
answers `195.186.4.210` / `2001:4d98:4:b10c::1`, a Swisscom block page);
there the user pins the hosts with `sudo ../host.sh on` (workspace root,
outside this repo; `off` removes them), rerun when the CDN addresses rotate.
- **DNS: a network can block the exchanges** (its resolver answers a block
page's address): check what the hosts resolve to before blaming the code.
A 403 is not DNS (see `docs/notes.md`, REST from scripts).
- **Write 64 MiB chunks**: `AlignedSbeReader::open` hardcodes 64 MiB
(`open_with_chunk_size` is private), so any other chunk size is unreadable.
@@ -120,9 +116,8 @@ and let the user settle it. Claude owns the data model.
Check `cat /proc/loadavg` before a live run whose numbers are compared, and
alternate the builds in short rounds rather than running them back to back.
- **Kalshi needs a key; nothing else does**: `KALSHI_KEY_ID_FILE` and
`KALSHI_PRIVATE_KEY_FILE` name its two files (on the server
`/run/secrets/kalshi-demo-key-id` and `kalshi-demo-private-key`; never read
or print them). The key is Ed25519 or RSA (RSA-PSS-SHA256), told apart
`KALSHI_PRIVATE_KEY_FILE` name its two files (never read or print
them). The key is Ed25519 or RSA (RSA-PSS-SHA256), told apart
by the parsed key. Opening ~100 connections at once gets some handshakes
refused with 429; each retries after 1 s and gets in.
- **cargo keeps a build whose source got an older mtime**: a script that
+5 -5
View File
@@ -1,7 +1,7 @@
# Latency experiments
Extension 4 of the challenge: ways to get the exchanges' changes sooner,
tried on Polymarket and Kalshi. Every trick the brief names (connecting
tried on Polymarket and Kalshi's demo. Every trick the brief names (connecting
first after maintenance, spamming connections until a fast one comes,
changing the TCP source port) rests on one question: **is a connection that
is fast now still fast later?** If its speed is noise from message to
@@ -177,8 +177,8 @@ as the median lag), but at the median it wins 9 ms, a tenth of the lag.
- **Connections differ, and keep their speed**: strongly on Polymarket
(rank correlation 0.96 and 0.98), less on Kalshi's demo (0.66 and 0.78).
Nothing we could choose at connect sets it (the address doesn't): it is
drawn per connection, mostly when it opens, though it can shift mid-way
The one thing we could choose at connect, the address, doesn't set it
(the source port could not be tried): it is drawn per connection, mostly when it opens, though it can shift mid-way
(Kalshi's connection 4 went from 75 to 3 ms over its second run).
- **On Polymarket it is worth under a millisecond.** Keeping the fastest
pair wins at the median, not in the tail, where bursts delay every
@@ -211,8 +211,8 @@ as the median lag), but at the median it wins 9 ms, a tenth of the lag.
connections, on the 100 busiest markets, differed far more than the
collector's; assumed to be because each of the collector's shares its
markets with many others and is judged against all their copies, which
evens the means out. So the mechanism is tested but was not seen
to fire live, and its gain on the collector was not measured; how the
evens the means out. So the mechanism is tested and was seen to fire
live, but its gain on the collector was not measured; how the
demo's figures carry to production, which runs elsewhere, is unknown.
- **Distance weighs more**: 14 ms of lag to Polymarket, about 100 ms to
Kalshi's demo, against a median gain from racing under 1 ms and of 9 ms.
+30 -13
View File
@@ -4,15 +4,17 @@ My answer to the Voluntas engineering challenge. The collector subscribes
to every live market of one exchange (Polymarket, or Kalshi), keeps an order
book for each symbol, and writes the template's SBE messages: a Delta for
every change, and a Snapshot of every book each minute and whenever a book
is replaced. All four extensions are done: redundant connections,
is replaced. The four extensions are built: redundant connections,
subscription monitoring, one core for both exchanges, and latency
experiments ([LATENCY.md](LATENCY.md)).
experiments ([LATENCY.md](LATENCY.md); of the brief's three ideas, two
could not be tried from here). What each leaves out is under
[Limits](#limits).
Built with Claude Code. I set the architecture (the threading model,
redundancy, the exchange seam) and settled every design decision; Claude
proposed the options, wrote the code and ran the experiments. Each
decision and its reasons are in [`roadmap/`](roadmap) and
[`docs/adr/`](docs/adr).
The design is mine: the threading model, the redundancy (two connections
per symbol, first one wins), the exchange seam, and every design decision,
each with its reasons in [`roadmap/`](roadmap) and [`docs/adr/`](docs/adr).
I built it with Claude Code, which proposed the options, wrote the code and
ran the experiments.
## Quick start
@@ -156,8 +158,9 @@ code relies on, several found only in live data, are in
- **Subscription monitoring.** A subscription is quiet when it delivered
nothing for a minute while the other connection delivered at least 3
changes of the same symbol: it is resubscribed, and moved if quiet
again. Judged against the other connection, not a rate, it holds for
the quietest markets too. Every
again. Judged against the other connection, not a rate, it needs no
threshold per market; it can't judge a market under 3 changes a minute,
nor one quiet on both connections at once (Limits). Every
connection is recycled about once an hour, one at a time, never two of a
market's; one silent for 30 s reconnects.
- **Generic core.** `src/core/` is the same for both exchanges. An exchange
@@ -166,14 +169,28 @@ code relies on, several found only in live data, are in
decoding, since telling a duplicate from a loss depends on what the
exchange sends (a book hash, or a sequence number and a clock).
- **Latency.** [LATENCY.md](LATENCY.md): a connection that delivers
changes sooner than the others keeps doing so, on both exchanges.
Keeping the fastest gains under a millisecond on Polymarket, and tens of
milliseconds on Kalshi's demo, where the collector now recycles a
connection that stays slow.
changes sooner than the others keeps doing so, on Polymarket and on
Kalshi's demo. Keeping the fastest gains under a millisecond on
Polymarket, and tens of milliseconds on Kalshi's demo, where the
collector now recycles a connection that stays slow. Connecting first
after maintenance and choosing the TCP source port were not tried; the
file says why, and how each would be.
## Limits
- A hard crash (SIGKILL, power) loses the chunk being filled, up to 64 MiB.
- Nothing covers the process itself going down, the fourth failure the
brief's extension 1 names: it is one process, and starting it again on
the same output path truncates the file. A second collector on another
host, the two outputs merged downstream, would cover it; not built.
- Subscription monitoring compares a symbol's two connections, so it
doesn't see a market under 3 changes a minute, nor a symbol quiet on both
at once: only the next fresh book shows what was missed. A rate per
symbol, as the brief describes, would see the second.
- On Kalshi, a change that one connection skipped without a gap in its
sequence numbers would be dropped as a duplicate when the other delivers
it, and the book would stay wrong until the next fresh book. Never
observed; the rule rests on the demo's behaviour.
- When a symbol is subscribed, the exchange's opening book holds a few
changes it never sends as changes: logged as a subscription gap.
- Polymarket changes held for ordering when the run stops (at most 1 s) are
+31 -49
View File
@@ -10,19 +10,24 @@ set -eu
cd "$(dirname "$0")/.."
failed=0
# check NAME LOG EXPECTED [AWK OPTION...]
# The sample logs are files and what the view must show comes on stdin: no
# here-document sits inside `$( )`, where the bash 3.2 that is macOS's sh
# stops at an apostrophe.
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
# check NAME LOG [AWK OPTION...], the expected view on stdin
check() {
name=$1 log=$2 expected=$3
shift 3
actual=$(printf '%s\n' "$log" | awk -v book_threads=2 "$@" -f scripts/watch.awk)
name=$1 log=$2
shift 2
expected=$(cat)
actual=$(awk -v book_threads=2 "$@" -f scripts/watch.awk <"$tmp/$log")
if [ "$actual" != "$expected" ]; then
printf 'watch.awk, %s: printed\n%s\nexpected\n%s\n' "$name" "$actual" "$expected"
failed=1
fi
}
polymarket=$(
cat <<'EOF'
cat >"$tmp/polymarket" <<'EOF'
12:00:00.100 polymarket: orchestrator: first scan, listing the exchange's markets
12:00:30.200 polymarket: orchestrator: first scan done, 40 live markets, subscribing
12:00:30.300 polymarket: connection 0: connected, 40 symbols
@@ -37,10 +42,8 @@ polymarket=$(
12:02:11.000 polymarket: connection 1: connected, 40 symbols
12:02:30.400 polymarket: orchestrator: 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1300 bytes/s
EOF
)
check "Polymarket, live" "$polymarket" "$(
cat <<'EOF'
check "Polymarket, live" polymarket <<'EOF'
12:00:00 polymarket: orchestrator: first scan, listing the exchange's markets
12:00:30 polymarket: orchestrator: first scan done, 40 live markets, subscribing
12:00:30 connections opening (counted in the minute line)
@@ -50,78 +53,60 @@ check "Polymarket, live" "$polymarket" "$(
12:02:10 polymarket: connection 1: recycled
12:02:30 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1300 bytes/s; 1 connected (0 refused, retried), 0 Snapshots with a cause
EOF
)"
check "Polymarket, final" "$polymarket" \
"12:02 books (2 threads): 40 symbols, 660 changes written, 330 duplicates, 9 rescued, 5 resyncs, 1 subscription gaps, 3 losses" \
-v final=1
check "Polymarket, final" polymarket -v final=1 <<'EOF'
12:02 books (2 threads): 40 symbols, 660 changes written, 330 duplicates, 9 rescued, 5 resyncs, 1 subscription gaps, 3 losses
EOF
# A minute with one of its two book threads' lines: printed with the first
# line 10 s after it, before the orchestrator's.
missing=$(
cat <<'EOF'
cat >"$tmp/missing" <<'EOF'
12:01:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:00.500 polymarket: connection 1: recycled
12:01:30.400 polymarket: orchestrator: 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1200 bytes/s
EOF
)
check "a book thread missing" "$missing" "$(
cat <<'EOF'
check "a book thread missing" missing <<'EOF'
12:01:00 polymarket: connection 1: recycled
12:01 books (1 threads): 10 symbols, 100 changes written, 50 duplicates, 2 rescued, 3 resyncs, 1 subscription gaps, 2 losses
12:01:30 40 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 1 recycled (0 slow), 0 silent this minute; busiest connection at 1200 bytes/s; 0 connected (0 refused, retried), 0 Snapshots with a cause
EOF
)"
# A thread's line 1.6 s after the first, past a line between them: still
# the same minute's, summed whole.
late=$(
cat <<'EOF'
cat >"$tmp/late" <<'EOF'
12:01:00.075 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:01.600 polymarket: connection 1: recycled
12:01:01.672 polymarket: book thread 1: 30 symbols; so far 300 changes written (290 verified), 150 duplicates, 4 rescued, 0 held dropped, 0 chains broken, 1 resyncs, 0 full books behind ours, 0 subscription gaps, 1 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 0 reconnect books judged (0 losses); duplicates' lateness 2.0 ms at the median, 8.0 ms at p99; this minute 450 changes received, at most 9 events waited
EOF
)
check "a thread 1.6 s late" "$late" "$(
cat <<'EOF'
check "a thread 1.6 s late" late <<'EOF'
12:01:01 polymarket: connection 1: recycled
12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses
EOF
)"
check "a thread 1.6 s late, final" "$late" \
"12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses" \
-v final=1
check "a thread 1.6 s late, final" late -v final=1 <<'EOF'
12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses
EOF
# Across midnight: a minute begun at 23:59 still ends 10 s on.
midnight=$(
cat <<'EOF'
cat >"$tmp/midnight" <<'EOF'
23:59:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
00:00:30.000 polymarket: connection 1: recycled
EOF
)
check "across midnight" "$midnight" "$(
cat <<'EOF'
check "across midnight" midnight <<'EOF'
23:59 books (1 threads): 10 symbols, 100 changes written, 50 duplicates, 2 rescued, 3 resyncs, 1 subscription gaps, 2 losses
00:00:30 polymarket: connection 1: recycled
EOF
)"
# A line between a minute's book threads: the minute is still summed whole.
interleaved=$(
cat <<'EOF'
cat >"$tmp/interleaved" <<'EOF'
12:01:00.001 polymarket: book thread 0: 10 symbols; so far 100 changes written (90 verified), 50 duplicates, 2 rescued, 0 held dropped, 1 chains broken, 3 resyncs, 0 full books behind ours, 1 subscription gaps, 2 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 4 reconnect books judged (1 losses); duplicates' lateness 1.5 ms at the median, 9.0 ms at p99; this minute 150 changes received, at most 4 events waited
12:01:00.002 polymarket: connection 1: recycled
12:01:00.003 polymarket: book thread 1: 30 symbols; so far 300 changes written (290 verified), 150 duplicates, 4 rescued, 0 held dropped, 0 chains broken, 1 resyncs, 0 full books behind ours, 0 subscription gaps, 1 losses, 0 switches, 0 quiet subscriptions resubscribed, 0 markets reported quiet again, 0 reconnect books judged (0 losses); duplicates' lateness 2.0 ms at the median, 8.0 ms at p99; this minute 450 changes received, at most 9 events waited
EOF
)
check "a line between book threads" "$interleaved" "$(
cat <<'EOF'
check "a line between book threads" interleaved <<'EOF'
12:01:00 polymarket: connection 1: recycled
12:01 books (2 threads): 40 symbols, 400 changes written, 200 duplicates, 6 rescued, 4 resyncs, 1 subscription gaps, 3 losses
EOF
)"
kalshi=$(
cat <<'EOF'
cat >"$tmp/kalshi" <<'EOF'
12:00:00.100 kalshi: orchestrator: first scan, listing the exchange's markets
12:00:10.100 kalshi: orchestrator: first scan, 1000 live markets listed so far
12:00:20.200 kalshi: orchestrator: first scan done, 1500 live markets, subscribing
@@ -136,10 +121,8 @@ kalshi=$(
12:02:00.002 kalshi: book thread 1: 800 symbols; so far 260 changes written, 250 duplicates, 0 before their connection's book, 1 ahead of the book, 2 fresh books (0 losses), 1 below zero, 1 resyncs, 0 quiet subscriptions resubscribed, 0 markets reported quiet again; this minute 120 changes received, at most 4 events waited
12:02:20.400 kalshi: orchestrator: 1500 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 8000 bytes/s
EOF
)
check "Kalshi, live" "$kalshi" "$(
cat <<'EOF'
check "Kalshi, live" kalshi <<'EOF'
12:00:00 kalshi: orchestrator: first scan, listing the exchange's markets
12:00:10 kalshi: orchestrator: first scan, 1000 live markets listed so far
12:00:20 kalshi: orchestrator: first scan done, 1500 live markets, subscribing
@@ -149,9 +132,8 @@ check "Kalshi, live" "$kalshi" "$(
12:02 books (2 threads): 1500 symbols, 410 changes written, 390 duplicates, 3 resyncs, 1 losses
12:02:20 1500 markets on 2 connections; 0 added, 0 removed, 0 moved (0 quiet), 0 splits, 0 recycled (0 slow), 0 silent this minute; busiest connection at 8000 bytes/s; 0 connected (0 refused, retried), 0 Snapshots with a cause
EOF
)"
check "Kalshi, final" "$kalshi" \
"12:02 books (2 threads): 1500 symbols, 410 changes written, 390 duplicates, 3 resyncs, 1 losses" \
-v final=1
check "Kalshi, final" kalshi -v final=1 <<'EOF'
12:02 books (2 threads): 1500 symbols, 410 changes written, 390 duplicates, 3 resyncs, 1 losses
EOF
exit "$failed"