docs: correct the README's claims, state the limits, and CI on macOS

The README said the quiet rule holds for the quietest markets (it can't
judge one under 3 changes a minute) and that all four extensions were
done (two of the brief's latency ideas were not tried); LATENCY.md said
the slow recycle was not seen to fire live, two lines after the run where
it did. Limits gains the process going down, the quiet rule's blind
spots and Kalshi's skipped-change case.

CLAUDE.md loses the home network's details and the server's key paths,
of no use to a reader of the public repo.

scripts/test_watch.sh no longer puts a here-document inside `$( )`:
bash 3.2, which is sh on macOS, stopped at an apostrophe there and
scripts/ci.sh ended on a syntax error.
This commit is contained in:
pascal committed 2026-11-12 23:17:10 +01:00
1 parent 6b3b2c1ca4
commit 00202bbc51
4 files changed
+71 -77

No files matched your search

+30 -13
View File
@@ -4,15 +4,17 @@ My answer to the Voluntas engineering challenge. The collector subscribes
to every live market of one exchange (Polymarket, or Kalshi), keeps an order
book for each symbol, and writes the template's SBE messages: a Delta for
every change, and a Snapshot of every book each minute and whenever a book
is replaced. All four extensions are done: redundant connections,
is replaced. The four extensions are built: redundant connections,
subscription monitoring, one core for both exchanges, and latency
experiments ([LATENCY.md](LATENCY.md)).
experiments ([LATENCY.md](LATENCY.md); of the brief's three ideas, two
could not be tried from here). What each leaves out is under
[Limits](#limits).
Built with Claude Code. I set the architecture (the threading model,
redundancy, the exchange seam) and settled every design decision; Claude
proposed the options, wrote the code and ran the experiments. Each
decision and its reasons are in [`roadmap/`](roadmap) and
[`docs/adr/`](docs/adr).
The design is mine: the threading model, the redundancy (two connections
per symbol, first one wins), the exchange seam, and every design decision,
each with its reasons in [`roadmap/`](roadmap) and [`docs/adr/`](docs/adr).
I built it with Claude Code, which proposed the options, wrote the code and
ran the experiments.
## Quick start
@@ -156,8 +158,9 @@ code relies on, several found only in live data, are in
- **Subscription monitoring.** A subscription is quiet when it delivered
nothing for a minute while the other connection delivered at least 3
changes of the same symbol: it is resubscribed, and moved if quiet
again. Judged against the other connection, not a rate, it holds for
the quietest markets too. Every
again. Judged against the other connection, not a rate, it needs no
threshold per market; it can't judge a market under 3 changes a minute,
nor one quiet on both connections at once (Limits). Every
connection is recycled about once an hour, one at a time, never two of a
market's; one silent for 30 s reconnects.
- **Generic core.** `src/core/` is the same for both exchanges. An exchange
@@ -166,14 +169,28 @@ code relies on, several found only in live data, are in
decoding, since telling a duplicate from a loss depends on what the
exchange sends (a book hash, or a sequence number and a clock).
- **Latency.** [LATENCY.md](LATENCY.md): a connection that delivers
changes sooner than the others keeps doing so, on both exchanges.
Keeping the fastest gains under a millisecond on Polymarket, and tens of
milliseconds on Kalshi's demo, where the collector now recycles a
connection that stays slow.
changes sooner than the others keeps doing so, on Polymarket and on
Kalshi's demo. Keeping the fastest gains under a millisecond on
Polymarket, and tens of milliseconds on Kalshi's demo, where the
collector now recycles a connection that stays slow. Connecting first
after maintenance and choosing the TCP source port were not tried; the
file says why, and how each would be.
## Limits
- A hard crash (SIGKILL, power) loses the chunk being filled, up to 64 MiB.
- Nothing covers the process itself going down, the fourth failure the
brief's extension 1 names: it is one process, and starting it again on
the same output path truncates the file. A second collector on another
host, the two outputs merged downstream, would cover it; not built.
- Subscription monitoring compares a symbol's two connections, so it
doesn't see a market under 3 changes a minute, nor a symbol quiet on both
at once: only the next fresh book shows what was missed. A rate per
symbol, as the brief describes, would see the second.
- On Kalshi, a change that one connection skipped without a gap in its
sequence numbers would be dropped as a duplicate when the other delivers
it, and the book would stay wrong until the next fresh book. Never
observed; the rule rests on the demo's behaviour.
- When a symbol is subscribed, the exchange's opening book holds a few
changes it never sends as changes: logged as a subscription gap.
- Polymarket changes held for ordering when the run stops (at most 1 s) are