Archived
docs: correct the README's claims, state the limits, and CI on macOS
The README said the quiet rule holds for the quietest markets (it can't judge one under 3 changes a minute) and that all four extensions were done (two of the brief's latency ideas were not tried); LATENCY.md said the slow recycle was not seen to fire live, two lines after the run where it did. Limits gains the process going down, the quiet rule's blind spots and Kalshi's skipped-change case. CLAUDE.md loses the home network's details and the server's key paths, of no use to a reader of the public repo. scripts/test_watch.sh no longer puts a here-document inside `$( )`: bash 3.2, which is sh on macOS, stopped at an apostrophe there and scripts/ci.sh ended on a syntax error.
This commit is contained in:
4 files changed
+71
-77
No files matched your search
@@ -4,15 +4,17 @@ My answer to the Voluntas engineering challenge. The collector subscribes
|
||||
to every live market of one exchange (Polymarket, or Kalshi), keeps an order
|
||||
book for each symbol, and writes the template's SBE messages: a Delta for
|
||||
every change, and a Snapshot of every book each minute and whenever a book
|
||||
is replaced. All four extensions are done: redundant connections,
|
||||
is replaced. The four extensions are built: redundant connections,
|
||||
subscription monitoring, one core for both exchanges, and latency
|
||||
experiments ([LATENCY.md](LATENCY.md)).
|
||||
experiments ([LATENCY.md](LATENCY.md); of the brief's three ideas, two
|
||||
could not be tried from here). What each leaves out is under
|
||||
[Limits](#limits).
|
||||
|
||||
Built with Claude Code. I set the architecture (the threading model,
|
||||
redundancy, the exchange seam) and settled every design decision; Claude
|
||||
proposed the options, wrote the code and ran the experiments. Each
|
||||
decision and its reasons are in [`roadmap/`](roadmap) and
|
||||
[`docs/adr/`](docs/adr).
|
||||
The design is mine: the threading model, the redundancy (two connections
|
||||
per symbol, first one wins), the exchange seam, and every design decision,
|
||||
each with its reasons in [`roadmap/`](roadmap) and [`docs/adr/`](docs/adr).
|
||||
I built it with Claude Code, which proposed the options, wrote the code and
|
||||
ran the experiments.
|
||||
|
||||
## Quick start
|
||||
|
||||
@@ -156,8 +158,9 @@ code relies on, several found only in live data, are in
|
||||
- **Subscription monitoring.** A subscription is quiet when it delivered
|
||||
nothing for a minute while the other connection delivered at least 3
|
||||
changes of the same symbol: it is resubscribed, and moved if quiet
|
||||
again. Judged against the other connection, not a rate, it holds for
|
||||
the quietest markets too. Every
|
||||
again. Judged against the other connection, not a rate, it needs no
|
||||
threshold per market; it can't judge a market under 3 changes a minute,
|
||||
nor one quiet on both connections at once (Limits). Every
|
||||
connection is recycled about once an hour, one at a time, never two of a
|
||||
market's; one silent for 30 s reconnects.
|
||||
- **Generic core.** `src/core/` is the same for both exchanges. An exchange
|
||||
@@ -166,14 +169,28 @@ code relies on, several found only in live data, are in
|
||||
decoding, since telling a duplicate from a loss depends on what the
|
||||
exchange sends (a book hash, or a sequence number and a clock).
|
||||
- **Latency.** [LATENCY.md](LATENCY.md): a connection that delivers
|
||||
changes sooner than the others keeps doing so, on both exchanges.
|
||||
Keeping the fastest gains under a millisecond on Polymarket, and tens of
|
||||
milliseconds on Kalshi's demo, where the collector now recycles a
|
||||
connection that stays slow.
|
||||
changes sooner than the others keeps doing so, on Polymarket and on
|
||||
Kalshi's demo. Keeping the fastest gains under a millisecond on
|
||||
Polymarket, and tens of milliseconds on Kalshi's demo, where the
|
||||
collector now recycles a connection that stays slow. Connecting first
|
||||
after maintenance and choosing the TCP source port were not tried; the
|
||||
file says why, and how each would be.
|
||||
|
||||
## Limits
|
||||
|
||||
- A hard crash (SIGKILL, power) loses the chunk being filled, up to 64 MiB.
|
||||
- Nothing covers the process itself going down, the fourth failure the
|
||||
brief's extension 1 names: it is one process, and starting it again on
|
||||
the same output path truncates the file. A second collector on another
|
||||
host, the two outputs merged downstream, would cover it; not built.
|
||||
- Subscription monitoring compares a symbol's two connections, so it
|
||||
doesn't see a market under 3 changes a minute, nor a symbol quiet on both
|
||||
at once: only the next fresh book shows what was missed. A rate per
|
||||
symbol, as the brief describes, would see the second.
|
||||
- On Kalshi, a change that one connection skipped without a gap in its
|
||||
sequence numbers would be dropped as a duplicate when the other delivers
|
||||
it, and the book would stay wrong until the next fresh book. Never
|
||||
observed; the rule rests on the demo's behaviour.
|
||||
- When a symbol is subscribed, the exchange's opening book holds a few
|
||||
changes it never sends as changes: logged as a subscription gap.
|
||||
- Polymarket changes held for ordering when the run stops (at most 1 s) are
|
||||
|
||||
Reference in new issue
Block a user