feat(latency): latency experiments, LATENCY.md, Kalshi's slow-connection recycle

Extension 4. `engineering-challenge latency EXCHANGE OUT.csv` opens 16
connections to the same busy markets and writes every copy of every change;
`scripts/latency/analyse.py` gives each connection's speed, whether it
persists, what keeping the fastest would gain, lateness by address, and lag
with the clock's offset. LATENCY.md writes up four runs: on Polymarket speed
persists but is worth under a millisecond, at the median only; on Kalshi's
demo keeping the fastest pair lowers p99 by 28% and 42%, the threshold
agreed before the runs.

So the collector now recycles a Kalshi connection whose mean lateness stays,
3 minutes running, in the slowest quarter and above twice everyone's, ahead
of its hourly turn. Kalshi's rule measures each duplicate's lateness; the
orchestrator judges once a minute. A duplicate's lag in Polymarket's minute
line is now its "lateness".

Roadmap milestone 7, roadmap/07.
This commit is contained in:
pascal committed 2026-11-12 15:05:25 +01:00
1 parent 1a5bbcd602
commit 515e85941d
22 files changed
+2132 -55

No files matched your search

+13
View File
@@ -64,6 +64,19 @@ _Avoid_: Source, primary, standby, failover (no connection has a rank)
**Duplicate**:
A change another connection already delivered.
**Lateness**:
How long after a change's first copy a connection delivered it; 0 for the
connection that delivered first. Needs no clock.
_Avoid_: Lag, delay
**Lag**:
A change's receive time minus the exchange's timestamp for it. It carries
the local clock's offset.
_Avoid_: Latency (the general idea, not a number)
**Speed**:
A connection's median lateness over a minute.
**Verified**:
Said of a symbol whose book gives the exchange's book hash after every
change. It stops being verified when a change fits on no connection in time.