Archived
feat(latency): latency experiments, LATENCY.md, Kalshi's slow-connection recycle
Extension 4. `engineering-challenge latency EXCHANGE OUT.csv` opens 16 connections to the same busy markets and writes every copy of every change; `scripts/latency/analyse.py` gives each connection's speed, whether it persists, what keeping the fastest would gain, lateness by address, and lag with the clock's offset. LATENCY.md writes up four runs: on Polymarket speed persists but is worth under a millisecond, at the median only; on Kalshi's demo keeping the fastest pair lowers p99 by 28% and 42%, the threshold agreed before the runs. So the collector now recycles a Kalshi connection whose mean lateness stays, 3 minutes running, in the slowest quarter and above twice everyone's, ahead of its hourly turn. Kalshi's rule measures each duplicate's lateness; the orchestrator judges once a minute. A duplicate's lag in Polymarket's minute line is now its "lateness". Roadmap milestone 7, roadmap/07.
This commit is contained in:
22 files changed
+2132
-55
No files matched your search
+5
-2
@@ -150,7 +150,8 @@ schema debate). As settled in sprints 04 and 05:
|
||||
└──────┬───▲───┘ ──── recycle ────────▶ connections
|
||||
Add / Move / Remove (a market and │ │ load reports, each minute
|
||||
its connections), to its thread │ │ (bytes/s per market); markets
|
||||
│ │ quiet twice on a connection
|
||||
│ │ quiet twice on a connection;
|
||||
│ │ lateness per connection (Kalshi)
|
||||
┌──────▼───┴───┐
|
||||
connection 0..N ──events───▶ │ book threads │ ──Snapshot | Delta──▶ writer
|
||||
(read, stamp, parse, route) │ 0..15 │
|
||||
@@ -160,7 +161,9 @@ schema debate). As settled in sprints 04 and 05:
|
||||
|
||||
- The **orchestrator** owns what is tracked (each market's two connections,
|
||||
the timers) and decides: placing, moving, splitting, and recycling one
|
||||
connection at a time, about hourly. It never blocks on HTTP.
|
||||
connection at a time, about hourly, and sooner a Kalshi connection whose
|
||||
mean lateness stays above twice everyone's (sprint 07, LATENCY.md). It
|
||||
never blocks on HTTP.
|
||||
- The two **fetchers** only page the exchange's market listing on request
|
||||
and stream what they read.
|
||||
- A **connection** reads, stamps the receive time, parses, and routes each
|
||||
|
||||
Reference in new issue
Block a user