Archived
feat(latency): latency experiments, LATENCY.md, Kalshi's slow-connection recycle
Extension 4. `engineering-challenge latency EXCHANGE OUT.csv` opens 16 connections to the same busy markets and writes every copy of every change; `scripts/latency/analyse.py` gives each connection's speed, whether it persists, what keeping the fastest would gain, lateness by address, and lag with the clock's offset. LATENCY.md writes up four runs: on Polymarket speed persists but is worth under a millisecond, at the median only; on Kalshi's demo keeping the fastest pair lowers p99 by 28% and 42%, the threshold agreed before the runs. So the collector now recycles a Kalshi connection whose mean lateness stays, 3 minutes running, in the slowest quarter and above twice everyone's, ahead of its hourly turn. Kalshi's rule measures each duplicate's lateness; the orchestrator judges once a minute. A duplicate's lag in Polymarket's minute line is now its "lateness". Roadmap milestone 7, roadmap/07.
This commit is contained in:
22 files changed
+2132
-55
No files matched your search
@@ -313,6 +313,24 @@ is the demo's.
|
||||
(2026-11-11, 19:29 UTC).
|
||||
- **Maintenance (docs):** Thursdays 3:00 to 5:00 AM ET; connections drop.
|
||||
|
||||
## Latency, from the server (observed 2026-11-12)
|
||||
|
||||
- **The server's clock is synced**: SNTP against `time.cloudflare.com` read
|
||||
−0.25 to +0.69 ms, 3 to 4 ms round trips (sprint 07's probe).
|
||||
- **What the exchange sees isn't ours to choose**: a NAT rewrites every
|
||||
connection's source port (not even stable for one local port), and there
|
||||
is no IPv6 route; Polymarket's websocket resolves to 4 IPv6 addresses
|
||||
beside its 4 IPv4 ones (CloudFront).
|
||||
- **Where the hosts are** (AWS's published ranges): Kalshi production's
|
||||
websocket in us-east-2, the demo's two addresses in us-west-2; lag about
|
||||
14 ms to Polymarket, about 100 ms to Kalshi's demo.
|
||||
- **Polymarket repeats a change on one connection**, rarely: 22 of 29,922
|
||||
changes came twice on every connection in a minute.
|
||||
- **Kalshi's listing order moves**: its first 30,000 open markets held the
|
||||
98 busiest at 11:12 and none of them at 11:50. Pick by counting every
|
||||
market, not by the listing's first pages. The demo reports
|
||||
`volume_24h_fp` 0.00 (267,691 open markets, a full listing in 75 s).
|
||||
|
||||
## Clock accuracy bounds time accuracy
|
||||
`received_at` comes from the local clock, so it carries the clock's error. On
|
||||
2026-11-04 the dev machine ran ~30 ms slow (`sntp`: +30.6 ms ± 13 ms), which
|
||||
|
||||
+5
-2
@@ -150,7 +150,8 @@ schema debate). As settled in sprints 04 and 05:
|
||||
└──────┬───▲───┘ ──── recycle ────────▶ connections
|
||||
Add / Move / Remove (a market and │ │ load reports, each minute
|
||||
its connections), to its thread │ │ (bytes/s per market); markets
|
||||
│ │ quiet twice on a connection
|
||||
│ │ quiet twice on a connection;
|
||||
│ │ lateness per connection (Kalshi)
|
||||
┌──────▼───┴───┐
|
||||
connection 0..N ──events───▶ │ book threads │ ──Snapshot | Delta──▶ writer
|
||||
(read, stamp, parse, route) │ 0..15 │
|
||||
@@ -160,7 +161,9 @@ schema debate). As settled in sprints 04 and 05:
|
||||
|
||||
- The **orchestrator** owns what is tracked (each market's two connections,
|
||||
the timers) and decides: placing, moving, splitting, and recycling one
|
||||
connection at a time, about hourly. It never blocks on HTTP.
|
||||
connection at a time, about hourly, and sooner a Kalshi connection whose
|
||||
mean lateness stays above twice everyone's (sprint 07, LATENCY.md). It
|
||||
never blocks on HTTP.
|
||||
- The two **fetchers** only page the exchange's market listing on request
|
||||
and stream what they read.
|
||||
- A **connection** reads, stamps the receive time, parses, and routes each
|
||||
|
||||
+22
-5
@@ -77,12 +77,19 @@ abstraction is *extracted* once a second exchange exists, not designed up front.
|
||||
higher than sprint 05's; Kalshi on its demo (about 245,000 markets on 110
|
||||
connections), production not run for lack of a key (sprint 06's Outcome).
|
||||
|
||||
7. **Extension 4 — latency experiments + writeup** — *not started*
|
||||
Experiment with faster connections (first-after-maintenance, connection
|
||||
racing, source-port tricks) on both exchanges; document findings in a
|
||||
markdown file, with the local clock's offset measured alongside (the dev
|
||||
machine ran ~30 ms slow; see notes.md).
|
||||
7. **Extension 4 — latency experiments + writeup** — *done (2026-11-12)*
|
||||
Is a connection that is fast now still fast later? 16 connections to the
|
||||
same markets, ranked by how long after a change's first copy each
|
||||
delivers it (no clock needed), with the clock's offset measured by SNTP;
|
||||
written up in [../LATENCY.md](../LATENCY.md).
|
||||
Sprint: [../roadmap/07. latency experiments.md](../roadmap/07.%20latency%20experiments.md).
|
||||
*Ends:* experiments documented.
|
||||
Result: speed persists on both exchanges; keeping the fastest wins under
|
||||
a millisecond on Polymarket (not in the tail), tens of milliseconds on
|
||||
Kalshi's demo, where the collector now recycles a connection whose mean
|
||||
lateness stays above twice everyone's (none was, in 20 live minutes).
|
||||
The source port can't be chosen here (a NAT rewrites it); first after
|
||||
maintenance wasn't tried.
|
||||
|
||||
8. **Polish + submission** — *not started*
|
||||
Final review, README for the solution (the template's README still says the
|
||||
@@ -138,3 +145,13 @@ abstraction is *extracted* once a second exchange exists, not designed up front.
|
||||
- Telling Kalshi's book at a market's close, and a connection's book merely
|
||||
ahead of the other's stream, from a real loss (sprint 06 logs all three as
|
||||
a loss; the Snapshot is right either way).
|
||||
- The first connection after Kalshi's maintenance (Thursdays 08:00 to
|
||||
10:00 UTC in winter): the probe at the window's end against an hour later
|
||||
(sprint 07 ran after one).
|
||||
- The TCP source port's effect on speed, from a host with a public address
|
||||
(here a NAT rewrites it).
|
||||
- The slow recycle's gain on Kalshi production, measured on the collector
|
||||
(sprint 07 measured racing with the probe, on the demo only).
|
||||
- Judging the connections' speed at the book threads' wall-clock minute,
|
||||
not on the orchestrator's own timer (a judgment near the crossing sees
|
||||
partial samples; sprint 07's review).
|
||||
Reference in new issue
Block a user