Archived
feat!: a generic core, and Kalshi through it
The orchestrator, the connection loop and the book thread's shell move to src/core/, generic over an Exchange and its loss rule (ADR 0004). Polymarket runs through it unchanged. Kalshi joins: a signed handshake (KALSHI_KEY_ID_FILE, KALSHI_PRIVATE_KEY_FILE), a reconnect on a skipped sequence number, first one wins by clock and rank, and its market listing with markets tracked from three hours before they open. Breaking: the exchange is now the first argument, `engineering-challenge <polymarket|kalshi|kalshi-demo> [OUTPUT]`.
This commit is contained in:
33 files changed
+8147
-1843
No files matched your search
@@ -13,22 +13,31 @@ outside this repo.
|
|||||||
agreed plan. Read the current sprint file before writing code.
|
agreed plan. Read the current sprint file before writing code.
|
||||||
- `docs/adr/`: hard-to-reverse decisions (0001: threads, shared-nothing;
|
- `docs/adr/`: hard-to-reverse decisions (0001: threads, shared-nothing;
|
||||||
0002: the orchestrator, its lanes superseded; 0003: two connections per
|
0002: the orchestrator, its lanes superseded; 0003: two connections per
|
||||||
symbol, first one wins).
|
symbol, first one wins; 0004: the exchange seam, a websocket layer and a
|
||||||
|
loss rule per exchange).
|
||||||
- `docs/notes.md`: domain rules and exchange protocol facts (delta semantics,
|
- `docs/notes.md`: domain rules and exchange protocol facts (delta semantics,
|
||||||
loss detection, message shapes). Read before touching exchange parsing.
|
loss detection, message shapes). Read before touching exchange parsing.
|
||||||
- `GLOSSARY.md`: the terms (symbol, connection, load, first one wins,
|
- `GLOSSARY.md`: the terms (symbol, connection, load, first one wins,
|
||||||
duplicate, held, rescued, follow, resync, loss, subscription gap, split,
|
duplicate, held, rescued, follow, resync, loss, subscription gap, split,
|
||||||
quiet, resubscribe, silent, recycle, Snapshot, Delta).
|
quiet, resubscribe, silent, recycle, sequence number, clock, Snapshot,
|
||||||
- `src/polymarket/orchestrator.rs`: what is tracked, which connections carry
|
Delta).
|
||||||
each market, their balancing and their recycling; `src/polymarket/gamma.rs`: the fetchers
|
- `src/core/`: what is the same for every exchange (ADR 0004).
|
||||||
(discovery); `src/polymarket/socket.rs`: a connection (read, stamp, decode,
|
`orchestrator.rs`: what is tracked, which connections carry each market,
|
||||||
route; reconnects when dropped, silent or recycled);
|
their balancing and their recycling; `socket.rs`: a connection (read,
|
||||||
`src/polymarket/wire.rs`: the exchange's messages;
|
stamp, decode, route; reconnects when dropped, silent, lost a message or
|
||||||
`src/polymarket/feed.rs`: a book thread, its loss rules (the subtle
|
recycled); `feed.rs`: a book thread's shell (Snapshots, moves, the quiet
|
||||||
part) and its watch for quiet subscriptions; `src/book.rs`: the book and fixed point; `src/reconcile.rs` +
|
watch) and the `Rule` trait; `mod.rs`: the `Exchange`, `Session`, `Market`
|
||||||
|
and `Event` traits.
|
||||||
|
- `src/polymarket/`: `gamma.rs` the fetchers (discovery), `wire.rs` the
|
||||||
|
exchange's messages, `feed.rs` the hash rule (the subtle part),
|
||||||
|
`mod.rs` the exchange and its session.
|
||||||
|
- `src/kalshi/`: `listing.rs` the fetchers, `wire.rs` the messages,
|
||||||
|
`sign.rs` the key and the handshake's signature, `feed.rs` the clock and
|
||||||
|
rank rule, `mod.rs` the exchange and its session (the `seq` check).
|
||||||
|
- `src/book.rs`: the book and fixed point; `src/reconcile.rs` +
|
||||||
`src/bin/reconcile.rs`: the output check, reading the log's cause lines
|
`src/bin/reconcile.rs`: the output check, reading the log's cause lines
|
||||||
(`src/cause.rs`). `tests/fixtures/`: recordings of the live feed, on one and two
|
(`src/cause.rs`). `tests/fixtures/`: recordings of the live feeds that the
|
||||||
connections, that the tests replay.
|
tests replay.
|
||||||
- `src/util/` and `sbe_schemas/`: the template's SBE machinery, used as given so
|
- `src/util/` and `sbe_schemas/`: the template's SBE machinery, used as given so
|
||||||
output stays readable by the template's `AlignedSbeReader`. Our code is the
|
output stays readable by the template's `AlignedSbeReader`. Our code is the
|
||||||
rest of `src/`.
|
rest of `src/`.
|
||||||
@@ -66,7 +75,7 @@ and let the user settle it. Claude owns the data model.
|
|||||||
- **Only a clean stop flushes the output**: SIGINT, SIGTERM or SIGHUP. SIGKILL
|
- **Only a clean stop flushes the output**: SIGINT, SIGTERM or SIGHUP. SIGKILL
|
||||||
or SIGALRM leaves the last 64 MiB chunk unwritten. Bound a run with GNU
|
or SIGALRM leaves the last 64 MiB chunk unwritten. Bound a run with GNU
|
||||||
`timeout -s INT`, then check the output against the log:
|
`timeout -s INT`, then check the output against the log:
|
||||||
`timeout -s INT 600 ./target/release/engineering-challenge out.sbe 2> run.log;
|
`timeout -s INT 600 ./target/release/engineering-challenge polymarket out.sbe 2> run.log;
|
||||||
./target/release/reconcile out.sbe run.log`. With the log, `reconcile`
|
./target/release/reconcile out.sbe run.log`. With the log, `reconcile`
|
||||||
pairs each mismatch with its cause line and fails only on one left
|
pairs each mismatch with its cause line and fails only on one left
|
||||||
unexplained.
|
unexplained.
|
||||||
@@ -81,6 +90,15 @@ and let the user settle it. Claude owns the data model.
|
|||||||
2026-11-11): a starved collector falls behind and gets closed with 1013.
|
2026-11-11): a starved collector falls behind and gets closed with 1013.
|
||||||
Check `cat /proc/loadavg` before a live run whose numbers are compared, and
|
Check `cat /proc/loadavg` before a live run whose numbers are compared, and
|
||||||
alternate the builds in short rounds rather than running them back to back.
|
alternate the builds in short rounds rather than running them back to back.
|
||||||
|
- **Kalshi needs a key; nothing else does**: `KALSHI_KEY_ID_FILE` and
|
||||||
|
`KALSHI_PRIVATE_KEY_FILE` name its two files (on the server
|
||||||
|
`/run/secrets/kalshi-demo-key-id` and `kalshi-demo-private-key`; never read
|
||||||
|
or print them). Opening ~100 connections at once gets some handshakes
|
||||||
|
refused with 429; each retries after 1 s and gets in.
|
||||||
|
- **cargo keeps a build whose source got an older mtime**: a script that
|
||||||
|
restores a file from a copy (`shutil.move`, `cp -p`) must `touch` it, or
|
||||||
|
the next `cargo test` runs the old binary (a mutation check did, and a
|
||||||
|
"flaky" test was only the last mutant).
|
||||||
- **Live data beats the docs**: Polymarket's market channel has rules its
|
- **Live data beats the docs**: Polymarket's market channel has rules its
|
||||||
docs don't state (`docs/notes.md`); check a protocol assumption on a
|
docs don't state (`docs/notes.md`); check a protocol assumption on a
|
||||||
recording of the real feed (`tests/fixtures/record.py`) before relying on it.
|
recording of the real feed (`tests/fixtures/record.py`) before relying on it.
|
||||||
|
|||||||
Generated
+3
@@ -301,12 +301,15 @@ name = "engineering-challenge"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
|
"base64",
|
||||||
"chrono",
|
"chrono",
|
||||||
"crossbeam",
|
"crossbeam",
|
||||||
"ctrlc",
|
"ctrlc",
|
||||||
"ergo-sbe",
|
"ergo-sbe",
|
||||||
"indexmap",
|
"indexmap",
|
||||||
|
"ring",
|
||||||
"rustls",
|
"rustls",
|
||||||
|
"rustls-pki-types",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
"serde_yaml_ng",
|
"serde_yaml_ng",
|
||||||
|
|||||||
@@ -8,13 +8,19 @@ default-run = "engineering-challenge"
|
|||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
anyhow = "1"
|
anyhow = "1"
|
||||||
|
# Kalshi's handshake signature.
|
||||||
|
base64 = "0.23"
|
||||||
chrono = "0.4"
|
chrono = "0.4"
|
||||||
crossbeam = "0.8.4"
|
crossbeam = "0.8.4"
|
||||||
# `termination`: SIGTERM and SIGHUP stop the collector like SIGINT.
|
# `termination`: SIGTERM and SIGHUP stop the collector like SIGINT.
|
||||||
ctrlc = { version = "3", features = ["termination"] }
|
ctrlc = { version = "3", features = ["termination"] }
|
||||||
# Not used directly: tungstenite leaves rustls's crypto provider to the app, and
|
# Not used directly: tungstenite leaves rustls's crypto provider to the app, and
|
||||||
# rustls panics at runtime unless exactly one (here ring) is enabled.
|
# rustls panics at runtime unless exactly one (here ring) is enabled.
|
||||||
|
# Signs Kalshi's handshake (Ed25519); already rustls's crypto provider.
|
||||||
|
ring = "0.17"
|
||||||
rustls = { version = "0.23", default-features = false, features = ["ring"] }
|
rustls = { version = "0.23", default-features = false, features = ["ring"] }
|
||||||
|
# Reads the PEM of Kalshi's key.
|
||||||
|
rustls-pki-types = { version = "1", features = ["std"] }
|
||||||
serde = { version = "1", features = ["derive"] }
|
serde = { version = "1", features = ["derive"] }
|
||||||
serde_json = "1"
|
serde_json = "1"
|
||||||
sha1 = "0.11"
|
sha1 = "0.11"
|
||||||
|
|||||||
+19
-5
@@ -7,24 +7,37 @@ for the backtests that read its output.
|
|||||||
|
|
||||||
**Market**:
|
**Market**:
|
||||||
A tradable question on an exchange. On Polymarket it has two outcomes (Yes,
|
A tradable question on an exchange. On Polymarket it has two outcomes (Yes,
|
||||||
No), each traded as its own symbol, and their books mirror each other.
|
No), each traded as its own symbol, and their books mirror each other. On
|
||||||
|
Kalshi it has one book, holding bids for Yes and bids for No.
|
||||||
_Avoid_: Event, contract
|
_Avoid_: Event, contract
|
||||||
|
|
||||||
**Symbol**:
|
**Symbol**:
|
||||||
The unit that has one order book: on Polymarket a token (one outcome of a
|
The unit that has one order book: on Polymarket a token (one outcome of a
|
||||||
market), named by its token id.
|
market), named by its token id; on Kalshi the market itself, named by its
|
||||||
|
ticker, a bid for No at a price being an ask at one minus that price.
|
||||||
_Avoid_: Asset, instrument, ticker (for Polymarket)
|
_Avoid_: Asset, instrument, ticker (for Polymarket)
|
||||||
|
|
||||||
**Live**:
|
**Live**:
|
||||||
Said of a market (and its symbols) the exchange lists as open for trading on
|
Said of a market (and its symbols) the exchange lists as open for trading on
|
||||||
its order book. A market stops being live when it closes or stops accepting
|
its order book. A market stops being live when it closes or stops accepting
|
||||||
orders, not at its end date.
|
orders, not at its end date. A Kalshi market about to open is tracked ahead
|
||||||
|
of being live, its book empty.
|
||||||
_Avoid_: Active (Gamma's flag, only one input of being live)
|
_Avoid_: Active (Gamma's flag, only one input of being live)
|
||||||
|
|
||||||
**Book hash**:
|
**Book hash**:
|
||||||
The exchange's fingerprint of a symbol's full book state, sent with every
|
Polymarket's fingerprint of a symbol's full book state, sent with every
|
||||||
full book and every change.
|
full book and every change.
|
||||||
|
|
||||||
|
**Sequence number**:
|
||||||
|
Kalshi's count of the messages it sent on one connection, all its markets
|
||||||
|
together: one skipped means the connection lost a message, for a market it
|
||||||
|
doesn't name.
|
||||||
|
|
||||||
|
**Clock**:
|
||||||
|
Kalshi's counter for one market, the same on every connection, carried by
|
||||||
|
every book and change. Several changes can share one clock, always in the
|
||||||
|
same order.
|
||||||
|
|
||||||
## Collector side
|
## Collector side
|
||||||
|
|
||||||
**Book**:
|
**Book**:
|
||||||
@@ -100,7 +113,8 @@ _Avoid_: Arrival time, local timestamp
|
|||||||
A change for a symbol that no connection delivered in time, which leaves its
|
A change for a symbol that no connection delivered in time, which leaves its
|
||||||
book silently wrong. A change only one connection missed is rescued, not
|
book silently wrong. A change only one connection missed is rescued, not
|
||||||
lost.
|
lost.
|
||||||
_Avoid_: Gap (implies sequence numbers)
|
_Avoid_: Gap (a skipped sequence number is one connection's, and may be no
|
||||||
|
loss at all)
|
||||||
|
|
||||||
**Resync**:
|
**Resync**:
|
||||||
Asking the exchange for a fresh full book of a symbol and replacing ours with
|
Asking the exchange for a fresh full book of a symbol and replacing ours with
|
||||||
|
|||||||
@@ -44,4 +44,5 @@ Trade-off: the routing step 0002 avoided is back, parsing moves into the
|
|||||||
connection thread, and the book thread's rules get subtler (held changes, a
|
connection thread, and the book thread's rules get subtler (held changes, a
|
||||||
ceiling, a chain declared broken). The design leans on an exchange that lets
|
ceiling, a chain declared broken). The design leans on an exchange that lets
|
||||||
the same change be recognised across connections: Polymarket's hash does;
|
the same change be recognised across connections: Polymarket's hash does;
|
||||||
Kalshi's `clock` looks as if it would (to confirm in M6).
|
on Kalshi a change's `clock` and its rank among the changes of that clock do
|
||||||
|
(recorded in M6, [sprint 06](../../roadmap/06.%20generic%20core%20and%20kalshi.md)).
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# 0004. The exchange seam: a codec and a loss rule per exchange
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Extension 3 asks for a generic service where "the only thing we need to
|
||||||
|
change is the encoding/decoding on the websocket layer". By M5 everything
|
||||||
|
lived in `src/polymarket/`, and M6 adds Kalshi. Most of the code is the same
|
||||||
|
for any exchange (the orchestrator, the connection loop, the book thread's
|
||||||
|
minute Snapshots, quiet watch and moves, the writer) or plainly per exchange
|
||||||
|
(URL, handshake, message formats, market listing). One part is neither: how
|
||||||
|
a loss is detected and how a symbol's two connections are merged
|
||||||
|
([0003](0003-first-one-wins.md)). Polymarket sends a hash of the whole book
|
||||||
|
with every change, which both verifies our book and says where a change from
|
||||||
|
either connection belongs. Kalshi has no hash: it numbers each connection's
|
||||||
|
messages and gives each market a clock. These are different mechanisms, not
|
||||||
|
two encodings of one.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
The seam has three parts. A **shared core** (`src/core/`) that names no
|
||||||
|
exchange. An **`Exchange` trait** for the websocket layer: the request, the
|
||||||
|
encoding of subscribe commands, the decoding of frames, the keepalive, and
|
||||||
|
the market fetcher. And, inside the book thread, a **rule supplied by the
|
||||||
|
exchange**: what a symbol's book does with an event from one of its
|
||||||
|
connections (a duplicate, the next change, held, a loss, a resync to ask
|
||||||
|
for), with its own per-symbol state.
|
||||||
|
|
||||||
|
The core understands of an event only the market it belongs to, which
|
||||||
|
routes it to a book thread, and whether it is a full book, which ends a move.
|
||||||
|
The event itself is the exchange's own type, handed to the exchange's rule
|
||||||
|
unread: the rule folds it into the symbol's book, which is the core's and
|
||||||
|
offers both ways a level changes (set to a quantity, moved by one).
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
Decode-only, as the brief's sentence reads, would put both loss rules in the
|
||||||
|
core, picked by exchange: the "generic" core would know each exchange's
|
||||||
|
quirks and a third exchange would mean editing it. A whole book thread per
|
||||||
|
exchange would keep the rules apart but write the minute Snapshots, the
|
||||||
|
quiet watch and the moves twice. With the rule behind the seam, the core is
|
||||||
|
really exchange-agnostic and the shell is written once. What the two rules
|
||||||
|
still share in shape (asking for a fresh book, and again when its connection
|
||||||
|
leaves or the minute passes) stays in each: their bookkeeping differs, and
|
||||||
|
Polymarket's was moved, not rewritten.
|
||||||
|
|
||||||
|
Trade-off: an exchange is more to write than a codec, and the seam is wider
|
||||||
|
than the brief asks. The book thread's shell and the rule also meet at more
|
||||||
|
points than a codec would (the minute's per-symbol questions, a connection
|
||||||
|
going down), so the rule's interface is the part most likely to move when a
|
||||||
|
third exchange arrives.
|
||||||
+95
-4
@@ -4,15 +4,17 @@
|
|||||||
A delta carries the **new total quantity** at a price level, not a change to it.
|
A delta carries the **new total quantity** at a price level, not a change to it.
|
||||||
`quantity: 400` means "this level is now 400," not "+400". Overwrite the level;
|
`quantity: 400` means "this level is now 400," not "+400". Overwrite the level;
|
||||||
never add. `quantity: 0` means delete the level. (Schema confirms: "absolute
|
never add. `quantity: 0` means delete the level. (Schema confirms: "absolute
|
||||||
available quantity".)
|
available quantity".) That is the output, and Polymarket's wire. Kalshi's wire
|
||||||
|
is relative (`delta_fp`: "+400"): the book adds it, and the Delta written
|
||||||
|
still carries the level's new total.
|
||||||
|
|
||||||
## A missed message means a wrong book: resync it
|
## A missed message means a wrong book: resync it
|
||||||
If a message is lost, the in-memory book is silently wrong and cannot be
|
If a message is lost, the in-memory book is silently wrong and cannot be
|
||||||
patched — throw it away and resync that symbol from a fresh full book. This is the
|
patched — throw it away and resync that symbol from a fresh full book. This is the
|
||||||
"completeness" requirement made mechanical. *How* a loss is detected depends on
|
"completeness" requirement made mechanical. *How* a loss is detected depends on
|
||||||
the exchange: a sequence-number gap where one exists (Kalshi, to verify in M6),
|
the exchange: a sequence-number gap where one exists (Kalshi: per connection,
|
||||||
something else where it doesn't — Polymarket has no sequence number, but its
|
below), something else where it doesn't — Polymarket has no sequence number,
|
||||||
book hash does the job (below).
|
but its book hash does the job (below).
|
||||||
The challenge brief prescribes no mechanism; it only asks for complete data.
|
The challenge brief prescribes no mechanism; it only asks for complete data.
|
||||||
|
|
||||||
## Polymarket market channel (observed 2026-11-04)
|
## Polymarket market channel (observed 2026-11-04)
|
||||||
@@ -222,6 +224,95 @@ connection recycled every 60 s (sprint 05).
|
|||||||
20 s): a check reaching back 5 minutes misses some. 195,130 markets were
|
20 s): a check reaching back 5 minutes misses some. 195,130 markets were
|
||||||
live that evening.
|
live that evening.
|
||||||
|
|
||||||
|
## Kalshi's websocket (demo, observed 2026-11-11)
|
||||||
|
|
||||||
|
From Kalshi's docs and four recordings of the demo with a throwaway recorder
|
||||||
|
(`../kalshi-spike/`, outside the repo). Production is untried where a number
|
||||||
|
is the demo's.
|
||||||
|
|
||||||
|
- **Hosts:** production `wss://external-api-ws.kalshi.com/trade-api/ws/v2`
|
||||||
|
and `https://external-api.kalshi.com/trade-api/v2`; demo
|
||||||
|
`wss://external-api-ws.demo.kalshi.co/trade-api/ws/v2` and
|
||||||
|
`https://external-api.demo.kalshi.co/trade-api/v2`. The older
|
||||||
|
`api.elections.kalshi.com` and `demo-api.kalshi.co` still answer.
|
||||||
|
- **Auth:** the websocket handshake carries `KALSHI-ACCESS-KEY` (the key id),
|
||||||
|
`KALSHI-ACCESS-TIMESTAMP` (ms) and `KALSHI-ACCESS-SIGNATURE`: base64 of the
|
||||||
|
Ed25519 signature of `<timestamp>GET/trade-api/ws/v2`. Without them: 401.
|
||||||
|
The market listing needs no key. Keys are created per environment
|
||||||
|
(Account → Profile → API Keys); the private key is a PKCS#8 PEM shown once.
|
||||||
|
- **Subscribe:** `{"id":1,"cmd":"subscribe","params":{"channels":
|
||||||
|
["orderbook_delta"],"market_tickers":[...]}}` is answered
|
||||||
|
`{"type":"subscribed","id":1,"msg":{"channel":"orderbook_delta","sid":1}}`,
|
||||||
|
then one `orderbook_snapshot` per market. A connection has one such
|
||||||
|
subscription: a second `subscribe`, or `{"cmd":"update_subscription",
|
||||||
|
"params":{"sids":[1],"market_tickers":[...],"action":"add_markets"}}`, adds
|
||||||
|
markets and sends their books; `delete_markets` stops a market's changes;
|
||||||
|
`get_snapshot` sends books again (carrying the command's `id`). Each `ok`
|
||||||
|
answer lists every market of the subscription unless `skip_ticker_ack:
|
||||||
|
true` is sent. An empty ticker list is error 14.
|
||||||
|
- **`orderbook_snapshot`:** `{"type","sid","seq","sending_ts_ms","msg":
|
||||||
|
{"market_ticker","market_id","clock","yes_dollars_fp":[["0.4800","80.00"],
|
||||||
|
...],"no_dollars_fp":[...]}}`. A side with no level is omitted. Bids only:
|
||||||
|
YES bids and NO bids. No time of its own.
|
||||||
|
- **`orderbook_delta`:** `msg` is `{"market_ticker","market_id","clock",
|
||||||
|
"price_dollars":"0.9600","delta_fp":"-54.00","side":"yes"|"no","ts",
|
||||||
|
"ts_ms"}`: a signed change of the level's quantity. Folding them gave the
|
||||||
|
exchange's book every time (1,352 `get_snapshot` books equal to ours, no
|
||||||
|
level below zero).
|
||||||
|
- **`use_yes_price: true`** in the subscribe gives NO levels at `1 − q`, the
|
||||||
|
YES ask they are (books and changes alike); the default, `false`, gives
|
||||||
|
`q` and is announced to flip.
|
||||||
|
- **`seq`** is contiguous per connection across all its markets, counting
|
||||||
|
books, changes, `ok` and `error` (not `subscribed`): no gap in 343,000
|
||||||
|
messages.
|
||||||
|
- **`clock`** is per market, the same on every connection, never going back,
|
||||||
|
and shared by 2 to 4 changes for 12% of its values (some identical in
|
||||||
|
content). A book's clock is its last change's: none of 1,378 books was
|
||||||
|
followed by a change of its own clock or an older one. Two connections
|
||||||
|
deliver a market's changes in the same order (2,643 changes on 26 markets),
|
||||||
|
the second copy 2 ms later at the median, 344 ms at worst.
|
||||||
|
- **No limit met per connection:** 30,000 markets in one subscribe message,
|
||||||
|
or in 30 messages 20 ms apart; every book within 1.7 s; ~1,560 changes and
|
||||||
|
566 KB a second. The docs give 200 connections per user and unnumbered
|
||||||
|
limits (errors 26 and 27).
|
||||||
|
- **Keepalive:** Kalshi sends a Ping frame (`heartbeat`) to a connection it
|
||||||
|
has sent nothing else for 10 s; a busy connection gets none. tungstenite
|
||||||
|
answers it by itself.
|
||||||
|
- **An unopened market can be subscribed:** its book is empty with `clock: 0`
|
||||||
|
and `market_id: ""`, and its changes flow from its first order after the
|
||||||
|
opening, with no new subscribe.
|
||||||
|
- **Time:** changes arrived 88 to 174 ms after their `ts_ms` (median 90),
|
||||||
|
from the server.
|
||||||
|
- **Listing:** `GET /markets?status=open&mve_filter=exclude&limit=1000`,
|
||||||
|
paged by `cursor`: 177,966 open markets on production, about 4 minutes
|
||||||
|
unauthenticated. `status=open` returns `active` markets, `unopened`
|
||||||
|
`initialized` ones, `closed` also `determined` ones. `min_created_ts`
|
||||||
|
(with no status: open and unopened alike), `status=closed&min_close_ts`
|
||||||
|
and `tickers=a,b` all work without a key. Markets open at their
|
||||||
|
`open_time` (none of the demo's 20,504 unopened ones was overdue), often
|
||||||
|
long after creation: of 15,000 open markets, 82% opened more than a minute
|
||||||
|
after, 19% more than 15 minutes.
|
||||||
|
- **Commands on an open subscription** (tried 2026-11-11): `skip_ticker_ack`
|
||||||
|
works on `update_subscription` too (the `ok` comes with an empty `msg`);
|
||||||
|
a subscription emptied by `delete_markets` stays and takes markets again,
|
||||||
|
with their books; subscribing a market already subscribed answers `ok` and
|
||||||
|
sends no book, so a fresh book takes `delete_markets` first; deleting an
|
||||||
|
unknown ticker answers `ok`.
|
||||||
|
- **A market's close sends its subscribers a fresh book**, unasked and
|
||||||
|
without the changes that lead to it (20-minute demo run, 2026-11-11: 54
|
||||||
|
markets closing at 20:00 UTC, each a newer, different book at 20:00:01 to
|
||||||
|
20:00:04). Taken as a book past ours that differs, it is logged as a loss;
|
||||||
|
the Snapshot is right.
|
||||||
|
- **A closed market answers like an unopened one**: an empty book, `clock:
|
||||||
|
0`, `market_id: ""`.
|
||||||
|
- **Handshakes are rate limited**: opening 110 connections at once, 54
|
||||||
|
handshakes were refused with 429; each got in on its retry a second later.
|
||||||
|
- **Production lists unopened markets past their `open_time`** (seen
|
||||||
|
2026-11-11: `initialized` 16 minutes after it); the demo had none.
|
||||||
|
- **The demo's size:** 246,365 markets open or opening within two hours
|
||||||
|
(2026-11-11, 19:29 UTC).
|
||||||
|
- **Maintenance (docs):** Thursdays 3:00 to 5:00 AM ET; connections drop.
|
||||||
|
|
||||||
## Clock accuracy bounds time accuracy
|
## Clock accuracy bounds time accuracy
|
||||||
`received_at` comes from the local clock, so it carries the clock's error. On
|
`received_at` comes from the local clock, so it carries the clock's error. On
|
||||||
2026-11-04 the dev machine ran ~30 ms slow (`sntp`: +30.6 ms ± 13 ms), which
|
2026-11-04 the dev machine ran ~30 ms slow (`sntp`: +30.6 ms ± 13 ms), which
|
||||||
|
|||||||
+43
-21
@@ -46,6 +46,9 @@ The interacting decisions it is made of:
|
|||||||
chosen from its market's id; an orchestrator decides which connections
|
chosen from its market's id; an orchestrator decides which connections
|
||||||
carry each market ([ADR 0002](adr/0002-lanes-and-orchestrator.md), its
|
carry each market ([ADR 0002](adr/0002-lanes-and-orchestrator.md), its
|
||||||
lanes superseded by [ADR 0003](adr/0003-first-one-wins.md) in sprint 04).
|
lanes superseded by [ADR 0003](adr/0003-first-one-wins.md) in sprint 04).
|
||||||
|
- **The exchange seam:** settled in sprint 06 — the pipeline is the shared
|
||||||
|
core; an exchange supplies its websocket layer and, inside the book
|
||||||
|
thread, its loss rule ([ADR 0004](adr/0004-exchange-seam.md)).
|
||||||
- **Backpressure:** bounded vs unbounded channels — the core correctness/perf
|
- **Backpressure:** bounded vs unbounded channels — the core correctness/perf
|
||||||
tension. Drop-on-full loses data (violates completeness); block adds latency
|
tension. Drop-on-full loses data (violates completeness); block adds latency
|
||||||
and fills socket buffers. This decision is the bet in miniature.
|
and fills socket buffers. This decision is the bet in miniature.
|
||||||
@@ -61,13 +64,16 @@ The output's value is completeness + time-accuracy, so whatever the topology, it
|
|||||||
must guarantee:
|
must guarantee:
|
||||||
|
|
||||||
1. **Receive time stamped at the socket edge**, before any queue or processing.
|
1. **Receive time stamped at the socket edge**, before any queue or processing.
|
||||||
2. **A per-symbol book folded from absolute deltas** (overwrite the level;
|
2. **A per-symbol book folded from the exchange's changes**, written as
|
||||||
quantity 0 deletes it).
|
absolute Deltas (the level's new quantity; 0 deletes it). Polymarket's
|
||||||
|
changes are already absolute; Kalshi's are relative and are added to the
|
||||||
|
level.
|
||||||
3. **Loss detection → resync** (a lost message means the book is silently
|
3. **Loss detection → resync** (a lost message means the book is silently
|
||||||
corrupt; replacing it with a fresh full book is the only safe response). How
|
corrupt; replacing it with a fresh full book is the only safe response). How
|
||||||
a loss is detected depends on the exchange: a sequence gap where one exists;
|
a loss is detected depends on the exchange: on Polymarket, recomputing its
|
||||||
on Polymarket, which has none, recomputing its book hash after every change
|
book hash after every change (settled in M2; the formula is in
|
||||||
(settled in M2; the formula is in [notes.md](notes.md)).
|
[notes.md](notes.md)); on Kalshi, a skipped sequence number on a
|
||||||
|
connection, which reconnects it (settled in M6).
|
||||||
|
|
||||||
These are proven first, before redundancy or scale.
|
These are proven first, before redundancy or scale.
|
||||||
|
|
||||||
@@ -112,8 +118,9 @@ Their operational form, for writing code, is in the project
|
|||||||
- **Scope of markets:** regular markets only; MVEs excluded; websocket exchanges
|
- **Scope of markets:** regular markets only; MVEs excluded; websocket exchanges
|
||||||
only.
|
only.
|
||||||
- **Output destination** (settled in sprints 01 and 02): `AlignedSbeWriter`
|
- **Output destination** (settled in sprints 01 and 02): `AlignedSbeWriter`
|
||||||
takes a file path; the program's only argument is that path, default
|
takes a file path; the program takes the exchange (`polymarket`, `kalshi`
|
||||||
`/dev/stdout`, and logs go to stderr. No rotation: one output per run
|
or `kalshi-demo`, required since sprint 06: one exchange per process), then
|
||||||
|
that path, default `/dev/stdout`, and logs go to stderr. No rotation: one output per run
|
||||||
(rotation can't apply to stdout; a consumer that wants files rotates them).
|
(rotation can't apply to stdout; a consumer that wants files rotates them).
|
||||||
Chunks must be 64 MiB: `AlignedSbeReader::open` reads nothing else.
|
Chunks must be 64 MiB: `AlignedSbeReader::open` reads nothing else.
|
||||||
- **Crash ceiling** (accepted in sprint 02): the writer holds the current
|
- **Crash ceiling** (accepted in sprint 02): the writer holds the current
|
||||||
@@ -154,7 +161,8 @@ schema debate). As settled in sprints 04 and 05:
|
|||||||
- The **orchestrator** owns what is tracked (each market's two connections,
|
- The **orchestrator** owns what is tracked (each market's two connections,
|
||||||
the timers) and decides: placing, moving, splitting, and recycling one
|
the timers) and decides: placing, moving, splitting, and recycling one
|
||||||
connection at a time, about hourly. It never blocks on HTTP.
|
connection at a time, about hourly. It never blocks on HTTP.
|
||||||
- The two **fetchers** only page Gamma on request and stream what they read.
|
- The two **fetchers** only page the exchange's market listing on request
|
||||||
|
and stream what they read.
|
||||||
- A **connection** reads, stamps the receive time, parses, and routes each
|
- A **connection** reads, stamps the receive time, parses, and routes each
|
||||||
event to the book thread of its market. Every symbol is on two connections,
|
event to the book thread of its market. Every symbol is on two connections,
|
||||||
chosen independently. One silent for 30 s reconnects.
|
chosen independently. One silent for 30 s reconnects.
|
||||||
@@ -170,6 +178,14 @@ Settled: OS threads, shared-nothing, no mutexes
|
|||||||
first one wins ([ADR 0003](adr/0003-first-one-wins.md)). Details:
|
first one wins ([ADR 0003](adr/0003-first-one-wins.md)). Details:
|
||||||
[sprint 04](../roadmap/04.%20redundant%20connections.md).
|
[sprint 04](../roadmap/04.%20redundant%20connections.md).
|
||||||
|
|
||||||
|
The pipeline is the same for every exchange. What an exchange supplies
|
||||||
|
([ADR 0004](adr/0004-exchange-seam.md)): the websocket layer of a connection
|
||||||
|
(request, encoding of commands, decoding of frames, keepalive), the fetchers,
|
||||||
|
and the rule a book thread applies to an event (duplicate, next, held, loss).
|
||||||
|
On Polymarket that rule is the hash chain above; on Kalshi a change is the
|
||||||
|
next one when its clock and rank pass the book's, and a held change doesn't
|
||||||
|
exist.
|
||||||
|
|
||||||
## Data model, first sketch
|
## Data model, first sketch
|
||||||
|
|
||||||
Supporting detail (mostly Claude's to settle), serving the pipeline above.
|
Supporting detail (mostly Claude's to settle), serving the pipeline above.
|
||||||
@@ -178,27 +194,33 @@ Not a DB — an in-memory domain plus the fixed SBE output schema. Main entities
|
|||||||
|
|
||||||
- **Symbol** (from discovery): one order book, with its trading status and any
|
- **Symbol** (from discovery): one order book, with its trading status and any
|
||||||
price/tick metadata the exchange gives. On Polymarket that is a token, not a
|
price/tick metadata the exchange gives. On Polymarket that is a token, not a
|
||||||
market: a market has two (Yes / No), each with its own, mirrored book.
|
market: a market has two (Yes / No), each with its own, mirrored book. On
|
||||||
- **NormalizedEvent** (internal, exchange-agnostic): the thing each adapter
|
Kalshi it is the market: its bids for Yes are the bids, its bids for No at
|
||||||
produces so the core never sees exchange formats. Either a full snapshot or a
|
`q` the asks at `1 − q`.
|
||||||
single delta, each carrying `symbol`, `exchange_timestamp`, `received_at`,
|
- **Event** (settled in sprint 06, in place of the `NormalizedEvent` first
|
||||||
whatever the exchange offers for loss detection (a sequence number, a book
|
sketched): each exchange's own type. The core reads only the market it
|
||||||
hash), and payload (side/price/quantity, or the full book).
|
belongs to and whether it is a full book; the exchange's rule reads the
|
||||||
|
rest (side, price, quantity, and what the exchange offers for loss
|
||||||
|
detection: a book hash, a sequence number, a clock) and folds it into the
|
||||||
|
symbol's book, setting a level's quantity (Polymarket) or moving it by a
|
||||||
|
signed amount (Kalshi).
|
||||||
- **Book** (one per symbol): bids and asks as ordered price→quantity maps, plus
|
- **Book** (one per symbol): bids and asks as ordered price→quantity maps, plus
|
||||||
the last loss-detection marker seen and last exchange timestamp. It is fed
|
the last loss-detection marker seen and last exchange timestamp. It is fed
|
||||||
by the symbol's two connections, first one wins.
|
by the symbol's two connections, first one wins.
|
||||||
- **Connection** (one websocket): the set of symbols it carries. Live markets
|
- **Connection** (one websocket): the set of symbols it carries. Live markets
|
||||||
are placed on connections by `conditionId`, both tokens together;
|
are placed on connections by market, all its symbols together;
|
||||||
Polymarket's lifecycle signal is Gamma (settled in sprint 03: a market is
|
Polymarket's lifecycle signal is Gamma (settled in sprint 03: a market is
|
||||||
live while active, not closed, with an order book and accepting orders).
|
live while active, not closed, with an order book and accepting orders),
|
||||||
|
Kalshi's its market listing (sprint 06: open, or opening within three
|
||||||
|
hours, tracked ahead so the book is there from its first order).
|
||||||
|
|
||||||
Output schema is fixed by the SBE schema: `Snapshot` and `PriceLevelUpdate`.
|
Output schema is fixed by the SBE schema: `Snapshot` and `PriceLevelUpdate`.
|
||||||
|
|
||||||
The debate lives in the **NormalizedEvent** shape (both exchanges must map
|
Redundancy was settled in sprint 04: no source is trusted, the first copy of
|
||||||
cleanly onto it). Redundancy was settled in sprint 04: no source is trusted,
|
each change wins ([ADR 0003](adr/0003-first-one-wins.md)). On Kalshi the
|
||||||
the first copy of each change wins ([ADR 0003](adr/0003-first-one-wins.md)).
|
first copy is recognised by clock and rank (sprint 06).
|
||||||
|
|
||||||
## Open questions
|
## Open questions
|
||||||
|
|
||||||
- **Loss detection on Kalshi.** Its sequence semantics: checked in M6.
|
None. (Loss detection on Kalshi was the last: settled in M6, by its
|
||||||
(Polymarket: settled in M2, by its book hash.)
|
per-connection sequence number.)
|
||||||
+18
-5
@@ -66,12 +66,16 @@ abstraction is *extracted* once a second exchange exists, not designed up front.
|
|||||||
than sprint 04's; quiet subscriptions are rare (26 resubscribed, none
|
than sprint 04's; quiet subscriptions are rare (26 resubscribed, none
|
||||||
quiet again) and no connection fell silent (sprint 05's Outcome).
|
quiet again) and no connection fell silent (sprint 05's Outcome).
|
||||||
|
|
||||||
6. **Extension 3 — generic core + Kalshi adapter** — *not started*
|
6. **Extension 3 — generic core + Kalshi adapter** — *done (2026-11-11)*
|
||||||
Extract the exchange-agnostic core (the `NormalizedEvent` seam) from the
|
The exchange-agnostic core extracted from the working Polymarket code,
|
||||||
working Polymarket code, then add a Kalshi adapter so both exchanges run
|
and Kalshi added through it. Each exchange supplies its websocket layer
|
||||||
through the same core. The abstraction is extracted from two concretes here,
|
and its loss rule ([ADR 0004](adr/0004-exchange-seam.md)); on Kalshi,
|
||||||
not before. Requires a Kalshi API key.
|
first one wins by clock and rank.
|
||||||
|
Sprint: [../roadmap/06. generic core and kalshi.md](../roadmap/06.%20generic%20core%20and%20kalshi.md).
|
||||||
*Ends:* both Polymarket and Kalshi work through one generic core.
|
*Ends:* both Polymarket and Kalshi work through one generic core.
|
||||||
|
Result: 10-minute live runs of each, reconciled; Polymarket's losses no
|
||||||
|
higher than sprint 05's; Kalshi on its demo (about 245,000 markets on 110
|
||||||
|
connections), production not run for lack of a key (sprint 06's Outcome).
|
||||||
|
|
||||||
7. **Extension 4 — latency experiments + writeup** — *not started*
|
7. **Extension 4 — latency experiments + writeup** — *not started*
|
||||||
Experiment with faster connections (first-after-maintenance, connection
|
Experiment with faster connections (first-after-maintenance, connection
|
||||||
@@ -125,3 +129,12 @@ abstraction is *extracted* once a second exchange exists, not designed up front.
|
|||||||
- Fewer Snapshots for unchanged books (the brief asks for every symbol every
|
- Fewer Snapshots for unchanged books (the brief asks for every symbol every
|
||||||
minute: ~130 MB a minute at full coverage).
|
minute: ~130 MB a minute at full coverage).
|
||||||
- Re-reading a market's `orderMinSize` / `negRisk` if they can change.
|
- Re-reading a market's `orderMinSize` / `negRisk` if they can change.
|
||||||
|
- A Kalshi production run, once a production key exists.
|
||||||
|
- Retrying a failed rescan sooner than the next hourly one (`maybe`: Kalshi's
|
||||||
|
three-hour opening window covers one failure).
|
||||||
|
- One place for the two rules' resync bookkeeping (asking for a fresh book,
|
||||||
|
again on leave and at the minute), in the shell (`maybe`: it means
|
||||||
|
rewriting Polymarket's).
|
||||||
|
- Telling Kalshi's book at a market's close, and a connection's book merely
|
||||||
|
ahead of the other's stream, from a real loss (sprint 06 logs all three as
|
||||||
|
a loss; the Snapshot is right either way).
|
||||||
@@ -0,0 +1,469 @@
|
|||||||
|
# 06. Generic core and Kalshi
|
||||||
|
|
||||||
|
**Status: done (2026-11-11)**
|
||||||
|
|
||||||
|
The collector does the hard things on one exchange. This sprint pulls the
|
||||||
|
exchange-agnostic core out of `src/polymarket/`, then adds Kalshi beside
|
||||||
|
Polymarket so both run through that core: Extension 3 of the brief, and the
|
||||||
|
exchange the brief names. The brief asks that only "the encoding/decoding on
|
||||||
|
the websocket layer" change per exchange; that holds for everything except
|
||||||
|
how a loss is detected and how a symbol's two connections are merged, which
|
||||||
|
differ in kind between the two, so each exchange also supplies that rule.
|
||||||
|
Design: [ADR 0004](../docs/adr/0004-exchange-seam.md),
|
||||||
|
[../docs/project.md](../docs/project.md) (pipeline, unchanged),
|
||||||
|
[ADR 0003](../docs/adr/0003-first-one-wins.md),
|
||||||
|
[../GLOSSARY.md](../GLOSSARY.md) (market, symbol).
|
||||||
|
|
||||||
|
## Kalshi, measured (2026-11-11)
|
||||||
|
|
||||||
|
From docs.kalshi.com, the public listing, and four recordings of the demo's
|
||||||
|
websocket with a throwaway recorder (`../kalshi-spike/`, outside the repo;
|
||||||
|
up to six connections, 200 to 240 s). The details go to `docs/notes.md`.
|
||||||
|
Production may differ where a number is the demo's.
|
||||||
|
|
||||||
|
- **The websocket needs a signed handshake; the listing doesn't.**
|
||||||
|
`GET /markets` answers 200 unauthenticated on production and the demo; an
|
||||||
|
unauthenticated websocket handshake gets 401 on both. Signed with `ring`
|
||||||
|
from the key files, the demo answered 101.
|
||||||
|
- **Size:** 177,966 open markets without combos (`mve_filter=exclude`) on
|
||||||
|
production, one book each. At most 200 connections per user (docs).
|
||||||
|
- **One book per market, bids only:** YES bids and NO bids. Prices are
|
||||||
|
dollar strings with 4 decimals, quantities strings with 2. With
|
||||||
|
`use_yes_price: true` a NO bid at `q` comes at `1 − q`, the YES ask it
|
||||||
|
is; the default (`false`, announced to flip) gives `q`.
|
||||||
|
- **A change is relative:** `delta_fp` is a signed change of a level's
|
||||||
|
quantity. Folding them gave the exchange's book every time: 1,352 books
|
||||||
|
asked for mid-stream (`get_snapshot`) all equalled ours, no level below
|
||||||
|
zero.
|
||||||
|
- **`seq` is contiguous per connection** across all its markets, counting
|
||||||
|
books, changes and the `ok` / `error` answers to commands (`subscribed`
|
||||||
|
has none): no gap in 343,000 messages on one connection.
|
||||||
|
- **`clock` is per market and the same on every connection, but several
|
||||||
|
changes share one**: 12% of clock values carried 2 to 4 changes, some
|
||||||
|
identical in content. Two connections on the same 26 changing markets
|
||||||
|
delivered the same changes in the same order (2,643 and 2,644, the
|
||||||
|
difference at the edges). A book's clock is that of the last change it
|
||||||
|
holds: none of 1,378 books was followed by a change of its own clock or
|
||||||
|
an older one. So a change is named by its clock and its rank among the
|
||||||
|
changes of that clock.
|
||||||
|
- **No limit met per connection:** 30,000 markets subscribed on one
|
||||||
|
connection, in one message or in 30 messages 20 ms apart, all books within
|
||||||
|
1.7 s, ~1,560 changes and 566 KB a second, no error. 52 `get_snapshot`
|
||||||
|
commands 3 s apart, none refused.
|
||||||
|
- **Adding markets** (a second `subscribe` or `add_markets`) sends their
|
||||||
|
books; `delete_markets` stops a market's changes. Each answer lists every
|
||||||
|
market of the subscription unless `skip_ticker_ack` is sent.
|
||||||
|
- **Kalshi pings only a connection it has sent nothing to for 10 s** (a
|
||||||
|
Ping frame, `heartbeat`): an idle connection got one every 10 s, a busy
|
||||||
|
one none.
|
||||||
|
- **A book has no time of its own**, only the frame's `sending_ts_ms`; a
|
||||||
|
change has `ts_ms`. Changes arrived 88 to 174 ms after it (median 90).
|
||||||
|
- **An unopened market can be subscribed:** it answers an empty book
|
||||||
|
(`clock: 0`, `market_id: ""`), and its changes flow from its first order
|
||||||
|
after the opening, with no new subscribe (two markets subscribed 107 s
|
||||||
|
ahead; a connection subscribing after the opening saw the same).
|
||||||
|
- **Markets are created ahead of their opening**, and the listing filters on
|
||||||
|
creation time, not opening time. Of the first 15,000 open markets listed
|
||||||
|
on production (not a random sample), 82% opened more than a minute after
|
||||||
|
creation and 19% more than 15 minutes after; at least 40,000 were
|
||||||
|
unopened, 755 of them opening within the hour. In 10 minutes, 352 markets
|
||||||
|
were created (45 still unopened) and 25 closed. None of the demo's 20,504
|
||||||
|
unopened markets was past its `open_time`: they open on time.
|
||||||
|
- **The demo** has its own hosts and keys and the same protocol; its books
|
||||||
|
are thin (of its first 60,000 open markets, 540 had bids on both sides,
|
||||||
|
by the research probe), but its bots keep ~1,500 changes a second going.
|
||||||
|
|
||||||
|
## The seam
|
||||||
|
|
||||||
|
Three parts ([ADR 0004](../docs/adr/0004-exchange-seam.md)):
|
||||||
|
|
||||||
|
```
|
||||||
|
src/core/ orchestrator, connection, book thread, the Exchange trait
|
||||||
|
src/polymarket/ messages, Gamma, the hash rule
|
||||||
|
src/kalshi/ messages, signing, the market listing, the sequence and clock rule
|
||||||
|
```
|
||||||
|
|
||||||
|
**Shared (`src/core/`)**, moved from `src/polymarket/` as it is:
|
||||||
|
|
||||||
|
- the orchestrator (`orchestrator.rs`): tracking, placing, balancing,
|
||||||
|
splitting, recycling;
|
||||||
|
- the connection loop (`socket.rs`): connect with a timeout, read, stamp,
|
||||||
|
route, reconnect, silent after 30 s, recycle, clean close;
|
||||||
|
- the book thread's shell (from `feed.rs`): markets and symbols, connections
|
||||||
|
up and down, the minute's Snapshots, load reports, the quiet watch, moves,
|
||||||
|
the minute line;
|
||||||
|
- `main.rs`'s writer, `book.rs`, `reconcile`, `cause.rs` (already neutral:
|
||||||
|
`cause::parse` looks for `Snapshot of` anywhere in a line).
|
||||||
|
|
||||||
|
**Per exchange, one trait (`Exchange`)**, chosen at compile time (the
|
||||||
|
orchestrator, connections and book threads are generic over it; `main`
|
||||||
|
picks by name):
|
||||||
|
|
||||||
|
- the websocket request (URL and headers: Kalshi signs it) and the messages
|
||||||
|
sent on connect;
|
||||||
|
- encoding `Subscribe`, `Unsubscribe` and `Resubscribe` into frames, with
|
||||||
|
the connection's own state (Kalshi's subscription id);
|
||||||
|
- decoding a frame into events, each with the market that routes it, and
|
||||||
|
saying when the connection itself lost something (Kalshi's `seq`);
|
||||||
|
- the keepalive we send, if any (Polymarket's `PING` text; none on Kalshi,
|
||||||
|
whose Ping frames tungstenite answers by itself);
|
||||||
|
- the fetcher: Gamma's `Request` / `Finding` interface
|
||||||
|
(`gamma.rs:26-47`) is kept as the core's, and each exchange implements it.
|
||||||
|
|
||||||
|
Not per exchange after all (see Decisions): the close that means "too slow"
|
||||||
|
(1013, websocket's own "try again later"), and the per-connection numbers.
|
||||||
|
|
||||||
|
**Per exchange, inside the book thread (its rule):** what a symbol's book
|
||||||
|
does with an event from one of its connections: a duplicate, the next change,
|
||||||
|
held, a loss, a resync to ask for. The core hands it the symbol's book, which
|
||||||
|
connections are up, and a place to put effects; the rule owns its own
|
||||||
|
per-symbol state.
|
||||||
|
|
||||||
|
**What the core understands of an event**: the market it belongs to, which
|
||||||
|
routes it, and whether it is a full book, which ends a move. The event is
|
||||||
|
the exchange's own type, handed to its rule unread, loss markers (hash,
|
||||||
|
`seq`, `clock`) included. The rule folds it into the symbol's book, the
|
||||||
|
core's: a level is set to a quantity (Polymarket) or moved by a signed
|
||||||
|
amount (Kalshi), and a Delta always carries the absolute quantity.
|
||||||
|
|
||||||
|
**A market** is `id`, its symbols, and the exchange's own details
|
||||||
|
(Polymarket: `min_order_size`, `neg_risk`; Kalshi: none). Its book thread
|
||||||
|
comes from a hash of the id (`std`'s `DefaultHasher`; it parsed the
|
||||||
|
conditionId's hex digits, which a ticker doesn't have).
|
||||||
|
|
||||||
|
## Extracting the core
|
||||||
|
|
||||||
|
A move, not a rewrite: Polymarket's rules, constants, log lines and tests
|
||||||
|
stay as they are, and every existing test stays green at each step.
|
||||||
|
|
||||||
|
The cut doesn't follow today's functions. In `feed.rs`, `Symbol` holds both
|
||||||
|
halves (shell: `book`, `connections`, `watches`, `quiet_on`, `last_written`;
|
||||||
|
rule: `state`, `recent_changes`, `held`, the hash inputs), and three
|
||||||
|
functions interleave them:
|
||||||
|
|
||||||
|
- `tick` (`feed.rs:826`): the shell's minute loop (loads, quiet judging,
|
||||||
|
Snapshots) calls the rule per symbol in an order that matters (count old
|
||||||
|
copies, "may this symbol be judged quiet", "does it have a book", ask
|
||||||
|
again for a fresh book).
|
||||||
|
- `on_book` (`feed.rs:1046`): ending a move and spotting a reconnect's
|
||||||
|
opening book are the shell's; judging the book is the rule's, and it
|
||||||
|
writes Snapshots and cause lines.
|
||||||
|
- `on_price_change` (`feed.rs:1248`): counting the change for the quiet
|
||||||
|
watch is the shell's, the rest the rule's.
|
||||||
|
|
||||||
|
So `Symbol` becomes the shell's fields plus the rule's state, and the rule
|
||||||
|
gets named hooks: an event, a connection down, a subscription leaving, the
|
||||||
|
ceilings' timer, and the minute's questions above. The minute line keeps its
|
||||||
|
text: the shell prints its counts, the rule adds its own.
|
||||||
|
|
||||||
|
The tests move with the code they exercise. Polymarket's 43 `feed.rs` tests
|
||||||
|
drive the shell through hash-correct frames, so they stay Polymarket's and
|
||||||
|
keep covering the shell; Kalshi's replay tests cover it a second time. No
|
||||||
|
stand-in rule is written for the shell alone.
|
||||||
|
|
||||||
|
## Kalshi
|
||||||
|
|
||||||
|
**Symbol.** The market, named by its ticker. YES bids are bids; a NO bid at
|
||||||
|
`q` is an ask at `1 − q`, exact in fixed point.
|
||||||
|
|
||||||
|
**Key.** `KALSHI_KEY_ID_FILE` and `KALSHI_PRIVATE_KEY_FILE` name two files
|
||||||
|
(on the server, in `/run/secrets`). Read once at startup; a missing or
|
||||||
|
unreadable one stops the run with the variable's name. Ed25519 only (Kalshi's
|
||||||
|
default). Each connect signs `<ms since epoch>GET/trade-api/ws/v2` and sends
|
||||||
|
`KALSHI-ACCESS-KEY`, `KALSHI-ACCESS-TIMESTAMP`, `KALSHI-ACCESS-SIGNATURE`
|
||||||
|
(base64). The key is never logged. The listing is read unauthenticated.
|
||||||
|
|
||||||
|
**Command line.** `engineering-challenge <polymarket|kalshi|kalshi-demo>
|
||||||
|
[OUTPUT]`, the exchange required; one exchange per process. `kalshi` and
|
||||||
|
`kalshi-demo` differ only by their hosts.
|
||||||
|
|
||||||
|
**A skipped `seq`.** The connection thread checks it (a book thread sees only
|
||||||
|
its own markets' share). On a gap the connection closes and reconnects, like
|
||||||
|
a silent one: every book it carries comes fresh, and each symbol's other
|
||||||
|
connection covers meanwhile.
|
||||||
|
|
||||||
|
**Merging two connections.** First one wins, by clock and rank. A symbol's
|
||||||
|
book stands at a position: the clock of its last change and how many changes
|
||||||
|
of that clock it holds. Each connection has its own position for the symbol,
|
||||||
|
counted the same way from the book it delivered. A change is applied and
|
||||||
|
written when its connection's position passes the book's, and is a duplicate
|
||||||
|
otherwise. A connection counts for a symbol only once it has delivered that
|
||||||
|
symbol's book since it came up; before that its changes are dropped. Nothing
|
||||||
|
is held: two connections deliver a market's changes in the same order, so
|
||||||
|
the first copy of each is always the next one.
|
||||||
|
|
||||||
|
**A fresh book** (on subscribe, reconnect or resubscribe) for a symbol that
|
||||||
|
has one: older than ours, it only sets its connection's position. At or past
|
||||||
|
our clock and equal to ours, the same. At or past our clock and different,
|
||||||
|
it replaces ours and is written as a Snapshot, with its cause line: the
|
||||||
|
other connection is behind or lost something, and we can't tell which.
|
||||||
|
|
||||||
|
**A change ahead of the book.** After a resync's answer the book can stand
|
||||||
|
behind what another connection has delivered. That connection's next change
|
||||||
|
would skip the changes between, so a change is taken only when its
|
||||||
|
connection's position before it was at or behind the book: the stream that
|
||||||
|
answered carries the book forward, and the other counts again once the book
|
||||||
|
reaches it. (From the review: without it, the changes between were dropped
|
||||||
|
as duplicates, with no loss logged.)
|
||||||
|
|
||||||
|
**A book that can't be right.** A level driven below zero by a change means
|
||||||
|
our book is wrong: the symbol is resynced (`Resubscribe` on one connection)
|
||||||
|
and the fresh book written as a Snapshot, with its cause line.
|
||||||
|
|
||||||
|
**Quiet, silent, recycle, split, balance.** The core's, unchanged. Silent is
|
||||||
|
30 s without a frame, Kalshi's Pings included (it pings a connection it has
|
||||||
|
nothing else to send to). `Resubscribe` is `delete_markets` then a
|
||||||
|
`subscribe`, which adds to the connection's one subscription as
|
||||||
|
`add_markets` would and needs no subscription id. Nothing on Kalshi answers
|
||||||
|
to Polymarket's 1013, so nothing splits. Errors 10 and 25 (a channel error,
|
||||||
|
a subscription buffer overflow: terminal, by Kalshi's docs) reconnect the
|
||||||
|
connection like a skipped `seq`; other errors are logged. At the minute, a
|
||||||
|
symbol no connection delivers (no book yet, or none after a reconnect) is
|
||||||
|
asked for again, and gets no Snapshot until one delivers it.
|
||||||
|
|
||||||
|
**Numbers.** Polymarket's: 5,000 symbols and 300 KB/s per connection, which
|
||||||
|
puts production's 178,000 markets, twice, on about 80 connections, under the
|
||||||
|
cap of 200. Subscribes go in messages of 500 tickers with `use_yes_price:
|
||||||
|
true` and `skip_ticker_ack: true`. A Snapshot's exchange time is its
|
||||||
|
frame's `sending_ts_ms`.
|
||||||
|
|
||||||
|
**Discovery.** The fetcher scans open and unopened markets at startup and
|
||||||
|
hourly (`status=open` / `status=unopened`, `mve_filter=exclude`, pages of
|
||||||
|
1,000). Each minute it asks for the markets created since the last check
|
||||||
|
(`min_created_ts`, which returns open and unopened ones alike: tried) and
|
||||||
|
those closed since (`status=closed&min_close_ts`: tried). A market is
|
||||||
|
tracked while open, or unopened with an `open_time` at most three hours
|
||||||
|
ahead: its book is subscribed, empty, before its first order. Three hours is
|
||||||
|
three hourly rescans, so every market is reached by a scan before it opens
|
||||||
|
even when one scan fails, and the fetcher keeps no memory of opening times.
|
||||||
|
A scan pages the unopened markets first: a market that opens meanwhile is
|
||||||
|
then on one list at least (open first, it could be on neither and be
|
||||||
|
removed). The core sees the same findings as from Gamma.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
- **The seam has three parts; each exchange supplies its loss rule**
|
||||||
|
([ADR 0004](../docs/adr/0004-exchange-seam.md)). *Rejected:* decode only,
|
||||||
|
as the brief's sentence reads (the core would hold both loss rules and a
|
||||||
|
third exchange would mean editing it); a whole book thread per exchange
|
||||||
|
(minute Snapshots, the quiet watch and moves written twice).
|
||||||
|
- **A Kalshi symbol is the market, asks derived from NO bids.** The output
|
||||||
|
reads like any order book. *Rejected:* two bid-only symbols per market
|
||||||
|
(the reader must know Kalshi's rule to find a spread).
|
||||||
|
- **The core reads only an event's market and whether it is a book; the
|
||||||
|
rule folds it into the core's book** (supersedes "the core's event is a
|
||||||
|
book or one change, set or moved": with each exchange supplying its rule,
|
||||||
|
a normalized event in the core would have no reader, the shell doing
|
||||||
|
nothing with a change but hand it on). *Rejected:* that normalized type,
|
||||||
|
both rules converting into it (more code for the same behaviour); a common
|
||||||
|
marker field (a hash and a sequence number have nothing in common).
|
||||||
|
- **A skipped `seq` reconnects the connection.** Both ways fetch every book
|
||||||
|
again; this one reuses sprint 05's path, and the other connection covers
|
||||||
|
the extra second. *Rejected:* `get_snapshot` for each market on the open
|
||||||
|
connection (a second recovery path, under an unpublished command limit).
|
||||||
|
- **First one wins by clock and rank.** The recording decided: the clock
|
||||||
|
alone can't name a change (12% of clock values carry several), the rank
|
||||||
|
within it can, since both connections deliver the same order. Simulated on
|
||||||
|
two recorded connections, one with a 5 s hole on five markets: 2,271
|
||||||
|
changes applied, 2,212 duplicates, 1,434 books compared at our clock, none
|
||||||
|
different. *Rejected:* the clock alone (drops the second change of a
|
||||||
|
clock); follow one connection and switch (the fallback, not needed: it
|
||||||
|
keeps the slower receive time and a Snapshot per switch).
|
||||||
|
- **A fresh book at or past our clock that differs replaces ours, with a
|
||||||
|
Snapshot.** The state is right at once. *Rejected:* holding the new
|
||||||
|
connection's changes until the other catches up (Polymarket's held
|
||||||
|
changes again, for a few ms after a reconnect).
|
||||||
|
- **A level below zero resyncs the symbol.** With relative changes it is the
|
||||||
|
one sign of a wrong book that `seq` doesn't give.
|
||||||
|
- **The rule was recorded before the sprint was agreed.** A rule taken from
|
||||||
|
the docs alone is what went wrong on Polymarket (CLAUDE.md, "Live data
|
||||||
|
beats the docs"); here the docs' "clock" was not enough by itself.
|
||||||
|
- **`use_yes_price: true`, sent explicitly.** It is the announced default,
|
||||||
|
and a NO bid then arrives as the ask it is. *Rejected:* leaving the
|
||||||
|
default (it is announced to flip, which would mirror every ask silently).
|
||||||
|
- **Polymarket's per-connection numbers on Kalshi.** No limit was met at
|
||||||
|
30,000 markets on the demo, so the cap of 200 connections doesn't bind;
|
||||||
|
one set of numbers is less to explain. Production's limits are untried.
|
||||||
|
- **The exchange is a required first argument; one exchange per process.**
|
||||||
|
The output has no exchange field, and one exchange failing doesn't stop
|
||||||
|
the other. *Rejected:* a default (Kalshi is the brief's, Polymarket the
|
||||||
|
one needing no key); both in one process.
|
||||||
|
- **The key comes as two file paths in the environment, Ed25519 only, signed
|
||||||
|
with `ring`.** Paths keep the secret out of the shell. `ring` is already
|
||||||
|
our TLS crypto; tried on a throwaway key: an OpenSSL Ed25519 PEM parsed
|
||||||
|
(`rustls-pki-types`, `Ed25519KeyPair::from_pkcs8_maybe_unchecked`) and
|
||||||
|
`openssl` verified the signature. `ring`, `rustls-pki-types` and `base64`
|
||||||
|
are already in `Cargo.lock`. *Rejected:* `ed25519-dalek` (a new
|
||||||
|
dependency for the same job); RSA keys too (a second signing path nobody
|
||||||
|
here uses).
|
||||||
|
- **An unopened market is tracked from three hours before its opening**
|
||||||
|
(supersedes "the fetcher remembers each opening time and reports the
|
||||||
|
market a minute before it", which superseded "ask for it by ticker once
|
||||||
|
its opening time has passed"). The two fetchers are separate threads and
|
||||||
|
the rescan's would have to hand the opening times to the minute check's;
|
||||||
|
a window over the rescan interval needs no memory at all, for a few
|
||||||
|
thousand empty books more (production: 755 markets opening within an
|
||||||
|
hour). Three hours rather than two, from the review: a failed rescan is
|
||||||
|
retried only an hour later, and two hours left no margin for one. *Rejected:* Polymarket's newest-created check alone (19% of
|
||||||
|
markets open more than 15 minutes after creation, so they would wait for
|
||||||
|
the hourly scan); subscribing every unopened market (tens of thousands of
|
||||||
|
empty books); the `market_lifecycle_v2` channel (no event at opening, by
|
||||||
|
the docs).
|
||||||
|
- **The 1013 close and the per-connection numbers stay in the core**
|
||||||
|
(supersedes "which close means too slow" and "its numbers" in the
|
||||||
|
`Exchange` trait). 1013 is the websocket's own "try again later", which
|
||||||
|
Kalshi never sent; Kalshi uses Polymarket's numbers (decided above). A
|
||||||
|
trait item each would be an abstraction with one value.
|
||||||
|
- **The extraction changes no Polymarket behaviour.** A change inside a move
|
||||||
|
of 3,300 lines can't be reviewed. One exception, with no effect on the
|
||||||
|
output: a market's book thread comes from a hash of its id.
|
||||||
|
- **No stand-in rule to test the shell alone.** Two real rules exercise it.
|
||||||
|
*Rejected:* a trivial test rule (a third implementation to maintain).
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- Every existing test, moved with its code, green and unchanged in what it
|
||||||
|
asserts.
|
||||||
|
- Kalshi replay: the one- and two-connection recordings folded into books;
|
||||||
|
the output reconciles; a NO bid becomes the right ask; relative changes
|
||||||
|
give the recorded books.
|
||||||
|
- Each Kalshi rule (duplicate by clock and rank, several changes of one
|
||||||
|
clock all applied, a connection not counted before its book, a fresh book
|
||||||
|
older / equal / different, a level below zero) has a test that fails with
|
||||||
|
the rule removed: checked by removing it.
|
||||||
|
- Signing: a fixed test key and string give a known signature; a missing
|
||||||
|
file, an unreadable PEM and an RSA key each stop with a clear error.
|
||||||
|
- The connection against a local server: the three headers arrive, the
|
||||||
|
server's Ping gets its Pong, a skipped `seq` makes it reconnect and tells
|
||||||
|
the book threads it went down.
|
||||||
|
- The fetcher on recorded listing pages, served by a local listing:
|
||||||
|
unopened then open scanned; an unopened market tracked from three hours
|
||||||
|
before its opening time and not before; created and closed ones reported;
|
||||||
|
a closed one not tracked.
|
||||||
|
- Live, each reconciled against its log: Polymarket 10 minutes; Kalshi 10
|
||||||
|
minutes on the demo, and on production if a key exists.
|
||||||
|
|
||||||
|
## Not in this sprint
|
||||||
|
|
||||||
|
- Combo (MVE) markets.
|
||||||
|
- The `market_lifecycle_v2` channel.
|
||||||
|
- Conflated updates (`update_interval_ms`).
|
||||||
|
- Thursday's maintenance (3:00 to 5:00 AM ET) beyond the ordinary reconnect.
|
||||||
|
- `get_snapshot`, RSA keys, authenticated REST.
|
||||||
|
- Any change to Polymarket's rules.
|
||||||
|
- Latency work (M7); the README (M8).
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- `engineering-challenge polymarket` and `engineering-challenge kalshi-demo`
|
||||||
|
run through the same `src/core/`; `src/core/` names no exchange.
|
||||||
|
- Polymarket, 10 minutes live: reconciled with every mismatch explained, and
|
||||||
|
losses per million changes no higher than sprint 05's (26.5 and 28.4).
|
||||||
|
- Kalshi demo, 10 minutes live: reconciled with every mismatch explained;
|
||||||
|
markets opened and closed during the run were added and removed; SIGINT
|
||||||
|
closes the connections and the output.
|
||||||
|
- Kalshi production: the same run if a key exists, "not run" otherwise.
|
||||||
|
- `docs/notes.md` has Kalshi's protocol facts as measured.
|
||||||
|
- `scripts/ci.sh` is green.
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Both exchanges run through one core. `src/core/` holds the orchestrator, the
|
||||||
|
connection loop and the book thread's shell, generic over an `Exchange`
|
||||||
|
(request, session, fetcher) and its `Rule`; it names no exchange.
|
||||||
|
Polymarket moved behind it with its rules, constants and log lines as they
|
||||||
|
were: its 79 tests moved with it and pass unchanged. Kalshi joined with a
|
||||||
|
signed handshake, the `seq` check in the connection, first one wins by clock
|
||||||
|
and rank, and a fetcher on its listing. `engineering-challenge` now takes
|
||||||
|
the exchange first.
|
||||||
|
|
||||||
|
**Live runs**, each reconciled with every mismatch explained by its log
|
||||||
|
line, no orphan Delta:
|
||||||
|
|
||||||
|
| Run | Markets, connections | Changes written | Mismatches | Peak memory, threads |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Polymarket, 10 min, before the review's fixes | 200,052 on 179 | 18,971,851 | 461 losses (24.3 / M), 2,839 gaps | 1.93 GB, 201 |
|
||||||
|
| Kalshi demo, 10 min, before | 246,349 on 110 | 2.4 M | 16, all "loss" | 0.67 GB, 131 |
|
||||||
|
| Kalshi demo, 21 min across 20:00 UTC, before | 245,383 on 110 | 5.6 M | 63, all "loss" | 0.68 GB, 131 |
|
||||||
|
| Kalshi demo, 10 min, after | 244,903 on 109 | 2,470,013 | 30, all "loss" | 0.66 GB, 130 |
|
||||||
|
| Polymarket, 5 min, after | 201,124 on 180 | 4,842,531 | 130 losses (26.8 / M), 2,014 gaps | 1.60 GB, 201 |
|
||||||
|
| Kalshi production | not run: no production key | | | |
|
||||||
|
|
||||||
|
Polymarket's losses are no higher than sprint 05's (26.5 and 28.4 per
|
||||||
|
million). After the review's fixes it changed only by renames and the shared
|
||||||
|
retry helper, so its rerun was 5 minutes, not 10. On Kalshi every change
|
||||||
|
came twice (2,464,467 duplicates for 2,470,013 written), none below zero,
|
||||||
|
none ahead of the book, no skipped `seq` in any run. The 21-minute run
|
||||||
|
removed 3,273 markets that closed (3,271 at 20:00, checked against the
|
||||||
|
listing) and added 1,382; SIGINT closed the output each time.
|
||||||
|
|
||||||
|
**Kalshi's "losses" are not known to be losses.** They are fresh books at or
|
||||||
|
past ours that differ, the case we agreed to write as a Snapshot without
|
||||||
|
being able to tell why. 54 of the 63 in the long run came at 20:00:01 to
|
||||||
|
20:00:04 for markets closing at 20:00: Kalshi sends a cleared book at the
|
||||||
|
close, without the changes that lead to it. The rest came at startup or
|
||||||
|
with a recycle, when one connection's book runs ahead of the other's
|
||||||
|
stream. The Snapshots are right; telling the three apart is in the roadmap's
|
||||||
|
features.
|
||||||
|
|
||||||
|
**Fixed at the close**, from the code review, each with a test that failed
|
||||||
|
first:
|
||||||
|
|
||||||
|
- After a resync's answer older than what the other connection had
|
||||||
|
delivered, the changes between were dropped as duplicates with no loss
|
||||||
|
logged: a change ahead of the book now waits for it.
|
||||||
|
- A market opening during a rescan could be on neither list and be removed
|
||||||
|
for up to an hour: unopened markets are paged first.
|
||||||
|
- A key file flattened onto one line made the error print the key's bytes
|
||||||
|
(the PEM reader's error quotes its input): the error names the file only.
|
||||||
|
- A symbol whose book didn't come back after both connections reconnected
|
||||||
|
was never asked for again: it is, at the minute.
|
||||||
|
- A first book asked from one connection and delivered by the other was
|
||||||
|
logged as a loss.
|
||||||
|
|
||||||
|
After them, each of Kalshi's 21 rules was removed in turn and a test failed
|
||||||
|
every time.
|
||||||
|
|
||||||
|
**Diverged from the plan**, each in Decisions or its section: an unopened
|
||||||
|
market is tracked from three hours ahead with nothing remembered (not "a
|
||||||
|
minute before", which needed the two fetcher threads to share opening
|
||||||
|
times); the core reads only an event's market and whether it is a book, the
|
||||||
|
rule folding it into the core's book (not a normalized event in the core;
|
||||||
|
agreed at the close); the 1013 close and the per-connection numbers stayed
|
||||||
|
in the core; `Resubscribe` uses `subscribe`, not `add_markets`; errors 10
|
||||||
|
and 25 reconnect. One Polymarket log text changed: a refused handshake reads
|
||||||
|
"handshake refused: <status>".
|
||||||
|
|
||||||
|
**Left out**, and where it went: Kalshi production (no key; the command is
|
||||||
|
`engineering-challenge kalshi` with production's two files); telling a
|
||||||
|
close's book and a book ahead from a real loss, retrying a failed rescan
|
||||||
|
sooner than the next hour, and sharing the two rules' resync bookkeeping
|
||||||
|
(Possible features). Not covered by a test: the orchestrator's loop, as in
|
||||||
|
sprints 04 and 05; `Key::from_env`; the 429s of a hundred handshakes at once
|
||||||
|
(seen live, each retried); the "ahead of the book" rule live (0 in 10
|
||||||
|
minutes: it needs a resync, and none was needed).
|
||||||
|
|
||||||
|
### Retrospective
|
||||||
|
|
||||||
|
- **Went well:** recording the demo before agreeing the sprint overturned
|
||||||
|
the rule on paper (the clock alone can't name a change) and answered the
|
||||||
|
three open points in an hour. Moving Polymarket's rule by script, with
|
||||||
|
every cut asserted, kept 3,300 lines reviewable: the reviewer found no
|
||||||
|
behaviour change. The reviewer's adversarial sequences on two connections
|
||||||
|
found what the tests didn't.
|
||||||
|
- **Went wrong:** three bugs in Kalshi's rule passed my own tests and the
|
||||||
|
mutation check, which proves the rules written, not the ones missing: I
|
||||||
|
never asked what happens when the book goes backwards. I changed an agreed
|
||||||
|
point (the minute-before opening) while coding and reported it after,
|
||||||
|
instead of asking first; and I didn't notice the core's event had drifted
|
||||||
|
from the agreed one until the review. A "flaky" test cost a detour: my
|
||||||
|
mutation script restored files with an older timestamp and cargo kept the
|
||||||
|
mutant's build. The live runs came before the review, so its fixes meant
|
||||||
|
running them again.
|
||||||
|
- **Tested enough?** The rules are, by removal. The orchestrator's loop
|
||||||
|
still isn't, production Kalshi not at all, and the fetchers only against a
|
||||||
|
local listing with one page per status.
|
||||||
|
- **Next time:** for each piece of state a rule keeps, ask what makes it go
|
||||||
|
backwards or stay empty, and write that test. Ask before diverging from an
|
||||||
|
agreed point, even a small one. Review before the live runs that close a
|
||||||
|
sprint.
|
||||||
+25
@@ -77,6 +77,19 @@ impl Book {
|
|||||||
previous.unwrap_or(0)
|
previous.unwrap_or(0)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Moves a level's quantity by `delta`, which can be negative, and returns
|
||||||
|
/// the sum. A level can't hold less than nothing: a negative sum removes
|
||||||
|
/// it, and tells the caller the book was wrong.
|
||||||
|
pub fn add(&mut self, side: Side, price: i64, delta: i64) -> i64 {
|
||||||
|
let levels = match side {
|
||||||
|
Side::Buy => &self.bids,
|
||||||
|
Side::Sell => &self.asks,
|
||||||
|
};
|
||||||
|
let quantity = levels.get(&price).copied().unwrap_or(0) + delta;
|
||||||
|
self.set(side, price, quantity.max(0));
|
||||||
|
quantity
|
||||||
|
}
|
||||||
|
|
||||||
/// Both sides in ascending price order.
|
/// Both sides in ascending price order.
|
||||||
pub fn snapshot(
|
pub fn snapshot(
|
||||||
&self,
|
&self,
|
||||||
@@ -157,6 +170,18 @@ mod tests {
|
|||||||
assert_eq!(format_fixed(parse_fixed("0.50").unwrap()), "0.5");
|
assert_eq!(format_fixed(parse_fixed("0.50").unwrap()), "0.5");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn adding_moves_a_level_and_never_leaves_it_below_zero() {
|
||||||
|
let mut book = Book::default();
|
||||||
|
assert_eq!(book.add(Side::Sell, 600_000, 5), 5);
|
||||||
|
assert_eq!(book.add(Side::Sell, 600_000, -2), 3);
|
||||||
|
assert_eq!(book.asks, BTreeMap::from([(600_000, 3)]));
|
||||||
|
assert_eq!(book.add(Side::Sell, 600_000, -3), 0);
|
||||||
|
assert!(book.asks.is_empty());
|
||||||
|
assert_eq!(book.add(Side::Buy, 400_000, -1), -1);
|
||||||
|
assert!(book.bids.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_zero_quantity_removes_the_level() {
|
fn a_zero_quantity_removes_the_level() {
|
||||||
let mut book = Book::default();
|
let mut book = Book::default();
|
||||||
|
|||||||
@@ -0,0 +1,772 @@
|
|||||||
|
//! A book thread: owns the books of its markets' symbols and folds into them
|
||||||
|
//! what their connections deliver, by the exchange's [`Rule`] (ADR 0004). The
|
||||||
|
//! shell here is the same for every exchange: which markets and symbols it
|
||||||
|
//! holds and on which connections, which connections are up, moves between
|
||||||
|
//! connections, each minute's Snapshots and load report, and the watch for a
|
||||||
|
//! subscription gone quiet on one connection (roadmap/05). What an event does
|
||||||
|
//! to a book, how a loss is found and when to resync is the rule's.
|
||||||
|
|
||||||
|
use std::collections::{BTreeMap, BTreeSet, HashMap};
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use chrono::{DateTime, DurationRound, TimeDelta, Utc};
|
||||||
|
use crossbeam::channel::{Receiver, Sender, never, select};
|
||||||
|
use engineering_challenge::Side;
|
||||||
|
use engineering_challenge::book::{Book, Output};
|
||||||
|
use engineering_challenge::util::dtos::challenge_orderbook::PriceLevelUpdateDomain;
|
||||||
|
|
||||||
|
use super::socket::Command;
|
||||||
|
use super::{Arrival, Event, Market, Received};
|
||||||
|
|
||||||
|
/// A market measured for less than this waits for the next load report: over
|
||||||
|
/// a few seconds, its opening books alone would read as a burst.
|
||||||
|
pub(crate) const MIN_MEASURED: TimeDelta = TimeDelta::seconds(30);
|
||||||
|
/// How often a book thread whose rule asks for it wakes for the rule's timer.
|
||||||
|
const WAKE_EVERY: Duration = Duration::from_millis(10);
|
||||||
|
/// A connection that delivered none of a symbol's changes in a minute while
|
||||||
|
/// another delivered at least this many is quiet for it (sprint 05: crossed
|
||||||
|
/// once in 21 live minutes; at 1, copies landing in the next minute would
|
||||||
|
/// count).
|
||||||
|
const QUIET_AGAINST: u32 = 3;
|
||||||
|
|
||||||
|
/// What the book thread does with an event or a command: write output, ask a
|
||||||
|
/// connection (by number) for something, or report to the orchestrator.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum Effect {
|
||||||
|
Write(Output),
|
||||||
|
Send(usize, Command),
|
||||||
|
Report(Report),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a book thread tells the orchestrator.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub enum Report {
|
||||||
|
Loads(Loads),
|
||||||
|
/// The market stayed quiet on the connection after being resubscribed
|
||||||
|
/// there: its subscription should move.
|
||||||
|
Quiet {
|
||||||
|
market: String,
|
||||||
|
connection: usize,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Each market's load (market id → bytes per second on one connection),
|
||||||
|
/// every market of the book thread measured for at least [`MIN_MEASURED`].
|
||||||
|
pub type Loads = HashMap<String, u64>;
|
||||||
|
|
||||||
|
/// What the orchestrator tells a book thread. A market's commands all go to
|
||||||
|
/// its one book thread, in order.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum BookCommand<M> {
|
||||||
|
/// A connection just opened, and how to reach it: sent to every book
|
||||||
|
/// thread before any command that names it.
|
||||||
|
Connected(usize, Sender<Command>),
|
||||||
|
/// Track the market, on these connections.
|
||||||
|
Add(M, Vec<usize>),
|
||||||
|
/// The market's subscription on `from` goes to `to`.
|
||||||
|
Move {
|
||||||
|
market: M,
|
||||||
|
from: usize,
|
||||||
|
to: usize,
|
||||||
|
},
|
||||||
|
Remove(M),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What an exchange's book thread does with its events: the part of a book
|
||||||
|
/// thread that differs between exchanges, with a state of its own per symbol
|
||||||
|
/// ([`Rule::Symbol`]) and for the thread. The shell calls it at each step
|
||||||
|
/// that concerns it, and it acts through the shell's symbols and the effects
|
||||||
|
/// it pushes.
|
||||||
|
pub trait Rule: Send + 'static {
|
||||||
|
type Market: Market;
|
||||||
|
type Event: Event;
|
||||||
|
/// Its state for one symbol.
|
||||||
|
type Symbol: Send;
|
||||||
|
|
||||||
|
/// A symbol of `market` starts being tracked.
|
||||||
|
fn symbol(&self, market: &Self::Market) -> Self::Symbol;
|
||||||
|
|
||||||
|
/// An event from `connection`, received at `received_at`.
|
||||||
|
fn on_event(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Self::Symbol>,
|
||||||
|
connection: usize,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
event: Self::Event,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
);
|
||||||
|
|
||||||
|
/// `connection` stopped carrying the symbol: its subscription there left
|
||||||
|
/// (the shell has taken the connection off the symbol's).
|
||||||
|
fn on_leave(
|
||||||
|
&mut self,
|
||||||
|
up: &[bool],
|
||||||
|
name: &str,
|
||||||
|
symbol: &mut Symbol<Self::Symbol>,
|
||||||
|
connection: usize,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
);
|
||||||
|
|
||||||
|
/// `connection` went down (the shell has marked it so).
|
||||||
|
fn on_down(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Self::Symbol>,
|
||||||
|
connection: usize,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
);
|
||||||
|
|
||||||
|
/// After every event, and while none comes.
|
||||||
|
fn on_timer(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Self::Symbol>,
|
||||||
|
now: DateTime<Utc>,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
);
|
||||||
|
|
||||||
|
/// Whether the book thread must wake soon for [`Rule::on_timer`].
|
||||||
|
fn wakes_soon(&self) -> bool;
|
||||||
|
|
||||||
|
/// The symbol is no longer tracked.
|
||||||
|
fn on_removed(&mut self, name: &str);
|
||||||
|
|
||||||
|
/// Whether the symbol has a book to write in a Snapshot.
|
||||||
|
fn has_book(symbol: &Self::Symbol) -> bool;
|
||||||
|
|
||||||
|
/// Whether the symbol may be judged quiet this minute: its book is in
|
||||||
|
/// order and no fresh one is on its way.
|
||||||
|
fn may_be_quiet(symbol: &Self::Symbol) -> bool;
|
||||||
|
|
||||||
|
/// At the minute, after the symbol's Snapshot.
|
||||||
|
fn on_minute(
|
||||||
|
&mut self,
|
||||||
|
up: &[bool],
|
||||||
|
name: &str,
|
||||||
|
symbol: &mut Symbol<Self::Symbol>,
|
||||||
|
now: DateTime<Utc>,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
);
|
||||||
|
|
||||||
|
/// Logs the minute line, the shell's figures in it.
|
||||||
|
fn log_minute(&mut self, minute: &Minute);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The shell's figures for the minute line.
|
||||||
|
pub struct Minute {
|
||||||
|
pub number: usize,
|
||||||
|
pub symbols: usize,
|
||||||
|
/// Subscriptions found quiet, each resubscribed, since the start.
|
||||||
|
pub quiet: u64,
|
||||||
|
/// Markets quiet again after their resubscribe, reported for a move.
|
||||||
|
pub quiet_reported: u64,
|
||||||
|
/// Changes received this minute, from all connections.
|
||||||
|
pub received: u64,
|
||||||
|
/// The most events waiting for the book thread since the last line: how
|
||||||
|
/// far behind it fell.
|
||||||
|
pub peak_waiting: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A symbol's subscription on one of its connections, as watched for going
|
||||||
|
/// quiet.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct Watch {
|
||||||
|
pub connection: usize,
|
||||||
|
/// Its changes received from this connection this minute: every one,
|
||||||
|
/// whatever the rule then made of it.
|
||||||
|
pub received: u32,
|
||||||
|
/// When it was last subscribed there: a minute it didn't cover whole is
|
||||||
|
/// not judged.
|
||||||
|
pub since: DateTime<Utc>,
|
||||||
|
/// When the connection last delivered a complete book of it: its first
|
||||||
|
/// since the connection came up is the reconnect's.
|
||||||
|
pub book_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Watch {
|
||||||
|
fn new(connection: usize, since: DateTime<Utc>) -> Self {
|
||||||
|
Self {
|
||||||
|
connection,
|
||||||
|
received: 0,
|
||||||
|
since,
|
||||||
|
book_at: DateTime::UNIX_EPOCH,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A symbol: its book, where it is subscribed, and the rule's state for it.
|
||||||
|
pub struct Symbol<S> {
|
||||||
|
pub book: Book,
|
||||||
|
/// Of the last change applied, or of the book that replaced ours: the
|
||||||
|
/// time a Snapshot's book is the exchange's as of.
|
||||||
|
pub exchange_timestamp: DateTime<Utc>,
|
||||||
|
/// Its market's id.
|
||||||
|
pub market: String,
|
||||||
|
/// The connections it is subscribed on (three while a move overlaps).
|
||||||
|
pub connections: Vec<usize>,
|
||||||
|
/// One per connection in `connections`.
|
||||||
|
pub watches: Vec<Watch>,
|
||||||
|
/// The connection it was quiet on last minute, and resubscribed.
|
||||||
|
pub quiet_on: Option<usize>,
|
||||||
|
/// The receive time of its last Delta or Snapshot: a later Delta is
|
||||||
|
/// stamped no earlier, so replaying the output by time keeps its order.
|
||||||
|
pub last_written: DateTime<Utc>,
|
||||||
|
pub rule: S,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<S> Symbol<S> {
|
||||||
|
fn new(market: &str, connections: &[usize], now: DateTime<Utc>, rule: S) -> Self {
|
||||||
|
Self {
|
||||||
|
book: Book::default(),
|
||||||
|
exchange_timestamp: DateTime::UNIX_EPOCH,
|
||||||
|
market: market.to_string(),
|
||||||
|
connections: connections.to_vec(),
|
||||||
|
watches: connections
|
||||||
|
.iter()
|
||||||
|
.map(|&connection| Watch::new(connection, now))
|
||||||
|
.collect(),
|
||||||
|
quiet_on: None,
|
||||||
|
last_written: DateTime::UNIX_EPOCH,
|
||||||
|
rule,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn watch_mut(&mut self, connection: usize) -> Option<&mut Watch> {
|
||||||
|
self.watches
|
||||||
|
.iter_mut()
|
||||||
|
.find(|watch| watch.connection == connection)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One of its connections that is up, other than `except`.
|
||||||
|
pub fn other_up(&self, up: &[bool], except: usize) -> Option<usize> {
|
||||||
|
self.connections
|
||||||
|
.iter()
|
||||||
|
.copied()
|
||||||
|
.find(|&connection| connection != except && is_up(up, connection))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Delta setting a level of its book to `quantity`, received at
|
||||||
|
/// `received_at`: stamped no earlier than what it last wrote.
|
||||||
|
pub fn delta(
|
||||||
|
&mut self,
|
||||||
|
name: &str,
|
||||||
|
side: Side,
|
||||||
|
price: i64,
|
||||||
|
quantity: i64,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
exchange_timestamp: DateTime<Utc>,
|
||||||
|
) -> Output {
|
||||||
|
self.last_written = self.last_written.max(received_at);
|
||||||
|
Output::Delta(PriceLevelUpdateDomain {
|
||||||
|
backtest_timestamp: self.last_written,
|
||||||
|
exchange_timestamp,
|
||||||
|
price_level: price,
|
||||||
|
quantity,
|
||||||
|
side,
|
||||||
|
symbol: name.to_string(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Snapshot of its book, received at `received_at`.
|
||||||
|
pub fn snapshot(&mut self, name: &str, received_at: DateTime<Utc>) -> Output {
|
||||||
|
self.last_written = self.last_written.max(received_at);
|
||||||
|
self.book
|
||||||
|
.snapshot(name, received_at, self.exchange_timestamp)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The connection quiet for it over the minute begun at `started`: one
|
||||||
|
/// that delivered none of its changes while another delivered at least
|
||||||
|
/// [`QUIET_AGAINST`], both up and subscribed for the whole minute.
|
||||||
|
fn quiet(&self, started: DateTime<Utc>, up_since: &[Option<DateTime<Utc>>]) -> Option<usize> {
|
||||||
|
let judged: Vec<&Watch> = self
|
||||||
|
.watches
|
||||||
|
.iter()
|
||||||
|
.filter(|watch| {
|
||||||
|
watch.since <= started
|
||||||
|
&& up_since
|
||||||
|
.get(watch.connection)
|
||||||
|
.copied()
|
||||||
|
.flatten()
|
||||||
|
.is_some_and(|up| up <= started)
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
let busiest = judged.iter().map(|watch| watch.received).max()?;
|
||||||
|
(busiest >= QUIET_AGAINST)
|
||||||
|
.then(|| judged.iter().find(|watch| watch.received == 0))
|
||||||
|
.flatten()
|
||||||
|
.map(|watch| watch.connection)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn is_up(up: &[bool], connection: usize) -> bool {
|
||||||
|
up.get(connection).copied().unwrap_or(false)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A market's frame bytes since `since`, from all of its connections.
|
||||||
|
struct Measure {
|
||||||
|
bytes: u64,
|
||||||
|
since: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Measure {
|
||||||
|
/// Bytes per second since `since` on one of `connections`, then starts
|
||||||
|
/// again from `now`; none before [`MIN_MEASURED`], the bytes kept for the
|
||||||
|
/// next report.
|
||||||
|
fn take(&mut self, now: DateTime<Utc>, connections: usize) -> Option<u64> {
|
||||||
|
let measured = now - self.since;
|
||||||
|
if measured < MIN_MEASURED {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let load =
|
||||||
|
self.bytes * 1_000 / measured.num_milliseconds() as u64 / connections.max(1) as u64;
|
||||||
|
*self = Measure {
|
||||||
|
bytes: 0,
|
||||||
|
since: now,
|
||||||
|
};
|
||||||
|
Some(load)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A market of the book thread.
|
||||||
|
pub struct Tracked {
|
||||||
|
/// Its symbols' names.
|
||||||
|
pub(crate) names: Vec<String>,
|
||||||
|
measure: Measure,
|
||||||
|
/// A move under way: the connection left once the new one, the second
|
||||||
|
/// here, delivers a book of the market. Until then both carry it, so the
|
||||||
|
/// move costs no change.
|
||||||
|
leaving: Option<(usize, usize)>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What every exchange's book thread holds: its symbols, its markets, its
|
||||||
|
/// connections' state and the minute's.
|
||||||
|
pub struct Shell<S> {
|
||||||
|
/// The exchange's name, for the logs.
|
||||||
|
pub exchange: &'static str,
|
||||||
|
/// Its book thread's number, for the logs.
|
||||||
|
pub number: usize,
|
||||||
|
pub symbols: BTreeMap<String, Symbol<S>>,
|
||||||
|
/// By market id.
|
||||||
|
pub markets: HashMap<String, Tracked>,
|
||||||
|
/// By connection number: whether it is up.
|
||||||
|
pub up: Vec<bool>,
|
||||||
|
/// By connection number: since when it is up, if it is.
|
||||||
|
pub up_since: Vec<Option<DateTime<Utc>>>,
|
||||||
|
/// When the minute being counted began: its quiet subscriptions are
|
||||||
|
/// judged at its end. None before the first.
|
||||||
|
minute_started: Option<DateTime<Utc>>,
|
||||||
|
/// Changes received this minute, from all connections: the rule counts
|
||||||
|
/// them, as it reads them.
|
||||||
|
pub received_this_minute: u64,
|
||||||
|
/// The receive time of the last event applied: the time a Snapshot's book
|
||||||
|
/// is exact as of.
|
||||||
|
pub last_received_at: Option<DateTime<Utc>>,
|
||||||
|
/// The next wall-clock minute: its Snapshots, load report and log line.
|
||||||
|
next_minute_at: Option<DateTime<Utc>>,
|
||||||
|
/// The most events waiting for the book thread since the last minute log.
|
||||||
|
pub peak_waiting: usize,
|
||||||
|
pub quiet: u64,
|
||||||
|
pub quiet_reported: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<S> Shell<S> {
|
||||||
|
/// A change of `name` from `connection`, counted for the quiet watch:
|
||||||
|
/// every one of a tracked symbol, before the rule makes anything of it.
|
||||||
|
pub fn count_received(&mut self, name: &str, connection: usize) {
|
||||||
|
let Some(symbol) = self.symbols.get_mut(name) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if let Some(watch) = symbol.watch_mut(connection) {
|
||||||
|
watch.received += 1;
|
||||||
|
}
|
||||||
|
self.received_this_minute += 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn set_up(&mut self, connection: usize, up: bool, at: DateTime<Utc>) {
|
||||||
|
if self.up.len() <= connection {
|
||||||
|
self.up.resize(connection + 1, false);
|
||||||
|
self.up_since.resize(connection + 1, None);
|
||||||
|
}
|
||||||
|
if up && !self.up[connection] {
|
||||||
|
self.up_since[connection] = Some(at);
|
||||||
|
} else if !up {
|
||||||
|
self.up_since[connection] = None;
|
||||||
|
}
|
||||||
|
self.up[connection] = up;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct Feed<R: Rule> {
|
||||||
|
pub shell: Shell<R::Symbol>,
|
||||||
|
pub rule: R,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<R: Rule> Feed<R> {
|
||||||
|
pub fn new(exchange: &'static str, number: usize, rule: R) -> Self {
|
||||||
|
Self {
|
||||||
|
shell: Shell {
|
||||||
|
exchange,
|
||||||
|
number,
|
||||||
|
symbols: BTreeMap::new(),
|
||||||
|
markets: HashMap::new(),
|
||||||
|
up: Vec::new(),
|
||||||
|
up_since: Vec::new(),
|
||||||
|
minute_started: None,
|
||||||
|
received_this_minute: 0,
|
||||||
|
last_received_at: None,
|
||||||
|
next_minute_at: None,
|
||||||
|
peak_waiting: 0,
|
||||||
|
quiet: 0,
|
||||||
|
quiet_reported: 0,
|
||||||
|
},
|
||||||
|
rule,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Tracks a market's symbols, then asks its connections to subscribe
|
||||||
|
/// them: a book can't arrive for a symbol not yet known. Its load is
|
||||||
|
/// measured from `now`.
|
||||||
|
pub fn add_market(
|
||||||
|
&mut self,
|
||||||
|
market: &R::Market,
|
||||||
|
connections: &[usize],
|
||||||
|
now: DateTime<Utc>,
|
||||||
|
) -> Vec<Effect> {
|
||||||
|
let names = market.symbols().to_vec();
|
||||||
|
for name in &names {
|
||||||
|
let rule = self.rule.symbol(market);
|
||||||
|
self.shell.symbols.insert(
|
||||||
|
name.clone(),
|
||||||
|
Symbol::new(market.id(), connections, now, rule),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
let tracked = Tracked {
|
||||||
|
names: names.clone(),
|
||||||
|
measure: Measure {
|
||||||
|
bytes: 0,
|
||||||
|
since: now,
|
||||||
|
},
|
||||||
|
leaving: None,
|
||||||
|
};
|
||||||
|
self.shell.markets.insert(market.id().to_string(), tracked);
|
||||||
|
connections
|
||||||
|
.iter()
|
||||||
|
.map(|&connection| Effect::Send(connection, Command::Subscribe(names.clone())))
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Moves a market's subscription from one connection to another: it
|
||||||
|
/// subscribes on `to` now, and leaves `from` once `to` delivers a book of
|
||||||
|
/// the market. The book stays as it is: no Snapshot, no subscription gap.
|
||||||
|
pub fn move_market(
|
||||||
|
&mut self,
|
||||||
|
market: &R::Market,
|
||||||
|
from: usize,
|
||||||
|
to: usize,
|
||||||
|
now: DateTime<Utc>,
|
||||||
|
) -> Vec<Effect> {
|
||||||
|
let Some(tracked) = self.shell.markets.get_mut(market.id()) else {
|
||||||
|
return Vec::new();
|
||||||
|
};
|
||||||
|
// A move still under way when the next comes: it ends now.
|
||||||
|
let mut effects = match tracked.leaving.take() {
|
||||||
|
Some((left, _)) => self.leave(market.id(), left),
|
||||||
|
None => Vec::new(),
|
||||||
|
};
|
||||||
|
if let Some(tracked) = self.shell.markets.get_mut(market.id()) {
|
||||||
|
tracked.leaving = Some((from, to));
|
||||||
|
}
|
||||||
|
for name in market.symbols() {
|
||||||
|
if let Some(symbol) = self.shell.symbols.get_mut(name)
|
||||||
|
&& !symbol.connections.contains(&to)
|
||||||
|
{
|
||||||
|
symbol.connections.push(to);
|
||||||
|
symbol.watches.push(Watch::new(to, now));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
effects.push(Effect::Send(
|
||||||
|
to,
|
||||||
|
Command::Subscribe(market.symbols().to_vec()),
|
||||||
|
));
|
||||||
|
effects
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The market's symbols stop being carried by `connection`: it is
|
||||||
|
/// unsubscribed there, and the rule told.
|
||||||
|
fn leave(&mut self, market: &str, connection: usize) -> Vec<Effect> {
|
||||||
|
let Some(tracked) = self.shell.markets.get(market) else {
|
||||||
|
return Vec::new();
|
||||||
|
};
|
||||||
|
let names = tracked.names.clone();
|
||||||
|
let mut effects = vec![Effect::Send(
|
||||||
|
connection,
|
||||||
|
Command::Unsubscribe(names.clone()),
|
||||||
|
)];
|
||||||
|
let Shell { symbols, up, .. } = &mut self.shell;
|
||||||
|
for name in &names {
|
||||||
|
if let Some(symbol) = symbols.get_mut(name) {
|
||||||
|
symbol.connections.retain(|&on| on != connection);
|
||||||
|
symbol
|
||||||
|
.watches
|
||||||
|
.retain(|watch| watch.connection != connection);
|
||||||
|
self.rule
|
||||||
|
.on_leave(up, name, symbol, connection, &mut effects);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
effects
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Forgets a market's symbols and unsubscribes them; nothing is written
|
||||||
|
/// for them. Their events still on the way are ignored, as any unknown
|
||||||
|
/// symbol's.
|
||||||
|
pub fn remove_market(&mut self, market: &R::Market) -> Vec<Effect> {
|
||||||
|
let mut connections = Vec::new();
|
||||||
|
for name in market.symbols() {
|
||||||
|
if let Some(symbol) = self.shell.symbols.remove(name) {
|
||||||
|
connections = symbol.connections;
|
||||||
|
}
|
||||||
|
self.rule.on_removed(name);
|
||||||
|
}
|
||||||
|
self.shell.markets.remove(market.id());
|
||||||
|
connections
|
||||||
|
.into_iter()
|
||||||
|
.map(|connection| {
|
||||||
|
Effect::Send(connection, Command::Unsubscribe(market.symbols().to_vec()))
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a connection sent: an event, or itself coming up or going down.
|
||||||
|
pub fn on_received(&mut self, received: Received<R::Event>) -> Vec<Effect> {
|
||||||
|
match received {
|
||||||
|
Received::Event(arrival) => self.on_event(arrival),
|
||||||
|
Received::Up(connection) => {
|
||||||
|
// Its first events come after this marker; any other
|
||||||
|
// connection's before it were received earlier.
|
||||||
|
let at = self.shell.last_received_at.unwrap_or(DateTime::UNIX_EPOCH);
|
||||||
|
self.shell.set_up(connection, true, at);
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
Received::Down(connection) => {
|
||||||
|
self.shell.set_up(connection, false, DateTime::UNIX_EPOCH);
|
||||||
|
let mut effects = Vec::new();
|
||||||
|
self.rule.on_down(&mut self.shell, connection, &mut effects);
|
||||||
|
effects
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn on_event(&mut self, arrival: Arrival<R::Event>) -> Vec<Effect> {
|
||||||
|
let (received_at, connection) = (arrival.received_at, arrival.connection);
|
||||||
|
// A connection that delivers is up, its marker possibly still behind.
|
||||||
|
if !is_up(&self.shell.up, connection) {
|
||||||
|
self.shell.set_up(connection, true, received_at);
|
||||||
|
}
|
||||||
|
// Minute Snapshots due before this event hold the books as they were.
|
||||||
|
let mut effects = self.tick(received_at);
|
||||||
|
self.shell.last_received_at = Some(received_at);
|
||||||
|
let market = arrival.event.market().map(str::to_string);
|
||||||
|
if let Some(tracked) = market
|
||||||
|
.as_ref()
|
||||||
|
.and_then(|market| self.shell.markets.get_mut(market))
|
||||||
|
{
|
||||||
|
tracked.measure.bytes += arrival.bytes;
|
||||||
|
// A move ends once the new connection delivers a book of the
|
||||||
|
// market.
|
||||||
|
let ends = tracked
|
||||||
|
.leaving
|
||||||
|
.filter(|&(_, to)| to == connection && arrival.event.is_book());
|
||||||
|
if let (Some((from, _)), Some(market)) = (ends, &market) {
|
||||||
|
tracked.leaving = None;
|
||||||
|
effects.extend(self.leave(market, from));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
self.rule.on_event(
|
||||||
|
&mut self.shell,
|
||||||
|
connection,
|
||||||
|
received_at,
|
||||||
|
arrival.event,
|
||||||
|
&mut effects,
|
||||||
|
);
|
||||||
|
self.rule
|
||||||
|
.on_timer(&mut self.shell, received_at, &mut effects);
|
||||||
|
effects
|
||||||
|
}
|
||||||
|
|
||||||
|
/// While no event comes: the minute, and the rule's timer.
|
||||||
|
pub fn idle(&mut self, now: DateTime<Utc>) -> Vec<Effect> {
|
||||||
|
let mut effects = self.tick(now);
|
||||||
|
self.rule.on_timer(&mut self.shell, now, &mut effects);
|
||||||
|
effects
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Once `now` reaches the next wall-clock minute: reports the loads,
|
||||||
|
/// emits the minute Snapshots, resubscribes (or reports) the
|
||||||
|
/// subscriptions quiet over the minute that ends, and lets the rule act
|
||||||
|
/// on each symbol.
|
||||||
|
pub fn tick(&mut self, now: DateTime<Utc>) -> Vec<Effect> {
|
||||||
|
let shell = &mut self.shell;
|
||||||
|
let next = *shell.next_minute_at.get_or_insert_with(|| next_minute(now));
|
||||||
|
if now < next {
|
||||||
|
return Vec::new();
|
||||||
|
}
|
||||||
|
shell.next_minute_at = Some(next_minute(now));
|
||||||
|
let started = shell.minute_started.replace(now);
|
||||||
|
let mut loads = Loads::new();
|
||||||
|
for (market, tracked) in &mut shell.markets {
|
||||||
|
let connections = tracked
|
||||||
|
.names
|
||||||
|
.first()
|
||||||
|
.and_then(|name| shell.symbols.get(name))
|
||||||
|
.map_or(1, |symbol| symbol.connections.len());
|
||||||
|
if let Some(load) = tracked.measure.take(now, connections) {
|
||||||
|
loads.insert(market.clone(), load);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let mut effects = vec![Effect::Report(Report::Loads(loads))];
|
||||||
|
let Some(received_at) = shell.last_received_at else {
|
||||||
|
return effects;
|
||||||
|
};
|
||||||
|
self.rule.log_minute(&Minute {
|
||||||
|
number: shell.number,
|
||||||
|
symbols: shell.symbols.len(),
|
||||||
|
quiet: shell.quiet,
|
||||||
|
quiet_reported: shell.quiet_reported,
|
||||||
|
received: shell.received_this_minute,
|
||||||
|
peak_waiting: shell.peak_waiting,
|
||||||
|
});
|
||||||
|
shell.peak_waiting = 0;
|
||||||
|
shell.received_this_minute = 0;
|
||||||
|
let mut reported = BTreeSet::new();
|
||||||
|
let exchange = shell.exchange;
|
||||||
|
for (name, symbol) in &mut shell.symbols {
|
||||||
|
let quiet = started
|
||||||
|
.filter(|_| R::may_be_quiet(&symbol.rule))
|
||||||
|
.and_then(|started| symbol.quiet(started, &shell.up_since));
|
||||||
|
for watch in &mut symbol.watches {
|
||||||
|
watch.received = 0;
|
||||||
|
}
|
||||||
|
match quiet {
|
||||||
|
// Resubscribed already, and still quiet: it should move.
|
||||||
|
Some(connection) if symbol.quiet_on == Some(connection) => {
|
||||||
|
symbol.quiet_on = None;
|
||||||
|
if reported.insert((symbol.market.clone(), connection)) {
|
||||||
|
shell.quiet_reported += 1;
|
||||||
|
effects.push(Effect::Report(Report::Quiet {
|
||||||
|
market: symbol.market.clone(),
|
||||||
|
connection,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The other connection carries it meanwhile: the book that
|
||||||
|
// comes back is ignored, no Snapshot, no subscription gap.
|
||||||
|
Some(connection) => {
|
||||||
|
log!("{exchange}: {name}: quiet on connection {connection}, resubscribed");
|
||||||
|
shell.quiet += 1;
|
||||||
|
symbol.quiet_on = Some(connection);
|
||||||
|
if let Some(watch) = symbol.watch_mut(connection) {
|
||||||
|
watch.since = now;
|
||||||
|
}
|
||||||
|
effects.push(Effect::Send(connection, Command::Resubscribe(name.clone())));
|
||||||
|
}
|
||||||
|
None => symbol.quiet_on = None,
|
||||||
|
}
|
||||||
|
if R::has_book(&symbol.rule) {
|
||||||
|
// A change held across the minute is written after it.
|
||||||
|
effects.push(Effect::Write(symbol.snapshot(name, received_at)));
|
||||||
|
}
|
||||||
|
self.rule
|
||||||
|
.on_minute(&shell.up, name, symbol, now, &mut effects);
|
||||||
|
}
|
||||||
|
effects
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A book thread: runs until every connection and the orchestrator have
|
||||||
|
/// dropped its channels, then drops its senders so the writer thread finishes
|
||||||
|
/// once every book thread has.
|
||||||
|
// ponytail: unbounded channels never block a socket read nor drop an event;
|
||||||
|
// the ceiling is memory if a consumer stays behind. The minute log's peak of
|
||||||
|
// events waiting makes that visible; bounded channels are a constructor change
|
||||||
|
// if it ever matters.
|
||||||
|
pub fn run<R: Rule>(
|
||||||
|
feed: Feed<R>,
|
||||||
|
events: Receiver<Received<R::Event>>,
|
||||||
|
commands: Receiver<BookCommand<R::Market>>,
|
||||||
|
outputs: Sender<Output>,
|
||||||
|
reports: Sender<Report>,
|
||||||
|
) {
|
||||||
|
serve(feed, Vec::new(), events, commands, outputs, reports)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`run`], knowing `connections` already (by number); the orchestrator
|
||||||
|
/// announces the others as it opens them.
|
||||||
|
pub(crate) fn serve<R: Rule>(
|
||||||
|
mut feed: Feed<R>,
|
||||||
|
mut connections: Vec<Sender<Command>>,
|
||||||
|
events: Receiver<Received<R::Event>>,
|
||||||
|
commands: Receiver<BookCommand<R::Market>>,
|
||||||
|
outputs: Sender<Output>,
|
||||||
|
reports: Sender<Report>,
|
||||||
|
) {
|
||||||
|
let no_commands = never();
|
||||||
|
let mut orchestrator_gone = false;
|
||||||
|
let exchange = feed.shell.exchange;
|
||||||
|
loop {
|
||||||
|
let mut wake = (next_minute(Utc::now()) - Utc::now())
|
||||||
|
.to_std()
|
||||||
|
.unwrap_or(Duration::ZERO);
|
||||||
|
if feed.rule.wakes_soon() {
|
||||||
|
wake = wake.min(WAKE_EVERY);
|
||||||
|
}
|
||||||
|
let commands = if orchestrator_gone {
|
||||||
|
&no_commands
|
||||||
|
} else {
|
||||||
|
&commands
|
||||||
|
};
|
||||||
|
let effects = select! {
|
||||||
|
recv(events) -> received => match received {
|
||||||
|
Ok(received) => {
|
||||||
|
feed.shell.peak_waiting = feed.shell.peak_waiting.max(events.len());
|
||||||
|
feed.on_received(received)
|
||||||
|
}
|
||||||
|
Err(_) => return,
|
||||||
|
},
|
||||||
|
recv(commands) -> command => match command {
|
||||||
|
Ok(BookCommand::Connected(number, connection)) => {
|
||||||
|
assert_eq!(number, connections.len(), "connections open in order");
|
||||||
|
connections.push(connection);
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
Ok(BookCommand::Add(market, on)) => feed.add_market(&market, &on, Utc::now()),
|
||||||
|
Ok(BookCommand::Move { market, from, to }) => {
|
||||||
|
feed.move_market(&market, from, to, Utc::now())
|
||||||
|
}
|
||||||
|
Ok(BookCommand::Remove(market)) => feed.remove_market(&market),
|
||||||
|
// It is stopping: the connections' end still decides ours.
|
||||||
|
Err(_) => {
|
||||||
|
orchestrator_gone = true;
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
},
|
||||||
|
default(wake) => feed.idle(Utc::now()),
|
||||||
|
};
|
||||||
|
for effect in effects {
|
||||||
|
match effect {
|
||||||
|
Effect::Write(output) => {
|
||||||
|
if outputs.send(output).is_err() {
|
||||||
|
return log!("{exchange}: the writer thread is gone, stopping");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The connection may be gone already, while stopping, and the
|
||||||
|
// orchestrator too.
|
||||||
|
Effect::Send(connection, command) => _ = connections[connection].send(command),
|
||||||
|
Effect::Report(report) => _ = reports.send(report),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn next_minute(now: DateTime<Utc>) -> DateTime<Utc> {
|
||||||
|
now.duration_trunc(TimeDelta::minutes(1))
|
||||||
|
.expect("a minute divides any UTC time")
|
||||||
|
+ TimeDelta::minutes(1)
|
||||||
|
}
|
||||||
+187
@@ -0,0 +1,187 @@
|
|||||||
|
//! The exchange-agnostic collector (ADR 0004): the orchestrator, the
|
||||||
|
//! connections and the book threads, generic over an [`Exchange`], which
|
||||||
|
//! supplies the websocket layer, the fetchers and the rule a book thread
|
||||||
|
//! applies to each event.
|
||||||
|
|
||||||
|
pub mod feed;
|
||||||
|
pub mod orchestrator;
|
||||||
|
pub mod socket;
|
||||||
|
|
||||||
|
use std::fmt::Debug;
|
||||||
|
use std::hash::{DefaultHasher, Hash, Hasher};
|
||||||
|
use std::thread;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use crossbeam::channel::{Receiver, Sender};
|
||||||
|
|
||||||
|
use feed::Rule;
|
||||||
|
|
||||||
|
/// How many book threads own the books. One handled ~30,000 frames a second
|
||||||
|
/// in sprint 02.
|
||||||
|
pub const BOOK_THREADS: usize = 16;
|
||||||
|
|
||||||
|
/// The book thread that owns a market's books, from a hash of its id.
|
||||||
|
/// Computed, so the orchestrator and every connection agree without a shared
|
||||||
|
/// table.
|
||||||
|
pub fn book_thread(market: &str) -> usize {
|
||||||
|
let mut hasher = DefaultHasher::new();
|
||||||
|
market.hash(&mut hasher);
|
||||||
|
(hasher.finish() % BOOK_THREADS as u64) as usize
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An exchange, as the collector uses it.
|
||||||
|
pub trait Exchange: Send + Sync + 'static {
|
||||||
|
type Market: Market;
|
||||||
|
type Event: Event;
|
||||||
|
type Session: Session<Event = Self::Event>;
|
||||||
|
type Rule: Rule<Market = Self::Market, Event = Self::Event>;
|
||||||
|
|
||||||
|
/// What its log lines start with.
|
||||||
|
const NAME: &'static str;
|
||||||
|
|
||||||
|
/// The websocket's handshake request, made again for every connect.
|
||||||
|
fn request(&self) -> anyhow::Result<tungstenite::handshake::client::Request>;
|
||||||
|
|
||||||
|
/// A connection's state on the exchange's side, fresh for every connect.
|
||||||
|
fn session(&self) -> Self::Session;
|
||||||
|
|
||||||
|
/// A book thread's rule.
|
||||||
|
fn rule(&self) -> Self::Rule;
|
||||||
|
|
||||||
|
/// A fetcher: answers each request with findings until either channel is
|
||||||
|
/// gone. Two run, so a full scan never holds up the minute check.
|
||||||
|
fn fetch(&self, requests: Receiver<Request>, findings: Sender<Finding<Self::Market>>);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A live market: its id and its symbols, plus what its exchange needs.
|
||||||
|
pub trait Market: Clone + Debug + Send + 'static {
|
||||||
|
/// The id it is tracked and routed by.
|
||||||
|
fn id(&self) -> &str;
|
||||||
|
/// Its symbols, each with a book.
|
||||||
|
fn symbols(&self) -> &[String];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An exchange's event, as a connection routes it.
|
||||||
|
pub trait Event: Debug + Send + 'static {
|
||||||
|
/// The market it belongs to, which decides its book thread; none for an
|
||||||
|
/// event no book needs.
|
||||||
|
fn market(&self) -> Option<&str>;
|
||||||
|
/// Whether it is a full book: a market's first from a connection ends a
|
||||||
|
/// move to it.
|
||||||
|
fn is_book(&self) -> bool;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One connection's side of the exchange: the frames that carry what a book
|
||||||
|
/// thread asks for, and the events of each frame received.
|
||||||
|
pub trait Session: Send {
|
||||||
|
type Event;
|
||||||
|
|
||||||
|
/// Sent on connect, before any subscribe.
|
||||||
|
fn opening(&mut self) -> Vec<String>;
|
||||||
|
/// The frames that subscribe `symbols` on an open connection.
|
||||||
|
fn subscribe(&mut self, symbols: &[&String]) -> Vec<String>;
|
||||||
|
fn unsubscribe(&mut self, symbols: &[&String]) -> Vec<String>;
|
||||||
|
/// The text sent every few seconds to keep the connection open, if the
|
||||||
|
/// exchange wants one.
|
||||||
|
fn keepalive(&self) -> Option<&'static str>;
|
||||||
|
/// A frame's events.
|
||||||
|
fn decode(&mut self, text: &str) -> Result<Vec<Self::Event>, Undecoded>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Why a frame gave no events.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum Undecoded {
|
||||||
|
/// Not a message we read: it is logged and dropped.
|
||||||
|
Unreadable(String),
|
||||||
|
/// The connection itself lost a message: it reconnects.
|
||||||
|
Lost(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A fetcher's request, tried again after `wait` when it fails: `tries` times
|
||||||
|
/// in all.
|
||||||
|
pub fn retried<T>(
|
||||||
|
tries: u32,
|
||||||
|
wait: Duration,
|
||||||
|
mut fetch: impl FnMut() -> anyhow::Result<T>,
|
||||||
|
) -> anyhow::Result<T> {
|
||||||
|
let mut tried = 1;
|
||||||
|
loop {
|
||||||
|
match fetch() {
|
||||||
|
Err(_) if tried < tries => {
|
||||||
|
tried += 1;
|
||||||
|
thread::sleep(wait);
|
||||||
|
}
|
||||||
|
result => return result,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the orchestrator asks a fetcher for.
|
||||||
|
pub enum Request {
|
||||||
|
/// Every live market.
|
||||||
|
Rescan,
|
||||||
|
/// The live markets created and the markets closed since then.
|
||||||
|
Recent { since: DateTime<Utc> },
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a fetcher reports.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum Finding<M> {
|
||||||
|
/// A rescan began: once it is done, a market known now and not seen in it
|
||||||
|
/// is gone.
|
||||||
|
ScanStarted,
|
||||||
|
/// A page of live markets.
|
||||||
|
ScanPage(Vec<M>),
|
||||||
|
ScanDone,
|
||||||
|
/// The scan gave up: what it reported says nothing of the rest.
|
||||||
|
ScanFailed,
|
||||||
|
/// Live markets created since the request's time.
|
||||||
|
Created(Vec<M>),
|
||||||
|
/// The ids of markets closed since then.
|
||||||
|
Closed(Vec<String>),
|
||||||
|
RecentDone,
|
||||||
|
RecentFailed,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a connection sends a book thread.
|
||||||
|
// Nearly every message is an event: boxing it to shrink the two rare markers
|
||||||
|
// would cost an allocation per event for nothing.
|
||||||
|
#[allow(clippy::large_enum_variant)]
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum Received<E> {
|
||||||
|
Event(Arrival<E>),
|
||||||
|
/// The connection came up: its subscriptions are being sent.
|
||||||
|
Up(usize),
|
||||||
|
/// The connection went down: nothing more comes from it until it is up.
|
||||||
|
Down(usize),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One event of a frame on its way to its market's book thread.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct Arrival<E> {
|
||||||
|
/// The frame's receive time, stamped right after the socket read.
|
||||||
|
pub received_at: DateTime<Utc>,
|
||||||
|
/// The connection that delivered it.
|
||||||
|
pub connection: usize,
|
||||||
|
/// Its share of the frame's bytes: a frame is one market's change, or
|
||||||
|
/// books of several, each taking its share.
|
||||||
|
pub bytes: u64,
|
||||||
|
pub event: E,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_market_always_gets_the_same_book_thread_and_markets_spread() {
|
||||||
|
let threads: std::collections::BTreeSet<usize> = (0..40)
|
||||||
|
.map(|market| book_thread(&format!("KXMARKET-{market}")))
|
||||||
|
.collect();
|
||||||
|
assert!(threads.iter().all(|&thread| thread < BOOK_THREADS));
|
||||||
|
// 40 markets over 16 threads: several threads, not one.
|
||||||
|
assert!(threads.len() > 5, "{threads:?}");
|
||||||
|
assert_eq!(book_thread("0xabc"), book_thread(&String::from("0xabc")));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
//! The orchestrator: the one owner of what is tracked. It keeps the tracked
|
//! The orchestrator: the one owner of what is tracked. It keeps the tracked
|
||||||
//! markets equal to Gamma's live ones, decides which connections carry each
|
//! markets equal to the exchange's live ones, decides which connections carry
|
||||||
//! market and owns every timer, recycling connections among them; the two
|
//! each market and owns every timer, recycling connections among them; the
|
||||||
//! fetchers only fetch (ADR 0002, ADR 0003, roadmap/04, roadmap/05).
|
//! two fetchers only fetch (ADR 0002, ADR 0003, roadmap/04, roadmap/05).
|
||||||
|
|
||||||
use std::cmp::Reverse;
|
use std::cmp::Reverse;
|
||||||
use std::collections::{HashMap, HashSet, VecDeque};
|
use std::collections::{HashMap, HashSet, VecDeque};
|
||||||
@@ -15,17 +15,15 @@ use crossbeam::channel::{Sender, select, unbounded};
|
|||||||
use engineering_challenge::book::Output;
|
use engineering_challenge::book::Output;
|
||||||
|
|
||||||
use super::feed::{self, BookCommand, Feed, Loads, Report};
|
use super::feed::{self, BookCommand, Feed, Loads, Report};
|
||||||
use super::gamma::{self, Finding, Request};
|
use super::socket::{self, Command, Notice};
|
||||||
use super::socket::{Command, Notice};
|
use super::{BOOK_THREADS, Exchange, Finding, Market, Received, Request, book_thread};
|
||||||
use super::wire::Received;
|
|
||||||
use super::{BOOK_THREADS, Market, book_thread, socket};
|
|
||||||
|
|
||||||
/// How many connections carry each market, first one wins (ADR 0003). 1 for
|
/// How many connections carry each market, first one wins (ADR 0003). 1 for
|
||||||
/// a run to compare with.
|
/// a run to compare with.
|
||||||
const SUBSCRIPTIONS: usize = 2;
|
const SUBSCRIPTIONS: usize = 2;
|
||||||
/// The most tokens a connection holds: a reconnect replays every one's book
|
/// The most symbols a connection holds: a reconnect replays every one's book
|
||||||
/// at once.
|
/// at once.
|
||||||
const CONNECTION_TOKENS: usize = 5_000;
|
const CONNECTION_SYMBOLS: usize = 5_000;
|
||||||
/// What the startup deal fills a connection to, leaving room for new markets.
|
/// What the startup deal fills a connection to, leaving room for new markets.
|
||||||
const STARTUP_FILL: usize = 4_500;
|
const STARTUP_FILL: usize = 4_500;
|
||||||
/// A connection's byte budget, under the lowest rate at which one was seen to
|
/// A connection's byte budget, under the lowest rate at which one was seen to
|
||||||
@@ -46,10 +44,10 @@ const MOVED_STAYS: Duration = Duration::from_secs(10 * 60);
|
|||||||
const SPLIT_WITHIN: Duration = Duration::from_secs(10 * 60);
|
const SPLIT_WITHIN: Duration = Duration::from_secs(10 * 60);
|
||||||
const RECENT_EVERY: Duration = Duration::from_secs(60);
|
const RECENT_EVERY: Duration = Duration::from_secs(60);
|
||||||
const RESCAN_EVERY: Duration = Duration::from_secs(60 * 60);
|
const RESCAN_EVERY: Duration = Duration::from_secs(60 * 60);
|
||||||
/// Gamma's indexing delay: each recent check reaches this far before the
|
/// The listing's indexing delay: each recent check reaches this far before
|
||||||
/// last successful one. New markets were listed up to 5 minutes after their
|
/// the last successful one. New markets were listed up to 5 minutes after
|
||||||
/// `createdAt` (docs/notes.md); known markets are skipped, so the margin costs
|
/// their creation (docs/notes.md); known markets are skipped, so the margin
|
||||||
/// only the pages it reads again.
|
/// costs only the pages it reads again.
|
||||||
const RECENT_MARGIN: TimeDelta = TimeDelta::minutes(15);
|
const RECENT_MARGIN: TimeDelta = TimeDelta::minutes(15);
|
||||||
/// Each connection is recycled about this often, one at a time, to shed what
|
/// Each connection is recycled about this often, one at a time, to shed what
|
||||||
/// accumulates on the exchange's side of it.
|
/// accumulates on the exchange's side of it.
|
||||||
@@ -60,7 +58,12 @@ const WAKE_EVERY: Duration = Duration::from_secs(1);
|
|||||||
/// Runs until `stop` is set, then waits for every connection and book thread.
|
/// Runs until `stop` is set, then waits for every connection and book thread.
|
||||||
/// Returns how many threads failed: those that panicked, and one if a fetcher
|
/// Returns how many threads failed: those that panicked, and one if a fetcher
|
||||||
/// died.
|
/// died.
|
||||||
pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
pub fn run<E: Exchange>(
|
||||||
|
exchange: Arc<E>,
|
||||||
|
outputs: Sender<Output>,
|
||||||
|
stop: &Arc<AtomicBool>,
|
||||||
|
) -> usize {
|
||||||
|
let name = E::NAME;
|
||||||
let (findings_tx, findings) = unbounded();
|
let (findings_tx, findings) = unbounded();
|
||||||
let (rescan, rescan_requests) = unbounded();
|
let (rescan, rescan_requests) = unbounded();
|
||||||
let (recent, recent_requests) = unbounded();
|
let (recent, recent_requests) = unbounded();
|
||||||
@@ -69,14 +72,14 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
let fetchers: Vec<JoinHandle<()>> = [rescan_requests, recent_requests]
|
let fetchers: Vec<JoinHandle<()>> = [rescan_requests, recent_requests]
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|requests| {
|
.map(|requests| {
|
||||||
let findings = findings_tx.clone();
|
let (findings, exchange) = (findings_tx.clone(), Arc::clone(&exchange));
|
||||||
thread::spawn(move || gamma::run(requests, findings))
|
thread::spawn(move || exchange.fetch(requests, findings))
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
drop(findings_tx);
|
drop(findings_tx);
|
||||||
let (reports_tx, reports) = unbounded::<Report>();
|
let (reports_tx, reports) = unbounded::<Report>();
|
||||||
let (notices_tx, notices) = unbounded::<Notice>();
|
let (notices_tx, notices) = unbounded::<Notice>();
|
||||||
let mut threads = Threads::start(&outputs, &reports_tx, ¬ices_tx, stop);
|
let mut threads = Threads::start(exchange, &outputs, &reports_tx, ¬ices_tx, stop);
|
||||||
drop(notices_tx);
|
drop(notices_tx);
|
||||||
// The book threads hold the only senders the writer waits for.
|
// The book threads hold the only senders the writer waits for.
|
||||||
drop(outputs);
|
drop(outputs);
|
||||||
@@ -89,10 +92,10 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
// Until the first scan is done: the live markets it found, waiting to be
|
// Until the first scan is done: the live markets it found, waiting to be
|
||||||
// dealt at random. Dealing page by page would keep markets created
|
// dealt at random. Dealing page by page would keep markets created
|
||||||
// together on the same connections.
|
// together on the same connections.
|
||||||
let mut first_scan: Option<Vec<Market>> = Some(Vec::new());
|
let mut first_scan: Option<Vec<E::Market>> = Some(Vec::new());
|
||||||
// The startup deal's commands still to send, and whatever came behind
|
// The startup deal's commands still to send, and whatever came behind
|
||||||
// them: a market's commands keep their order.
|
// them: a market's commands keep their order.
|
||||||
let mut queue: VecDeque<BookCommand> = VecDeque::new();
|
let mut queue: VecDeque<BookCommand<E::Market>> = VecDeque::new();
|
||||||
let mut next_deal = Instant::now();
|
let mut next_deal = Instant::now();
|
||||||
let (mut added, mut removed, mut moved, mut splits) = (0, 0, 0, 0);
|
let (mut added, mut removed, mut moved, mut splits) = (0, 0, 0, 0);
|
||||||
let (mut recycled, mut quiet_moves, mut silent) = (0, 0, 0);
|
let (mut recycled, mut quiet_moves, mut silent) = (0, 0, 0);
|
||||||
@@ -110,7 +113,7 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
// of a request, whose answer would never come: nothing would keep the
|
// of a request, whose answer would never come: nothing would keep the
|
||||||
// set current.
|
// set current.
|
||||||
if fetchers.iter().any(JoinHandle::is_finished) {
|
if fetchers.iter().any(JoinHandle::is_finished) {
|
||||||
log!("polymarket: orchestrator: a fetcher died, stopping");
|
log!("{name}: orchestrator: a fetcher died, stopping");
|
||||||
stop.store(true, Ordering::Relaxed);
|
stop.store(true, Ordering::Relaxed);
|
||||||
fetcher_died = true;
|
fetcher_died = true;
|
||||||
break;
|
break;
|
||||||
@@ -138,7 +141,7 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
commands.extend(moves);
|
commands.extend(moves);
|
||||||
let busiest = tracker.connections.iter().map(|one| one.load).max();
|
let busiest = tracker.connections.iter().map(|one| one.load).max();
|
||||||
log!(
|
log!(
|
||||||
"polymarket: orchestrator: {} markets on {} connections; {added} added, {removed} \
|
"{name}: orchestrator: {} markets on {} connections; {added} added, {removed} \
|
||||||
removed, {moved} moved ({quiet_moves} quiet), {splits} splits, {recycled} recycled, \
|
removed, {moved} moved ({quiet_moves} quiet), {splits} splits, {recycled} recycled, \
|
||||||
{silent} silent this minute; busiest connection at {} bytes/s",
|
{silent} silent this minute; busiest connection at {} bytes/s",
|
||||||
tracker.markets.len(),
|
tracker.markets.len(),
|
||||||
@@ -217,7 +220,7 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
Ok(Finding::Closed(ids)) => {
|
Ok(Finding::Closed(ids)) => {
|
||||||
// Closed while the first scan runs: never dealt.
|
// Closed while the first scan runs: never dealt.
|
||||||
if let Some(waiting) = &mut first_scan {
|
if let Some(waiting) = &mut first_scan {
|
||||||
waiting.retain(|market| !ids.contains(&market.condition_id));
|
waiting.retain(|market| !ids.iter().any(|id| id == market.id()));
|
||||||
}
|
}
|
||||||
let gone: Vec<_> = ids.iter().filter_map(|id| tracker.remove(id)).collect();
|
let gone: Vec<_> = ids.iter().filter_map(|id| tracker.remove(id)).collect();
|
||||||
removed += gone.len();
|
removed += gone.len();
|
||||||
@@ -237,7 +240,7 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
Ok(Report::Quiet { market, connection }) => {
|
Ok(Report::Quiet { market, connection }) => {
|
||||||
if let Some(command) = tracker.quiet(&market, connection, Instant::now()) {
|
if let Some(command) = tracker.quiet(&market, connection, Instant::now()) {
|
||||||
log!(
|
log!(
|
||||||
"polymarket: orchestrator: {market} still quiet on connection \
|
"{name}: orchestrator: {market} still quiet on connection \
|
||||||
{connection} after a resubscribe, moved"
|
{connection} after a resubscribe, moved"
|
||||||
);
|
);
|
||||||
quiet_moves += 1;
|
quiet_moves += 1;
|
||||||
@@ -255,7 +258,7 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
let moves = tracker.closed(connection, Instant::now());
|
let moves = tracker.closed(connection, Instant::now());
|
||||||
if !moves.is_empty() {
|
if !moves.is_empty() {
|
||||||
log!(
|
log!(
|
||||||
"polymarket: orchestrator: connection {connection} closed with 1013 \
|
"{name}: orchestrator: connection {connection} closed with 1013 \
|
||||||
twice, split: {} markets moved off",
|
twice, split: {} markets moved off",
|
||||||
moves.len()
|
moves.len()
|
||||||
);
|
);
|
||||||
@@ -285,11 +288,12 @@ pub fn run(outputs: Sender<Output>, stop: &Arc<AtomicBool>) -> usize {
|
|||||||
|
|
||||||
/// The threads the orchestrator feeds: the book threads, there from the
|
/// The threads the orchestrator feeds: the book threads, there from the
|
||||||
/// start, and the connections it opens.
|
/// start, and the connections it opens.
|
||||||
struct Threads {
|
struct Threads<E: Exchange> {
|
||||||
book_commands: Vec<Sender<BookCommand>>,
|
exchange: Arc<E>,
|
||||||
|
book_commands: Vec<Sender<BookCommand<E::Market>>>,
|
||||||
/// Every book thread's events channel, by number: each connection gets a
|
/// Every book thread's events channel, by number: each connection gets a
|
||||||
/// clone of them all.
|
/// clone of them all.
|
||||||
events: Vec<Sender<Received>>,
|
events: Vec<Sender<Received<E::Event>>>,
|
||||||
notices: Sender<Notice>,
|
notices: Sender<Notice>,
|
||||||
/// Each connection's commands, by number, for its recycles.
|
/// Each connection's commands, by number, for its recycles.
|
||||||
connection_commands: Vec<Sender<Command>>,
|
connection_commands: Vec<Sender<Command>>,
|
||||||
@@ -298,14 +302,16 @@ struct Threads {
|
|||||||
stop: Arc<AtomicBool>,
|
stop: Arc<AtomicBool>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Threads {
|
impl<E: Exchange> Threads<E> {
|
||||||
fn start(
|
fn start(
|
||||||
|
exchange: Arc<E>,
|
||||||
outputs: &Sender<Output>,
|
outputs: &Sender<Output>,
|
||||||
reports: &Sender<Report>,
|
reports: &Sender<Report>,
|
||||||
notices: &Sender<Notice>,
|
notices: &Sender<Notice>,
|
||||||
stop: &Arc<AtomicBool>,
|
stop: &Arc<AtomicBool>,
|
||||||
) -> Self {
|
) -> Self {
|
||||||
let mut threads = Self {
|
let mut threads = Self {
|
||||||
|
exchange,
|
||||||
book_commands: Vec::new(),
|
book_commands: Vec::new(),
|
||||||
events: Vec::new(),
|
events: Vec::new(),
|
||||||
notices: notices.clone(),
|
notices: notices.clone(),
|
||||||
@@ -318,10 +324,11 @@ impl Threads {
|
|||||||
let (commands, commands_rx) = unbounded();
|
let (commands, commands_rx) = unbounded();
|
||||||
let (events, events_rx) = unbounded();
|
let (events, events_rx) = unbounded();
|
||||||
let (outputs, reports) = (outputs.clone(), reports.clone());
|
let (outputs, reports) = (outputs.clone(), reports.clone());
|
||||||
|
let feed = Feed::new(E::NAME, number, threads.exchange.rule());
|
||||||
threads.book_commands.push(commands);
|
threads.book_commands.push(commands);
|
||||||
threads.events.push(events);
|
threads.events.push(events);
|
||||||
threads.book_threads.push(thread::spawn(move || {
|
threads.book_threads.push(thread::spawn(move || {
|
||||||
feed::run(Feed::new(number), events_rx, commands_rx, outputs, reports)
|
feed::run(feed, events_rx, commands_rx, outputs, reports)
|
||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
threads
|
threads
|
||||||
@@ -334,10 +341,10 @@ impl Threads {
|
|||||||
let number = self.connections.len();
|
let number = self.connections.len();
|
||||||
let (commands, commands_rx) = unbounded();
|
let (commands, commands_rx) = unbounded();
|
||||||
let (events, stop) = (self.events.clone(), Arc::clone(&self.stop));
|
let (events, stop) = (self.events.clone(), Arc::clone(&self.stop));
|
||||||
let notices = self.notices.clone();
|
let (notices, exchange) = (self.notices.clone(), Arc::clone(&self.exchange));
|
||||||
self.connections.push(thread::spawn(move || {
|
self.connections.push(thread::spawn(move || {
|
||||||
socket::run(
|
socket::run(
|
||||||
socket::MARKET_CHANNEL_URL,
|
&*exchange,
|
||||||
number,
|
number,
|
||||||
&events,
|
&events,
|
||||||
&commands_rx,
|
&commands_rx,
|
||||||
@@ -359,14 +366,14 @@ impl Threads {
|
|||||||
|
|
||||||
/// To the market's book thread. One that died keeps its markets unserved,
|
/// To the market's book thread. One that died keeps its markets unserved,
|
||||||
/// and its connections stop the run.
|
/// and its connections stop the run.
|
||||||
fn send(&self, command: BookCommand) {
|
fn send(&self, command: BookCommand<E::Market>) {
|
||||||
let market = match &command {
|
let market = match &command {
|
||||||
BookCommand::Add(market, _)
|
BookCommand::Add(market, _)
|
||||||
| BookCommand::Move { market, .. }
|
| BookCommand::Move { market, .. }
|
||||||
| BookCommand::Remove(market) => market,
|
| BookCommand::Remove(market) => market,
|
||||||
BookCommand::Connected(..) => unreachable!("sent by open() to every book thread"),
|
BookCommand::Connected(..) => unreachable!("sent by open() to every book thread"),
|
||||||
};
|
};
|
||||||
_ = self.book_commands[book_thread(&market.condition_id)].send(command);
|
_ = self.book_commands[book_thread(market.id())].send(command);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Waits for the connections, which stop on the flag, then for the book
|
/// Waits for the connections, which stop on the flag, then for the book
|
||||||
@@ -388,8 +395,8 @@ impl Threads {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// A tracked market, where it is and what it weighs.
|
/// A tracked market, where it is and what it weighs.
|
||||||
struct Tracked {
|
struct Tracked<M> {
|
||||||
market: Market,
|
market: M,
|
||||||
/// The connections that carry it, all different.
|
/// The connections that carry it, all different.
|
||||||
connections: Vec<usize>,
|
connections: Vec<usize>,
|
||||||
/// Bytes per second on one connection, over its book thread's last
|
/// Bytes per second on one connection, over its book thread's last
|
||||||
@@ -398,11 +405,11 @@ struct Tracked {
|
|||||||
moved_at: Option<Instant>,
|
moved_at: Option<Instant>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A connection's tokens and load, against its two limits, its 1013 closes,
|
/// A connection's symbols and load, against its two limits, its 1013 closes,
|
||||||
/// and whether and since when it is up.
|
/// and whether and since when it is up.
|
||||||
#[derive(Debug, Default, Clone, PartialEq, Eq)]
|
#[derive(Debug, Default, Clone, PartialEq, Eq)]
|
||||||
struct ConnectionState {
|
struct ConnectionState {
|
||||||
tokens: usize,
|
symbols: usize,
|
||||||
/// The sum of its markets' loads, bytes per second.
|
/// The sum of its markets' loads, bytes per second.
|
||||||
load: u64,
|
load: u64,
|
||||||
/// When it was last closed with 1013.
|
/// When it was last closed with 1013.
|
||||||
@@ -415,10 +422,10 @@ struct ConnectionState {
|
|||||||
|
|
||||||
/// Which connections carry each market and what it weighs, and which markets
|
/// Which connections carry each market and what it weighs, and which markets
|
||||||
/// a rescan in progress hasn't seen yet.
|
/// a rescan in progress hasn't seen yet.
|
||||||
struct Tracker {
|
struct Tracker<M> {
|
||||||
subscriptions: usize,
|
subscriptions: usize,
|
||||||
/// By `conditionId`.
|
/// By market id.
|
||||||
markets: HashMap<String, Tracked>,
|
markets: HashMap<String, Tracked<M>>,
|
||||||
/// By connection number.
|
/// By connection number.
|
||||||
connections: Vec<ConnectionState>,
|
connections: Vec<ConnectionState>,
|
||||||
/// While a rescan runs: the markets known at its start, not seen since.
|
/// While a rescan runs: the markets known at its start, not seen since.
|
||||||
@@ -430,7 +437,7 @@ struct Tracker {
|
|||||||
recycling: Option<usize>,
|
recycling: Option<usize>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Tracker {
|
impl<M: Market> Tracker<M> {
|
||||||
fn new(subscriptions: usize, seed: u64) -> Self {
|
fn new(subscriptions: usize, seed: u64) -> Self {
|
||||||
Self {
|
Self {
|
||||||
subscriptions,
|
subscriptions,
|
||||||
@@ -447,14 +454,14 @@ impl Tracker {
|
|||||||
/// connections, each subscription dealt in an order of its own, so a
|
/// connections, each subscription dealt in an order of its own, so a
|
||||||
/// market's connections hold different neighbours and no connection gets
|
/// market's connections hold different neighbours and no connection gets
|
||||||
/// the markets created together.
|
/// the markets created together.
|
||||||
fn deal(&mut self, markets: Vec<Market>) -> Vec<BookCommand> {
|
fn deal(&mut self, markets: Vec<M>) -> Vec<BookCommand<M>> {
|
||||||
let mut known = HashSet::new();
|
let mut known = HashSet::new();
|
||||||
let markets: Vec<Market> = markets
|
let markets: Vec<M> = markets
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.filter(|market| known.insert(market.condition_id.clone()))
|
.filter(|market| known.insert(market.id().to_string()))
|
||||||
.collect();
|
.collect();
|
||||||
let tokens: usize = markets.iter().map(|market| market.tokens.len()).sum();
|
let symbols: usize = markets.iter().map(|market| market.symbols().len()).sum();
|
||||||
let count = (self.subscriptions * tokens)
|
let count = (self.subscriptions * symbols)
|
||||||
.div_ceil(STARTUP_FILL)
|
.div_ceil(STARTUP_FILL)
|
||||||
.max(self.subscriptions);
|
.max(self.subscriptions);
|
||||||
self.connections = vec![ConnectionState::default(); count];
|
self.connections = vec![ConnectionState::default(); count];
|
||||||
@@ -468,7 +475,7 @@ impl Tracker {
|
|||||||
connection = (connection + 1) % count;
|
connection = (connection + 1) % count;
|
||||||
}
|
}
|
||||||
placed[market].push(connection);
|
placed[market].push(connection);
|
||||||
self.connections[connection].tokens += markets[market].tokens.len();
|
self.connections[connection].symbols += markets[market].symbols().len();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
markets
|
markets
|
||||||
@@ -480,43 +487,43 @@ impl Tracker {
|
|||||||
|
|
||||||
/// A market it doesn't know goes on different random connections with
|
/// A market it doesn't know goes on different random connections with
|
||||||
/// room: its Add.
|
/// room: its Add.
|
||||||
fn add(&mut self, market: Market) -> Option<BookCommand> {
|
fn add(&mut self, market: M) -> Option<BookCommand<M>> {
|
||||||
if let Some(unseen) = &mut self.unseen {
|
if let Some(unseen) = &mut self.unseen {
|
||||||
unseen.remove(&market.condition_id);
|
unseen.remove(market.id());
|
||||||
}
|
}
|
||||||
if self.markets.contains_key(&market.condition_id) {
|
if self.markets.contains_key(market.id()) {
|
||||||
return None;
|
return None;
|
||||||
}
|
}
|
||||||
let mut connections = Vec::new();
|
let mut connections = Vec::new();
|
||||||
for _ in 0..self.subscriptions {
|
for _ in 0..self.subscriptions {
|
||||||
let connection = self.pick(market.tokens.len(), 0, &connections);
|
let connection = self.pick(market.symbols().len(), 0, &connections);
|
||||||
self.connections[connection].tokens += market.tokens.len();
|
self.connections[connection].symbols += market.symbols().len();
|
||||||
connections.push(connection);
|
connections.push(connection);
|
||||||
}
|
}
|
||||||
Some(self.track(market, connections))
|
Some(self.track(market, connections))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn track(&mut self, market: Market, connections: Vec<usize>) -> BookCommand {
|
fn track(&mut self, market: M, connections: Vec<usize>) -> BookCommand<M> {
|
||||||
let tracked = Tracked {
|
let tracked = Tracked {
|
||||||
market: market.clone(),
|
market: market.clone(),
|
||||||
connections: connections.clone(),
|
connections: connections.clone(),
|
||||||
load: 0,
|
load: 0,
|
||||||
moved_at: None,
|
moved_at: None,
|
||||||
};
|
};
|
||||||
self.markets.insert(market.condition_id.clone(), tracked);
|
self.markets.insert(market.id().to_string(), tracked);
|
||||||
BookCommand::Add(market, connections)
|
BookCommand::Add(market, connections)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A random connection with room for `tokens` more and `load` more
|
/// A random connection with room for `symbols` more and `load` more
|
||||||
/// bytes/s, none of `except`; a new connection if none has. Random, since
|
/// bytes/s, none of `except`; a new connection if none has. Random, since
|
||||||
/// the least loaded one would take every market of a batch created
|
/// the least loaded one would take every market of a batch created
|
||||||
/// together: the cluster a load average can't see.
|
/// together: the cluster a load average can't see.
|
||||||
fn pick(&mut self, tokens: usize, load: u64, except: &[usize]) -> usize {
|
fn pick(&mut self, symbols: usize, load: u64, except: &[usize]) -> usize {
|
||||||
let with_room: Vec<usize> = (0..self.connections.len())
|
let with_room: Vec<usize> = (0..self.connections.len())
|
||||||
.filter(|number| {
|
.filter(|number| {
|
||||||
let connection = &self.connections[*number];
|
let connection = &self.connections[*number];
|
||||||
!except.contains(number)
|
!except.contains(number)
|
||||||
&& connection.tokens + tokens <= CONNECTION_TOKENS
|
&& connection.symbols + symbols <= CONNECTION_SYMBOLS
|
||||||
&& connection.load + load <= CONNECTION_BYTES_PER_SECOND
|
&& connection.load + load <= CONNECTION_BYTES_PER_SECOND
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
@@ -527,11 +534,11 @@ impl Tracker {
|
|||||||
with_room[self.random.below(with_room.len())]
|
with_room[self.random.below(with_room.len())]
|
||||||
}
|
}
|
||||||
|
|
||||||
fn remove(&mut self, condition_id: &str) -> Option<BookCommand> {
|
fn remove(&mut self, id: &str) -> Option<BookCommand<M>> {
|
||||||
let tracked = self.markets.remove(condition_id)?;
|
let tracked = self.markets.remove(id)?;
|
||||||
for &connection in &tracked.connections {
|
for &connection in &tracked.connections {
|
||||||
let connection = &mut self.connections[connection];
|
let connection = &mut self.connections[connection];
|
||||||
connection.tokens -= tracked.market.tokens.len();
|
connection.symbols -= tracked.market.symbols().len();
|
||||||
connection.load -= tracked.load;
|
connection.load -= tracked.load;
|
||||||
}
|
}
|
||||||
Some(BookCommand::Remove(tracked.market))
|
Some(BookCommand::Remove(tracked.market))
|
||||||
@@ -540,8 +547,8 @@ impl Tracker {
|
|||||||
/// A book thread's loads per market, each weighing on every connection
|
/// A book thread's loads per market, each weighing on every connection
|
||||||
/// that carries the market.
|
/// that carries the market.
|
||||||
fn report(&mut self, loads: &Loads) {
|
fn report(&mut self, loads: &Loads) {
|
||||||
for (condition_id, &load) in loads {
|
for (id, &load) in loads {
|
||||||
let Some(tracked) = self.markets.get_mut(condition_id) else {
|
let Some(tracked) = self.markets.get_mut(id) else {
|
||||||
continue;
|
continue;
|
||||||
};
|
};
|
||||||
for &connection in &tracked.connections {
|
for &connection in &tracked.connections {
|
||||||
@@ -556,7 +563,7 @@ impl Tracker {
|
|||||||
/// byte budget, busiest connection first, each to a random connection with
|
/// byte budget, busiest connection first, each to a random connection with
|
||||||
/// room that doesn't carry the market, until it is under: at most
|
/// room that doesn't carry the market, until it is under: at most
|
||||||
/// [`MOVES_PER_MINUTE`], and none moved in the last [`MOVED_STAYS`].
|
/// [`MOVES_PER_MINUTE`], and none moved in the last [`MOVED_STAYS`].
|
||||||
fn rebalance(&mut self, now: Instant) -> Vec<BookCommand> {
|
fn rebalance(&mut self, now: Instant) -> Vec<BookCommand<M>> {
|
||||||
let mut moves = Vec::new();
|
let mut moves = Vec::new();
|
||||||
let mut over: Vec<usize> = (0..self.connections.len())
|
let mut over: Vec<usize> = (0..self.connections.len())
|
||||||
.filter(|&number| self.connections[number].load > CONNECTION_BYTES_PER_SECOND)
|
.filter(|&number| self.connections[number].load > CONNECTION_BYTES_PER_SECOND)
|
||||||
@@ -579,12 +586,12 @@ impl Tracker {
|
|||||||
.collect();
|
.collect();
|
||||||
busiest.sort_by_key(|&(_, load)| Reverse(load));
|
busiest.sort_by_key(|&(_, load)| Reverse(load));
|
||||||
for (id, load) in busiest {
|
for (id, load) in busiest {
|
||||||
let size = self.markets[&id].market.tokens.len();
|
let size = self.markets[&id].market.symbols().len();
|
||||||
// Alone on its connection, a market would only take its load
|
// Alone on its connection, a market would only take its load
|
||||||
// along; one with no load would lighten nothing.
|
// along; one with no load would lighten nothing.
|
||||||
if self.connections[from].load <= CONNECTION_BYTES_PER_SECOND
|
if self.connections[from].load <= CONNECTION_BYTES_PER_SECOND
|
||||||
|| moves.len() == MOVES_PER_MINUTE
|
|| moves.len() == MOVES_PER_MINUTE
|
||||||
|| self.connections[from].tokens == size
|
|| self.connections[from].symbols == size
|
||||||
|| load == 0
|
|| load == 0
|
||||||
{
|
{
|
||||||
break;
|
break;
|
||||||
@@ -597,14 +604,14 @@ impl Tracker {
|
|||||||
|
|
||||||
/// Moves the market's subscription on `from` to a random connection with
|
/// Moves the market's subscription on `from` to a random connection with
|
||||||
/// room that doesn't carry it yet: its Move.
|
/// room that doesn't carry it yet: its Move.
|
||||||
fn move_off(&mut self, id: &str, from: usize, now: Instant) -> BookCommand {
|
fn move_off(&mut self, id: &str, from: usize, now: Instant) -> BookCommand<M> {
|
||||||
let tracked = &self.markets[id];
|
let tracked = &self.markets[id];
|
||||||
let (size, load) = (tracked.market.tokens.len(), tracked.load);
|
let (size, load) = (tracked.market.symbols().len(), tracked.load);
|
||||||
let carrying = tracked.connections.clone();
|
let carrying = tracked.connections.clone();
|
||||||
let to = self.pick(size, load, &carrying);
|
let to = self.pick(size, load, &carrying);
|
||||||
self.connections[from].tokens -= size;
|
self.connections[from].symbols -= size;
|
||||||
self.connections[from].load -= load;
|
self.connections[from].load -= load;
|
||||||
self.connections[to].tokens += size;
|
self.connections[to].symbols += size;
|
||||||
self.connections[to].load += load;
|
self.connections[to].load += load;
|
||||||
let tracked = self.markets.get_mut(id).expect("tracked");
|
let tracked = self.markets.get_mut(id).expect("tracked");
|
||||||
for connection in &mut tracked.connections {
|
for connection in &mut tracked.connections {
|
||||||
@@ -623,7 +630,7 @@ impl Tracker {
|
|||||||
/// A connection closed with 1013: on its second close within
|
/// A connection closed with 1013: on its second close within
|
||||||
/// [`SPLIT_WITHIN`], every second of its markets, in load order, moves off
|
/// [`SPLIT_WITHIN`], every second of its markets, in load order, moves off
|
||||||
/// at once, beyond rebalancing's limits.
|
/// at once, beyond rebalancing's limits.
|
||||||
fn closed(&mut self, connection: usize, now: Instant) -> Vec<BookCommand> {
|
fn closed(&mut self, connection: usize, now: Instant) -> Vec<BookCommand<M>> {
|
||||||
let Some(closed) = self.connections.get_mut(connection) else {
|
let Some(closed) = self.connections.get_mut(connection) else {
|
||||||
return Vec::new();
|
return Vec::new();
|
||||||
};
|
};
|
||||||
@@ -654,7 +661,7 @@ impl Tracker {
|
|||||||
/// A market the book thread found quiet on `connection` again after a
|
/// A market the book thread found quiet on `connection` again after a
|
||||||
/// resubscribe: its subscription there moves, beyond rebalancing's
|
/// resubscribe: its subscription there moves, beyond rebalancing's
|
||||||
/// limits. None if it no longer is there.
|
/// limits. None if it no longer is there.
|
||||||
fn quiet(&mut self, id: &str, connection: usize, now: Instant) -> Option<BookCommand> {
|
fn quiet(&mut self, id: &str, connection: usize, now: Instant) -> Option<BookCommand<M>> {
|
||||||
let tracked = self.markets.get(id)?;
|
let tracked = self.markets.get(id)?;
|
||||||
// A market moved lately stays, as for rebalancing: its last move may
|
// A market moved lately stays, as for rebalancing: its last move may
|
||||||
// still be under way, and a second one would end it early.
|
// still be under way, and a second one would end it early.
|
||||||
@@ -724,7 +731,7 @@ impl Tracker {
|
|||||||
|
|
||||||
/// Removes the markets known at the scan's start and not seen in it:
|
/// Removes the markets known at the scan's start and not seen in it:
|
||||||
/// those added since were never at risk.
|
/// those added since were never at risk.
|
||||||
fn scan_done(&mut self) -> Vec<BookCommand> {
|
fn scan_done(&mut self) -> Vec<BookCommand<M>> {
|
||||||
let unseen = self.unseen.take().unwrap_or_default();
|
let unseen = self.unseen.take().unwrap_or_default();
|
||||||
unseen.iter().filter_map(|id| self.remove(id)).collect()
|
unseen.iter().filter_map(|id| self.remove(id)).collect()
|
||||||
}
|
}
|
||||||
@@ -762,12 +769,30 @@ impl Random {
|
|||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
fn market(id: usize) -> Market {
|
/// A market of two symbols.
|
||||||
Market {
|
#[derive(Debug, Clone)]
|
||||||
condition_id: format!("0x{id}"),
|
struct TestMarket {
|
||||||
tokens: vec![format!("{id}a"), format!("{id}b")],
|
id: String,
|
||||||
min_order_size: "5".to_string(),
|
symbols: Vec<String>,
|
||||||
neg_risk: false,
|
}
|
||||||
|
|
||||||
|
impl Market for TestMarket {
|
||||||
|
fn id(&self) -> &str {
|
||||||
|
&self.id
|
||||||
|
}
|
||||||
|
|
||||||
|
fn symbols(&self) -> &[String] {
|
||||||
|
&self.symbols
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type Tracker = super::Tracker<TestMarket>;
|
||||||
|
type BookCommand = super::BookCommand<TestMarket>;
|
||||||
|
|
||||||
|
fn market(id: usize) -> TestMarket {
|
||||||
|
TestMarket {
|
||||||
|
id: format!("0x{id}"),
|
||||||
|
symbols: vec![format!("{id}a"), format!("{id}b")],
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -776,8 +801,8 @@ mod tests {
|
|||||||
Tracker::new(2, 7)
|
Tracker::new(2, 7)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn tokens(tracker: &Tracker) -> Vec<usize> {
|
fn symbols(tracker: &Tracker) -> Vec<usize> {
|
||||||
tracker.connections.iter().map(|one| one.tokens).collect()
|
tracker.connections.iter().map(|one| one.symbols).collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
fn connections(tracker: &Tracker, id: usize) -> Vec<usize> {
|
fn connections(tracker: &Tracker, id: usize) -> Vec<usize> {
|
||||||
@@ -798,9 +823,7 @@ mod tests {
|
|||||||
commands
|
commands
|
||||||
.iter()
|
.iter()
|
||||||
.map(|command| match command {
|
.map(|command| match command {
|
||||||
BookCommand::Move { market, from, to } => {
|
BookCommand::Move { market, from, to } => (market.id.as_str(), *from, *to),
|
||||||
(market.condition_id.as_str(), *from, *to)
|
|
||||||
}
|
|
||||||
other => panic!("not a move: {other:?}"),
|
other => panic!("not a move: {other:?}"),
|
||||||
})
|
})
|
||||||
.collect()
|
.collect()
|
||||||
@@ -809,21 +832,21 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn the_startup_deal_is_even_and_gives_each_market_different_connections() {
|
fn the_startup_deal_is_even_and_gives_each_market_different_connections() {
|
||||||
let mut tracker = tracker();
|
let mut tracker = tracker();
|
||||||
// 10,000 markets, 20,000 tokens, twice: 9 connections of 4,500.
|
// 10,000 markets, 20,000 symbols, twice: 9 connections of 4,500.
|
||||||
let adds = tracker.deal((0..10_000).map(market).collect());
|
let adds = tracker.deal((0..10_000).map(market).collect());
|
||||||
assert_eq!(adds.len(), 10_000);
|
assert_eq!(adds.len(), 10_000);
|
||||||
assert_eq!(tracker.connections.len(), 9);
|
assert_eq!(tracker.connections.len(), 9);
|
||||||
let tokens = tokens(&tracker);
|
let symbols = symbols(&tracker);
|
||||||
assert_eq!(tokens.iter().sum::<usize>(), 40_000);
|
assert_eq!(symbols.iter().sum::<usize>(), 40_000);
|
||||||
assert!(tokens.iter().all(|&held| held <= CONNECTION_TOKENS));
|
assert!(symbols.iter().all(|&held| held <= CONNECTION_SYMBOLS));
|
||||||
// Even, but for the markets whose second connection had to be the next.
|
// Even, but for the markets whose second connection had to be the next.
|
||||||
assert!(tokens.iter().max().unwrap() - tokens.iter().min().unwrap() <= 100);
|
assert!(symbols.iter().max().unwrap() - symbols.iter().min().unwrap() <= 100);
|
||||||
for command in &adds {
|
for command in &adds {
|
||||||
let BookCommand::Add(market, on) = command else {
|
let BookCommand::Add(market, on) = command else {
|
||||||
panic!("not an add: {command:?}");
|
panic!("not an add: {command:?}");
|
||||||
};
|
};
|
||||||
assert_eq!(on.len(), 2);
|
assert_eq!(on.len(), 2);
|
||||||
assert_ne!(on[0], on[1], "{}", market.condition_id);
|
assert_ne!(on[0], on[1], "{}", market.id);
|
||||||
}
|
}
|
||||||
// Markets created together (neighbours in the list) are spread: the
|
// Markets created together (neighbours in the list) are spread: the
|
||||||
// first 100 don't share one pair of connections.
|
// first 100 don't share one pair of connections.
|
||||||
@@ -836,7 +859,7 @@ mod tests {
|
|||||||
let mut tracker = tracker();
|
let mut tracker = tracker();
|
||||||
let adds = tracker.deal(vec![market(0), market(1), market(0)]);
|
let adds = tracker.deal(vec![market(0), market(1), market(0)]);
|
||||||
assert_eq!(adds.len(), 2);
|
assert_eq!(adds.len(), 2);
|
||||||
assert_eq!(tokens(&tracker).iter().sum::<usize>(), 8);
|
assert_eq!(symbols(&tracker).iter().sum::<usize>(), 8);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -845,22 +868,22 @@ mod tests {
|
|||||||
// Nothing open: its two connections open.
|
// Nothing open: its two connections open.
|
||||||
tracker.add(market(0)).unwrap();
|
tracker.add(market(0)).unwrap();
|
||||||
assert_eq!(connections(&tracker, 0).len(), 2);
|
assert_eq!(connections(&tracker, 0).len(), 2);
|
||||||
assert_eq!(tokens(&tracker), [2, 2]);
|
assert_eq!(symbols(&tracker), [2, 2]);
|
||||||
// Both fill together; then a third and a fourth open.
|
// Both fill together; then a third and a fourth open.
|
||||||
for id in 1..CONNECTION_TOKENS / 2 {
|
for id in 1..CONNECTION_SYMBOLS / 2 {
|
||||||
tracker.add(market(id)).unwrap();
|
tracker.add(market(id)).unwrap();
|
||||||
}
|
}
|
||||||
assert_eq!(tokens(&tracker), [CONNECTION_TOKENS, CONNECTION_TOKENS]);
|
assert_eq!(symbols(&tracker), [CONNECTION_SYMBOLS, CONNECTION_SYMBOLS]);
|
||||||
tracker.add(market(9_999)).unwrap();
|
tracker.add(market(9_999)).unwrap();
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
tokens(&tracker),
|
symbols(&tracker),
|
||||||
[CONNECTION_TOKENS, CONNECTION_TOKENS, 2, 2]
|
[CONNECTION_SYMBOLS, CONNECTION_SYMBOLS, 2, 2]
|
||||||
);
|
);
|
||||||
// Room on one connection only: the other subscription opens a new one.
|
// Room on one connection only: the other subscription opens a new one.
|
||||||
tracker.remove("0x0").unwrap();
|
tracker.remove("0x0").unwrap();
|
||||||
tracker.connections[1].tokens = CONNECTION_TOKENS;
|
tracker.connections[1].symbols = CONNECTION_SYMBOLS;
|
||||||
tracker.connections[2].tokens = CONNECTION_TOKENS;
|
tracker.connections[2].symbols = CONNECTION_SYMBOLS;
|
||||||
tracker.connections[3].tokens = CONNECTION_TOKENS;
|
tracker.connections[3].symbols = CONNECTION_SYMBOLS;
|
||||||
tracker.add(market(10_000)).unwrap();
|
tracker.add(market(10_000)).unwrap();
|
||||||
let mut on = connections(&tracker, 10_000);
|
let mut on = connections(&tracker, 10_000);
|
||||||
on.sort();
|
on.sort();
|
||||||
@@ -883,7 +906,7 @@ mod tests {
|
|||||||
let mut tracker = tracker();
|
let mut tracker = tracker();
|
||||||
assert!(tracker.add(market(1)).is_some());
|
assert!(tracker.add(market(1)).is_some());
|
||||||
assert!(tracker.add(market(1)).is_none());
|
assert!(tracker.add(market(1)).is_none());
|
||||||
assert_eq!(tokens(&tracker), [2, 2]);
|
assert_eq!(symbols(&tracker), [2, 2]);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -900,7 +923,7 @@ mod tests {
|
|||||||
// A market gone takes its load along.
|
// A market gone takes its load along.
|
||||||
tracker.remove("0x1");
|
tracker.remove("0x1");
|
||||||
assert_eq!(tracker.connections[1].load, 100);
|
assert_eq!(tracker.connections[1].load, 100);
|
||||||
assert_eq!(tokens(&tracker), [2, 2]);
|
assert_eq!(symbols(&tracker), [2, 2]);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -932,7 +955,7 @@ mod tests {
|
|||||||
);
|
);
|
||||||
assert_eq!(moved.len(), 1 + 5);
|
assert_eq!(moved.len(), 1 + 5);
|
||||||
assert!(tracker.connections.iter().all(|one| one.load <= budget));
|
assert!(tracker.connections.iter().all(|one| one.load <= budget));
|
||||||
assert_eq!(tokens(&tracker).iter().sum::<usize>(), 40);
|
assert_eq!(symbols(&tracker).iter().sum::<usize>(), 40);
|
||||||
// Under budget now: nothing more moves.
|
// Under budget now: nothing more moves.
|
||||||
assert!(tracker.rebalance(now).is_empty());
|
assert!(tracker.rebalance(now).is_empty());
|
||||||
}
|
}
|
||||||
@@ -968,13 +991,13 @@ mod tests {
|
|||||||
tracker.connections.push(ConnectionState::default());
|
tracker.connections.push(ConnectionState::default());
|
||||||
// One connection so far: put markets 2 and 3 on another.
|
// One connection so far: put markets 2 and 3 on another.
|
||||||
for id in [2, 3] {
|
for id in [2, 3] {
|
||||||
tracker.connections[0].tokens = CONNECTION_TOKENS;
|
tracker.connections[0].symbols = CONNECTION_SYMBOLS;
|
||||||
tracker.connections[2].tokens = CONNECTION_TOKENS;
|
tracker.connections[2].symbols = CONNECTION_SYMBOLS;
|
||||||
tracker.add(market(id));
|
tracker.add(market(id));
|
||||||
}
|
}
|
||||||
tracker.connections[0].tokens = 4;
|
tracker.connections[0].symbols = 4;
|
||||||
tracker.connections[2].tokens = 0;
|
tracker.connections[2].symbols = 0;
|
||||||
assert_eq!(tokens(&tracker), [4, 4, 0]);
|
assert_eq!(symbols(&tracker), [4, 4, 0]);
|
||||||
let budget = CONNECTION_BYTES_PER_SECOND;
|
let budget = CONNECTION_BYTES_PER_SECOND;
|
||||||
report(&mut tracker, &[(0, budget), (1, budget / 10)]);
|
report(&mut tracker, &[(0, budget), (1, budget / 10)]);
|
||||||
report(&mut tracker, &[(2, budget / 2), (3, budget)]);
|
report(&mut tracker, &[(2, budget / 2), (3, budget)]);
|
||||||
@@ -1014,7 +1037,7 @@ mod tests {
|
|||||||
assert!(moves(&second).iter().all(|&(_, from, _)| from == 0));
|
assert!(moves(&second).iter().all(|&(_, from, _)| from == 0));
|
||||||
// Ten minutes on, connection 0 holds one market: it is left alone.
|
// Ten minutes on, connection 0 holds one market: it is left alone.
|
||||||
assert!(tracker.rebalance(now + MOVED_STAYS).is_empty());
|
assert!(tracker.rebalance(now + MOVED_STAYS).is_empty());
|
||||||
assert_eq!(tracker.connections[0].tokens, 2);
|
assert_eq!(tracker.connections[0].symbols, 2);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Over budget on its own, a market would only carry its load from
|
/// Over budget on its own, a market would only carry its load from
|
||||||
@@ -1037,7 +1060,7 @@ mod tests {
|
|||||||
let now = Instant::now();
|
let now = Instant::now();
|
||||||
// Market 0 moves to a new connection, which then turns busy.
|
// Market 0 moves to a new connection, which then turns busy.
|
||||||
assert_eq!(moves(&tracker.rebalance(now)), [("0x0", 0, 1)]);
|
assert_eq!(moves(&tracker.rebalance(now)), [("0x0", 0, 1)]);
|
||||||
tracker.connections[0].tokens = CONNECTION_TOKENS;
|
tracker.connections[0].symbols = CONNECTION_SYMBOLS;
|
||||||
tracker.add(market(2));
|
tracker.add(market(2));
|
||||||
assert_eq!(connections(&tracker, 2), [1]);
|
assert_eq!(connections(&tracker, 2), [1]);
|
||||||
report(&mut tracker, &[(0, budget * 7 / 10), (2, budget * 6 / 10)]);
|
report(&mut tracker, &[(0, budget * 7 / 10), (2, budget * 6 / 10)]);
|
||||||
@@ -1127,7 +1150,7 @@ mod tests {
|
|||||||
.iter()
|
.iter()
|
||||||
.all(|&(_, from, to)| from == 0 && to != 0)
|
.all(|&(_, from, to)| from == 0 && to != 0)
|
||||||
);
|
);
|
||||||
assert_eq!(tracker.connections[0].tokens, 10);
|
assert_eq!(tracker.connections[0].symbols, 10);
|
||||||
// The split connection is left alone for ten minutes.
|
// The split connection is left alone for ten minutes.
|
||||||
let later = now + Duration::from_secs(6 * 60);
|
let later = now + Duration::from_secs(6 * 60);
|
||||||
assert!(tracker.closed(0, later).is_empty());
|
assert!(tracker.closed(0, later).is_empty());
|
||||||
@@ -1174,12 +1197,12 @@ mod tests {
|
|||||||
|
|
||||||
assert!(matches!(
|
assert!(matches!(
|
||||||
removed.as_slice(),
|
removed.as_slice(),
|
||||||
[BookCommand::Remove(market)] if market.condition_id == "0x2"
|
[BookCommand::Remove(market)] if market.id == "0x2"
|
||||||
));
|
));
|
||||||
let mut tracked: Vec<_> = tracker.markets.keys().cloned().collect();
|
let mut tracked: Vec<_> = tracker.markets.keys().cloned().collect();
|
||||||
tracked.sort();
|
tracked.sort();
|
||||||
assert_eq!(tracked, ["0x0", "0x9"]);
|
assert_eq!(tracked, ["0x0", "0x9"]);
|
||||||
assert_eq!(tokens(&tracker), [4, 4]);
|
assert_eq!(symbols(&tracker), [4, 4]);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -0,0 +1,397 @@
|
|||||||
|
//! A connection: reads the exchange's websocket, stamps each frame's receive
|
||||||
|
//! time, decodes it and sends each event to the book thread of its market;
|
||||||
|
//! and sends the exchange what the book threads ask for (subscriptions,
|
||||||
|
//! resyncs). It reconnects when it drops, falls silent, loses a message, or is
|
||||||
|
//! recycled.
|
||||||
|
|
||||||
|
use std::collections::BTreeSet;
|
||||||
|
use std::io::ErrorKind;
|
||||||
|
use std::net::{TcpStream, ToSocketAddrs};
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
use std::thread;
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use crossbeam::channel::{Receiver, Sender};
|
||||||
|
use tungstenite::protocol::frame::coding::CloseCode;
|
||||||
|
use tungstenite::stream::MaybeTlsStream;
|
||||||
|
use tungstenite::{Message, WebSocket};
|
||||||
|
|
||||||
|
use super::{Arrival, Event, Exchange, Received, Session, Undecoded, book_thread};
|
||||||
|
|
||||||
|
/// For the TCP connect and the TLS + websocket handshake, and for every send.
|
||||||
|
const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
|
||||||
|
/// How often the loop wakes when no frame comes: tungstenite has no `split`,
|
||||||
|
/// so the reads and the writes below (keepalive, commands, close) share this
|
||||||
|
/// thread.
|
||||||
|
const READ_TIMEOUT: Duration = Duration::from_millis(200);
|
||||||
|
const PING_EVERY: Duration = Duration::from_secs(10);
|
||||||
|
const RECONNECT_AFTER: Duration = Duration::from_secs(1);
|
||||||
|
const CLOSE_TIMEOUT: Duration = Duration::from_secs(2);
|
||||||
|
/// Nothing received this long, not even a keepalive's answer or a ping: the
|
||||||
|
/// connection is silent, and reconnects.
|
||||||
|
pub const SILENT_AFTER: Duration = Duration::from_secs(30);
|
||||||
|
/// What a book thread asks a connection for.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub enum Command {
|
||||||
|
Subscribe(Vec<String>),
|
||||||
|
Unsubscribe(Vec<String>),
|
||||||
|
/// Unsubscribe and subscribe again: the server then sends a fresh book.
|
||||||
|
/// A resync asks for that book; a quiet subscription only for the new
|
||||||
|
/// subscription.
|
||||||
|
Resubscribe(String),
|
||||||
|
/// Close and reconnect, subscribing every symbol again.
|
||||||
|
Recycle,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a connection tells the orchestrator.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub enum Notice {
|
||||||
|
/// Connected, its symbols subscribed.
|
||||||
|
Up(usize),
|
||||||
|
Down(usize),
|
||||||
|
/// Closed by the server with 1013: it couldn't send to us fast enough.
|
||||||
|
TooSlow(usize),
|
||||||
|
/// Closed by us: nothing came for [`SILENT_AFTER`].
|
||||||
|
Silent(usize),
|
||||||
|
}
|
||||||
|
|
||||||
|
type Socket = WebSocket<MaybeTlsStream<TcpStream>>;
|
||||||
|
|
||||||
|
/// Runs until `stop` is set or a book thread is gone, reconnecting (and
|
||||||
|
/// subscribing its symbols again) whenever the connection drops, falls silent
|
||||||
|
/// for `silent_after`, loses a message or is recycled, and telling every book
|
||||||
|
/// thread and the orchestrator (`notices`) when it comes up and goes down.
|
||||||
|
/// `book_threads` holds every book thread's channel, by number.
|
||||||
|
pub fn run<E: Exchange>(
|
||||||
|
exchange: &E,
|
||||||
|
number: usize,
|
||||||
|
book_threads: &[Sender<Received<E::Event>>],
|
||||||
|
commands: &Receiver<Command>,
|
||||||
|
notices: &Sender<Notice>,
|
||||||
|
silent_after: Duration,
|
||||||
|
stop: &AtomicBool,
|
||||||
|
) {
|
||||||
|
let name = E::NAME;
|
||||||
|
let tell = |received: fn(usize) -> Received<E::Event>, notice: fn(usize) -> Notice| {
|
||||||
|
for book_thread in book_threads {
|
||||||
|
_ = book_thread.send(received(number));
|
||||||
|
}
|
||||||
|
_ = notices.send(notice(number));
|
||||||
|
};
|
||||||
|
let mut symbols = BTreeSet::new();
|
||||||
|
while !stop.load(Ordering::Relaxed) {
|
||||||
|
// Asked for while disconnected: the subscribe on connect covers them,
|
||||||
|
// and its books are fresh, so resyncs have nothing left to do.
|
||||||
|
let mut session = exchange.session();
|
||||||
|
for command in commands.try_iter() {
|
||||||
|
_ = apply(&mut session, &mut symbols, &command);
|
||||||
|
}
|
||||||
|
match connect(exchange, &mut session, &symbols) {
|
||||||
|
Ok(mut socket) => {
|
||||||
|
log!(
|
||||||
|
"{name}: connection {number}: connected, {} symbols",
|
||||||
|
symbols.len()
|
||||||
|
);
|
||||||
|
tell(Received::Up, Notice::Up);
|
||||||
|
let ended = read(
|
||||||
|
&mut socket,
|
||||||
|
&mut session,
|
||||||
|
number,
|
||||||
|
&mut symbols,
|
||||||
|
book_threads,
|
||||||
|
commands,
|
||||||
|
notices,
|
||||||
|
silent_after,
|
||||||
|
stop,
|
||||||
|
name,
|
||||||
|
);
|
||||||
|
if !matches!(ended, Ok(Ended::Stopped)) {
|
||||||
|
tell(Received::Down, Notice::Down);
|
||||||
|
}
|
||||||
|
match ended {
|
||||||
|
Ok(Ended::Stopped) => {
|
||||||
|
return close(&mut socket, &mut session, number, book_threads, name);
|
||||||
|
}
|
||||||
|
Ok(Ended::Recycled) => log!("{name}: connection {number}: recycled"),
|
||||||
|
Ok(Ended::BookThreadGone) => {
|
||||||
|
// Only a panic ends a book thread before us: its
|
||||||
|
// markets would go unserved, so the run stops.
|
||||||
|
log!("{name}: connection {number}: a book thread is gone, stopping");
|
||||||
|
return stop.store(true, Ordering::Relaxed);
|
||||||
|
}
|
||||||
|
Err(error) => {
|
||||||
|
log!("{name}: connection {number}: connection lost: {error}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(error) => log!("{name}: connection {number}: connect failed: {error}"),
|
||||||
|
}
|
||||||
|
thread::sleep(RECONNECT_AFTER);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
enum Ended {
|
||||||
|
Stopped,
|
||||||
|
BookThreadGone,
|
||||||
|
Recycled,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Updates the symbol set, and returns the frames that carry `command` out
|
||||||
|
/// on an open connection: none for a resync of a symbol not in the set.
|
||||||
|
#[must_use]
|
||||||
|
pub(crate) fn apply<S: Session>(
|
||||||
|
session: &mut S,
|
||||||
|
symbols: &mut BTreeSet<String>,
|
||||||
|
command: &Command,
|
||||||
|
) -> Vec<String> {
|
||||||
|
match command {
|
||||||
|
Command::Subscribe(added) => {
|
||||||
|
symbols.extend(added.iter().cloned());
|
||||||
|
session.subscribe(&added.iter().collect::<Vec<_>>())
|
||||||
|
}
|
||||||
|
Command::Unsubscribe(removed) => {
|
||||||
|
removed.iter().for_each(|symbol| _ = symbols.remove(symbol));
|
||||||
|
session.unsubscribe(&removed.iter().collect::<Vec<_>>())
|
||||||
|
}
|
||||||
|
// Subscribing a symbol already subscribed sends no fresh book:
|
||||||
|
// unsubscribing first makes the server send one.
|
||||||
|
Command::Resubscribe(symbol) if symbols.contains(symbol) => {
|
||||||
|
[session.unsubscribe(&[symbol]), session.subscribe(&[symbol])].concat()
|
||||||
|
}
|
||||||
|
Command::Resubscribe(_) => Vec::new(),
|
||||||
|
// The read loop acts on it; disconnected, the reconnect does.
|
||||||
|
Command::Recycle => Vec::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn connect<E: Exchange>(
|
||||||
|
exchange: &E,
|
||||||
|
session: &mut E::Session,
|
||||||
|
symbols: &BTreeSet<String>,
|
||||||
|
) -> anyhow::Result<Socket> {
|
||||||
|
let request = exchange.request()?;
|
||||||
|
let uri = request.uri().clone();
|
||||||
|
let host = uri
|
||||||
|
.host()
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("{uri} has no host"))?;
|
||||||
|
// `wss://` on TLS's port; the tests name theirs.
|
||||||
|
let port = uri.port_u16().unwrap_or(443);
|
||||||
|
// Timed, unlike `tungstenite::connect`, so a stalled connect or handshake
|
||||||
|
// can't keep SIGINT waiting. The name lookup itself has no timeout.
|
||||||
|
let address = (host, port)
|
||||||
|
.to_socket_addrs()?
|
||||||
|
.next()
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("{host} has no address"))?;
|
||||||
|
let stream = TcpStream::connect_timeout(&address, CONNECT_TIMEOUT)?;
|
||||||
|
stream.set_read_timeout(Some(CONNECT_TIMEOUT))?;
|
||||||
|
stream.set_write_timeout(Some(CONNECT_TIMEOUT))?;
|
||||||
|
let (mut socket, _response) = tungstenite::client_tls(request, stream).map_err(|error| {
|
||||||
|
// A refusal's status says why: 401 for a key the exchange refuses.
|
||||||
|
match error {
|
||||||
|
tungstenite::HandshakeError::Failure(tungstenite::Error::Http(response)) => {
|
||||||
|
anyhow::anyhow!("handshake refused: {}", response.status())
|
||||||
|
}
|
||||||
|
error => anyhow::anyhow!("handshake failed: {error}"),
|
||||||
|
}
|
||||||
|
})?;
|
||||||
|
let stream = match socket.get_ref() {
|
||||||
|
MaybeTlsStream::Plain(stream) => stream,
|
||||||
|
MaybeTlsStream::Rustls(stream) => stream.get_ref(),
|
||||||
|
_ => anyhow::bail!("unexpected TLS stream"),
|
||||||
|
};
|
||||||
|
stream.set_read_timeout(Some(READ_TIMEOUT))?;
|
||||||
|
for message in session.opening() {
|
||||||
|
socket.send(Message::text(message))?;
|
||||||
|
}
|
||||||
|
if !symbols.is_empty() {
|
||||||
|
for message in session.subscribe(&symbols.iter().collect::<Vec<_>>()) {
|
||||||
|
socket.send(Message::text(message))?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(socket)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What [`route`] made of a frame.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub(crate) enum Routed {
|
||||||
|
/// Its events, if any, went to their book threads.
|
||||||
|
Sent,
|
||||||
|
BookThreadGone,
|
||||||
|
/// The connection lost a message: it must reconnect.
|
||||||
|
Lost(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decodes a frame and sends each event to the book thread of its market.
|
||||||
|
pub(crate) fn route<S: Session>(
|
||||||
|
session: &mut S,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
text: &str,
|
||||||
|
number: usize,
|
||||||
|
book_threads: &[Sender<Received<S::Event>>],
|
||||||
|
name: &str,
|
||||||
|
) -> Routed
|
||||||
|
where
|
||||||
|
S::Event: Event,
|
||||||
|
{
|
||||||
|
let events = match session.decode(text) {
|
||||||
|
Ok(events) => events,
|
||||||
|
Err(Undecoded::Unreadable(error)) => {
|
||||||
|
log!("{name}: unreadable frame ({error}): {}", truncate(text));
|
||||||
|
return Routed::Sent;
|
||||||
|
}
|
||||||
|
Err(Undecoded::Lost(why)) => return Routed::Lost(why),
|
||||||
|
};
|
||||||
|
let bytes = (text.len() / events.len().max(1)) as u64;
|
||||||
|
for event in events {
|
||||||
|
// An event of no market (none we ask for) has no book thread.
|
||||||
|
let Some(thread) = event.market().map(book_thread) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let arrival = Arrival {
|
||||||
|
received_at,
|
||||||
|
connection: number,
|
||||||
|
bytes,
|
||||||
|
event,
|
||||||
|
};
|
||||||
|
if book_threads[thread].send(Received::Event(arrival)).is_err() {
|
||||||
|
return Routed::BookThreadGone;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Routed::Sent
|
||||||
|
}
|
||||||
|
|
||||||
|
fn truncate(text: &str) -> &str {
|
||||||
|
text.char_indices()
|
||||||
|
.nth(200)
|
||||||
|
.map_or(text, |(end, _)| &text[..end])
|
||||||
|
}
|
||||||
|
|
||||||
|
// The connection's whole context, each a different part of it.
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
fn read<S: Session>(
|
||||||
|
socket: &mut Socket,
|
||||||
|
session: &mut S,
|
||||||
|
number: usize,
|
||||||
|
symbols: &mut BTreeSet<String>,
|
||||||
|
book_threads: &[Sender<Received<S::Event>>],
|
||||||
|
commands: &Receiver<Command>,
|
||||||
|
notices: &Sender<Notice>,
|
||||||
|
silent_after: Duration,
|
||||||
|
stop: &AtomicBool,
|
||||||
|
name: &str,
|
||||||
|
) -> anyhow::Result<Ended>
|
||||||
|
where
|
||||||
|
S::Event: Event,
|
||||||
|
{
|
||||||
|
let mut last_ping = Instant::now();
|
||||||
|
let mut last_heard = Instant::now();
|
||||||
|
loop {
|
||||||
|
let message = socket.read();
|
||||||
|
// Stamp as soon as the frame is off the wire, before any processing.
|
||||||
|
let received_at = Utc::now();
|
||||||
|
if message.is_ok() {
|
||||||
|
last_heard = Instant::now();
|
||||||
|
}
|
||||||
|
match message {
|
||||||
|
Ok(Message::Text(text)) => {
|
||||||
|
match route(
|
||||||
|
session,
|
||||||
|
received_at,
|
||||||
|
text.as_str(),
|
||||||
|
number,
|
||||||
|
book_threads,
|
||||||
|
name,
|
||||||
|
) {
|
||||||
|
Routed::Sent => {}
|
||||||
|
Routed::BookThreadGone => return Ok(Ended::BookThreadGone),
|
||||||
|
Routed::Lost(why) => anyhow::bail!("lost a message: {why}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(Message::Close(frame)) => {
|
||||||
|
// 1013: the server couldn't send to us fast enough. The
|
||||||
|
// orchestrator splits a connection closed so twice.
|
||||||
|
if frame
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|frame| frame.code == CloseCode::Again)
|
||||||
|
{
|
||||||
|
_ = notices.send(Notice::TooSlow(number));
|
||||||
|
}
|
||||||
|
anyhow::bail!("closed by the server: {frame:?}")
|
||||||
|
}
|
||||||
|
Ok(_) => {}
|
||||||
|
Err(tungstenite::Error::Io(error)) if timed_out(&error) => {}
|
||||||
|
Err(error) => return Err(error.into()),
|
||||||
|
}
|
||||||
|
|
||||||
|
if stop.load(Ordering::Relaxed) {
|
||||||
|
return Ok(Ended::Stopped);
|
||||||
|
}
|
||||||
|
if last_heard.elapsed() >= silent_after {
|
||||||
|
_ = notices.send(Notice::Silent(number));
|
||||||
|
anyhow::bail!("silent for {} s", silent_after.as_secs_f64());
|
||||||
|
}
|
||||||
|
if last_ping.elapsed() >= PING_EVERY {
|
||||||
|
if let Some(keepalive) = session.keepalive() {
|
||||||
|
socket.send(Message::text(keepalive))?;
|
||||||
|
}
|
||||||
|
last_ping = Instant::now();
|
||||||
|
}
|
||||||
|
for command in commands.try_iter() {
|
||||||
|
if command == Command::Recycle {
|
||||||
|
// Not waited on: whatever still comes would arrive from a
|
||||||
|
// connection already down for the book threads. The commands
|
||||||
|
// behind it are applied before the reconnect.
|
||||||
|
_ = socket.close(None);
|
||||||
|
_ = socket.flush();
|
||||||
|
return Ok(Ended::Recycled);
|
||||||
|
}
|
||||||
|
for message in apply(session, symbols, &command) {
|
||||||
|
socket.send(Message::text(message))?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Closes the connection, forwarding what still arrives until the server
|
||||||
|
/// confirms or [`CLOSE_TIMEOUT`] passes.
|
||||||
|
fn close<S: Session>(
|
||||||
|
socket: &mut Socket,
|
||||||
|
session: &mut S,
|
||||||
|
number: usize,
|
||||||
|
book_threads: &[Sender<Received<S::Event>>],
|
||||||
|
name: &str,
|
||||||
|
) where
|
||||||
|
S::Event: Event,
|
||||||
|
{
|
||||||
|
if let Err(error) = socket.close(None) {
|
||||||
|
return log!("{name}: connection {number}: close failed: {error}");
|
||||||
|
}
|
||||||
|
let deadline = Instant::now() + CLOSE_TIMEOUT;
|
||||||
|
while Instant::now() < deadline {
|
||||||
|
let message = socket.read();
|
||||||
|
let received_at = Utc::now();
|
||||||
|
match message {
|
||||||
|
Ok(Message::Text(text)) => {
|
||||||
|
_ = route(
|
||||||
|
session,
|
||||||
|
received_at,
|
||||||
|
text.as_str(),
|
||||||
|
number,
|
||||||
|
book_threads,
|
||||||
|
name,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
Ok(_) => {}
|
||||||
|
Err(tungstenite::Error::Io(error)) if timed_out(&error) => {}
|
||||||
|
// ConnectionClosed: the close handshake is done.
|
||||||
|
Err(_) => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A read timeout shows up as `WouldBlock` on macOS and `TimedOut` elsewhere;
|
||||||
|
/// tungstenite keeps a partly read frame, so the next read resumes it.
|
||||||
|
fn timed_out(error: &std::io::Error) -> bool {
|
||||||
|
matches!(error.kind(), ErrorKind::WouldBlock | ErrorKind::TimedOut)
|
||||||
|
}
|
||||||
@@ -0,0 +1,823 @@
|
|||||||
|
//! Kalshi's rule for a book thread: first one wins by clock and rank
|
||||||
|
//! (roadmap/06). A market's changes carry its clock, the same on every
|
||||||
|
//! connection, and come in the same order on each; several can share a
|
||||||
|
//! clock, so a change is named by its clock and its rank among the changes of
|
||||||
|
//! that clock. The book stands at the position of the last change it holds;
|
||||||
|
//! each connection's stream at the position of the last change it delivered,
|
||||||
|
//! counted from the book it last delivered. A change passing the book's
|
||||||
|
//! position is the next one; any other is a duplicate. Nothing is held.
|
||||||
|
//! A connection that lost a message reconnects (its `seq`, in
|
||||||
|
//! `src/kalshi/mod.rs`); the book then comes fresh.
|
||||||
|
|
||||||
|
use std::collections::BTreeMap;
|
||||||
|
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use engineering_challenge::book::Book;
|
||||||
|
use engineering_challenge::cause::{self, Cause};
|
||||||
|
|
||||||
|
use super::Market;
|
||||||
|
use super::wire::{BookMessage, ChangeMessage, Event};
|
||||||
|
use crate::core::feed::{self, Effect, Minute, Rule, Shell, is_up};
|
||||||
|
use crate::core::socket::Command;
|
||||||
|
|
||||||
|
/// A clock, and a rank among the changes of that clock.
|
||||||
|
type Position = (u64, u32);
|
||||||
|
|
||||||
|
/// A book's rank: it holds every change of its clock.
|
||||||
|
const BOOK: u32 = u32::MAX;
|
||||||
|
|
||||||
|
/// A symbol's state for the rule.
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
pub struct Clocked {
|
||||||
|
/// Where the book stands; none before its first book.
|
||||||
|
at: Option<Position>,
|
||||||
|
/// By connection: where its stream stands, since the book it last
|
||||||
|
/// delivered. A connection with none has delivered no book since it came
|
||||||
|
/// up or subscribed: its changes have nowhere to go.
|
||||||
|
on: BTreeMap<usize, Position>,
|
||||||
|
/// The connection a fresh book was asked from: its answer replaces ours.
|
||||||
|
asked: Option<usize>,
|
||||||
|
}
|
||||||
|
|
||||||
|
type Symbol = feed::Symbol<Clocked>;
|
||||||
|
|
||||||
|
/// What the minute line counts, since the start.
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
struct Counts {
|
||||||
|
written: u64,
|
||||||
|
duplicates: u64,
|
||||||
|
/// Changes from a connection that had delivered no book of the symbol.
|
||||||
|
unplaced: u64,
|
||||||
|
/// Fresh books after a symbol's first.
|
||||||
|
books: u64,
|
||||||
|
/// Of them, books at or past ours that differed: written as a Snapshot.
|
||||||
|
losses: u64,
|
||||||
|
/// Changes from a connection past the book, after the book went back to
|
||||||
|
/// an answer: dropped until the book reaches it.
|
||||||
|
ahead: u64,
|
||||||
|
/// Changes that took a level below zero: our book was wrong.
|
||||||
|
below_zero: u64,
|
||||||
|
resyncs: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Kalshi's rule, for one book thread.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct Clock {
|
||||||
|
counts: Counts,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Clock {
|
||||||
|
fn on_book(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Clocked>,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
connection: usize,
|
||||||
|
message: BookMessage,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
let ticker = message.ticker;
|
||||||
|
let Some(symbol) = shell.symbols.get_mut(&ticker) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if !symbol.connections.contains(&connection) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let position = (message.clock, BOOK);
|
||||||
|
symbol.rule.on.insert(connection, position);
|
||||||
|
let book = Book {
|
||||||
|
bids: message.bids.into_iter().collect(),
|
||||||
|
asks: message.asks.into_iter().collect(),
|
||||||
|
};
|
||||||
|
let asked = symbol.rule.asked == Some(connection);
|
||||||
|
let cause = match symbol.rule.at {
|
||||||
|
None => None,
|
||||||
|
Some(at) => {
|
||||||
|
self.counts.books += 1;
|
||||||
|
// Behind ours, unasked: it only says where its connection is.
|
||||||
|
if position < at && !asked {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if book == symbol.book {
|
||||||
|
symbol.rule.at = Some(at.max(position));
|
||||||
|
if asked {
|
||||||
|
symbol.rule.asked = None;
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
Some(at)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// The first book, an answer, or a book at or past ours that differs:
|
||||||
|
// ours is replaced. Which connection missed what can't be told: the
|
||||||
|
// other may only be behind.
|
||||||
|
symbol.book = book;
|
||||||
|
symbol.exchange_timestamp = message.sent_at;
|
||||||
|
symbol.rule.at = Some(position);
|
||||||
|
// A first book answers whichever connection it was asked from.
|
||||||
|
if asked || cause.is_none() {
|
||||||
|
symbol.rule.asked = None;
|
||||||
|
}
|
||||||
|
effects.push(Effect::Write(symbol.snapshot(&ticker, received_at)));
|
||||||
|
if let Some(ours) = cause {
|
||||||
|
self.counts.losses += 1;
|
||||||
|
let detail = format!(" (its clock {}, ours {})", message.clock, ours.0);
|
||||||
|
log!(
|
||||||
|
"kalshi: {}",
|
||||||
|
cause::line(&ticker, received_at, Cause::Loss, &detail)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_change(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Clocked>,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
connection: usize,
|
||||||
|
change: ChangeMessage,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
let ticker = change.ticker;
|
||||||
|
shell.count_received(&ticker, connection);
|
||||||
|
let Some(symbol) = shell.symbols.get_mut(&ticker) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let Some(on) = symbol.rule.on.get_mut(&connection) else {
|
||||||
|
self.counts.unplaced += 1;
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let before = *on;
|
||||||
|
*on = match before {
|
||||||
|
(clock, rank) if clock == change.clock => (clock, rank.saturating_add(1)),
|
||||||
|
_ => (change.clock, 1),
|
||||||
|
};
|
||||||
|
let position = *on;
|
||||||
|
match symbol.rule.at {
|
||||||
|
Some(at) if position <= at => {
|
||||||
|
self.counts.duplicates += 1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Its stream was already past the book, which went back to an
|
||||||
|
// answer: the changes between are still to come from the
|
||||||
|
// connection that answered, and this one would skip them.
|
||||||
|
Some(at) if before > at => {
|
||||||
|
self.counts.ahead += 1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
symbol.rule.at = Some(position);
|
||||||
|
let quantity = symbol.book.add(change.side, change.price, change.delta);
|
||||||
|
let quantity = if quantity < 0 {
|
||||||
|
// A level can't hold less than nothing: ours was wrong.
|
||||||
|
self.counts.below_zero += 1;
|
||||||
|
log!(
|
||||||
|
"kalshi: {ticker}: a change takes a level below zero, resync on connection \
|
||||||
|
{connection}"
|
||||||
|
);
|
||||||
|
if symbol.rule.asked.is_none() {
|
||||||
|
Self::ask_fresh_book(&ticker, symbol, connection, effects);
|
||||||
|
self.counts.resyncs += 1;
|
||||||
|
}
|
||||||
|
0
|
||||||
|
} else {
|
||||||
|
quantity
|
||||||
|
};
|
||||||
|
symbol.exchange_timestamp = change.at;
|
||||||
|
self.counts.written += 1;
|
||||||
|
effects.push(Effect::Write(symbol.delta(
|
||||||
|
&ticker,
|
||||||
|
change.side,
|
||||||
|
change.price,
|
||||||
|
quantity,
|
||||||
|
received_at,
|
||||||
|
change.at,
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Asks `connection` for a fresh book: unsubscribe and subscribe again.
|
||||||
|
fn ask_fresh_book(
|
||||||
|
ticker: &str,
|
||||||
|
symbol: &mut Symbol,
|
||||||
|
connection: usize,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
symbol.rule.asked = Some(connection);
|
||||||
|
effects.push(Effect::Send(
|
||||||
|
connection,
|
||||||
|
Command::Resubscribe(ticker.to_string()),
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Rule for Clock {
|
||||||
|
type Market = Market;
|
||||||
|
type Event = Event;
|
||||||
|
type Symbol = Clocked;
|
||||||
|
|
||||||
|
fn symbol(&self, _market: &Market) -> Clocked {
|
||||||
|
Clocked::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_event(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Clocked>,
|
||||||
|
connection: usize,
|
||||||
|
received_at: DateTime<Utc>,
|
||||||
|
event: Event,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
match event {
|
||||||
|
Event::Book(book) => self.on_book(shell, received_at, connection, book, effects),
|
||||||
|
Event::Change(change) => {
|
||||||
|
self.on_change(shell, received_at, connection, change, effects)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Its stream there counts no more; a fresh book asked from it is asked
|
||||||
|
/// from another.
|
||||||
|
fn on_leave(
|
||||||
|
&mut self,
|
||||||
|
up: &[bool],
|
||||||
|
ticker: &str,
|
||||||
|
symbol: &mut Symbol,
|
||||||
|
connection: usize,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
symbol.rule.on.remove(&connection);
|
||||||
|
if symbol.rule.asked == Some(connection)
|
||||||
|
&& let Some(other) = symbol.other_up(up, connection)
|
||||||
|
{
|
||||||
|
Self::ask_fresh_book(ticker, symbol, other, effects);
|
||||||
|
self.counts.resyncs += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Its streams count no more: back up, it delivers fresh books, and one
|
||||||
|
/// asked from it is its answer.
|
||||||
|
fn on_down(
|
||||||
|
&mut self,
|
||||||
|
shell: &mut Shell<Clocked>,
|
||||||
|
connection: usize,
|
||||||
|
_effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
for symbol in shell.symbols.values_mut() {
|
||||||
|
symbol.rule.on.remove(&connection);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_timer(
|
||||||
|
&mut self,
|
||||||
|
_shell: &mut Shell<Clocked>,
|
||||||
|
_now: DateTime<Utc>,
|
||||||
|
_effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
}
|
||||||
|
|
||||||
|
fn wakes_soon(&self) -> bool {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
fn on_removed(&mut self, _ticker: &str) {}
|
||||||
|
|
||||||
|
/// While no connection delivers it, the book may be missing changes: no
|
||||||
|
/// Snapshot of it until one does.
|
||||||
|
fn has_book(symbol: &Clocked) -> bool {
|
||||||
|
symbol.at.is_some() && !symbol.on.is_empty()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn may_be_quiet(symbol: &Clocked) -> bool {
|
||||||
|
symbol.at.is_some() && symbol.asked.is_none()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Asks again for a fresh book still not come, and for the book of a
|
||||||
|
/// symbol no connection delivers: its first, or one a reconnect didn't
|
||||||
|
/// bring.
|
||||||
|
fn on_minute(
|
||||||
|
&mut self,
|
||||||
|
up: &[bool],
|
||||||
|
ticker: &str,
|
||||||
|
symbol: &mut Symbol,
|
||||||
|
_now: DateTime<Utc>,
|
||||||
|
effects: &mut Vec<Effect>,
|
||||||
|
) {
|
||||||
|
let again = symbol.rule.asked.or_else(|| {
|
||||||
|
let up = |&connection: &usize| is_up(up, connection);
|
||||||
|
symbol
|
||||||
|
.rule
|
||||||
|
.on
|
||||||
|
.is_empty()
|
||||||
|
.then(|| symbol.connections.iter().copied().find(up))
|
||||||
|
.flatten()
|
||||||
|
});
|
||||||
|
if let Some(connection) = again {
|
||||||
|
Self::ask_fresh_book(ticker, symbol, connection, effects);
|
||||||
|
self.counts.resyncs += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn log_minute(&mut self, minute: &Minute) {
|
||||||
|
let counts = &self.counts;
|
||||||
|
log!(
|
||||||
|
"kalshi: book thread {}: {} symbols; so far {} changes written, {} duplicates, \
|
||||||
|
{} before their connection's book, {} ahead of the book, {} fresh books \
|
||||||
|
({} losses), {} below zero, \
|
||||||
|
{} resyncs, {} quiet subscriptions resubscribed, {} markets reported quiet again; \
|
||||||
|
this minute {} changes received, at most {} events waited",
|
||||||
|
minute.number,
|
||||||
|
minute.symbols,
|
||||||
|
counts.written,
|
||||||
|
counts.duplicates,
|
||||||
|
counts.unplaced,
|
||||||
|
counts.ahead,
|
||||||
|
counts.books,
|
||||||
|
counts.losses,
|
||||||
|
counts.below_zero,
|
||||||
|
counts.resyncs,
|
||||||
|
minute.quiet,
|
||||||
|
minute.quiet_reported,
|
||||||
|
minute.received,
|
||||||
|
minute.peak_waiting
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
use chrono::TimeDelta;
|
||||||
|
use engineering_challenge::util::dtos::challenge_orderbook::PriceLevelUpdateDomain;
|
||||||
|
use serde::Deserialize;
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
use crate::core::{Arrival, Received, feed::next_minute};
|
||||||
|
use crate::kalshi::wire;
|
||||||
|
use engineering_challenge::Side;
|
||||||
|
use engineering_challenge::book::Output;
|
||||||
|
|
||||||
|
type Feed = feed::Feed<Clock>;
|
||||||
|
|
||||||
|
fn at(millis: i64) -> DateTime<Utc> {
|
||||||
|
DateTime::from_timestamp_millis(1_794_422_000_000 + millis).unwrap()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn feed_of(tickers: &[&str], connections: &[usize]) -> Feed {
|
||||||
|
let mut feed = Feed::new("kalshi", 0, Clock::default());
|
||||||
|
for ticker in tickers {
|
||||||
|
feed.add_market(
|
||||||
|
&Market {
|
||||||
|
ticker: ticker.to_string(),
|
||||||
|
},
|
||||||
|
connections,
|
||||||
|
at(0),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
feed
|
||||||
|
}
|
||||||
|
|
||||||
|
fn book(ticker: &str, clock: u64, bids: &[(i64, i64)], asks: &[(i64, i64)]) -> Event {
|
||||||
|
Event::Book(BookMessage {
|
||||||
|
ticker: ticker.into(),
|
||||||
|
clock,
|
||||||
|
bids: bids.to_vec(),
|
||||||
|
asks: asks.to_vec(),
|
||||||
|
sent_at: at(0),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn change(ticker: &str, clock: u64, side: Side, price: i64, delta: i64) -> Event {
|
||||||
|
Event::Change(ChangeMessage {
|
||||||
|
ticker: ticker.into(),
|
||||||
|
clock,
|
||||||
|
side,
|
||||||
|
price,
|
||||||
|
delta,
|
||||||
|
at: at(0),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Feed {
|
||||||
|
fn from(&mut self, connection: usize, millis: i64, event: Event) -> Vec<Effect> {
|
||||||
|
self.on_event(Arrival {
|
||||||
|
received_at: at(millis),
|
||||||
|
connection,
|
||||||
|
bytes: 0,
|
||||||
|
event,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn deltas(effects: &[Effect]) -> Vec<&PriceLevelUpdateDomain> {
|
||||||
|
effects
|
||||||
|
.iter()
|
||||||
|
.filter_map(|effect| match effect {
|
||||||
|
Effect::Write(Output::Delta(delta)) => Some(delta),
|
||||||
|
_ => None,
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn snapshots(effects: &[Effect]) -> usize {
|
||||||
|
effects
|
||||||
|
.iter()
|
||||||
|
.filter(|effect| matches!(effect, Effect::Write(Output::Snapshot(_))))
|
||||||
|
.count()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resyncs(effects: &[Effect]) -> Vec<(usize, &str)> {
|
||||||
|
effects
|
||||||
|
.iter()
|
||||||
|
.filter_map(|effect| match effect {
|
||||||
|
Effect::Send(connection, Command::Resubscribe(ticker)) => {
|
||||||
|
Some((*connection, ticker.as_str()))
|
||||||
|
}
|
||||||
|
_ => None,
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A market with the same book from both connections.
|
||||||
|
fn both_on(ticker: &str) -> Feed {
|
||||||
|
let mut feed = feed_of(&[ticker], &[0, 1]);
|
||||||
|
let first = feed.from(0, 1, book(ticker, 10, &[(400_000, 5_000_000)], &[]));
|
||||||
|
assert_eq!(snapshots(&first), 1);
|
||||||
|
let second = feed.from(1, 2, book(ticker, 10, &[(400_000, 5_000_000)], &[]));
|
||||||
|
assert!(second.is_empty(), "{second:?}");
|
||||||
|
feed
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_change_is_written_once_from_whichever_delivers_it_first() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
let first = feed.from(1, 5, change("M", 11, Side::Buy, 400_000, 2_000_000));
|
||||||
|
let written = deltas(&first);
|
||||||
|
assert_eq!(written.len(), 1);
|
||||||
|
// The absolute quantity, at the first copy's receive time.
|
||||||
|
assert_eq!(written[0].quantity, 7_000_000);
|
||||||
|
assert_eq!(written[0].backtest_timestamp, at(5));
|
||||||
|
assert!(
|
||||||
|
deltas(&feed.from(0, 6, change("M", 11, Side::Buy, 400_000, 2_000_000))).is_empty()
|
||||||
|
);
|
||||||
|
assert_eq!(feed.rule.counts.duplicates, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_change_of_one_clock_is_applied_once() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
// Two identical changes share a clock: both are real.
|
||||||
|
let twice = [
|
||||||
|
change("M", 11, Side::Sell, 600_000, 1_000_000),
|
||||||
|
change("M", 11, Side::Sell, 600_000, 1_000_000),
|
||||||
|
];
|
||||||
|
let mut written = Vec::new();
|
||||||
|
for event in twice {
|
||||||
|
written.extend(feed.from(0, 5, event));
|
||||||
|
}
|
||||||
|
let quantities: Vec<i64> = deltas(&written)
|
||||||
|
.iter()
|
||||||
|
.map(|delta| delta.quantity)
|
||||||
|
.collect();
|
||||||
|
assert_eq!(quantities, [1_000_000, 2_000_000]);
|
||||||
|
for event in [
|
||||||
|
change("M", 11, Side::Sell, 600_000, 1_000_000),
|
||||||
|
change("M", 11, Side::Sell, 600_000, 1_000_000),
|
||||||
|
] {
|
||||||
|
assert!(deltas(&feed.from(1, 6, event)).is_empty());
|
||||||
|
}
|
||||||
|
// The second connection's third change of that clock is new.
|
||||||
|
let third = feed.from(1, 7, change("M", 11, Side::Sell, 600_000, 1_000_000));
|
||||||
|
assert_eq!(deltas(&third)[0].quantity, 3_000_000);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_connection_counts_for_a_symbol_only_from_its_book() {
|
||||||
|
let mut feed = feed_of(&["M"], &[0, 1]);
|
||||||
|
feed.from(0, 1, book("M", 10, &[], &[]));
|
||||||
|
// Connection 1 hasn't delivered its book: its change has no place.
|
||||||
|
assert!(feed.from(1, 2, change("M", 11, Side::Buy, 1, 1)).is_empty());
|
||||||
|
assert_eq!(feed.rule.counts.unplaced, 1);
|
||||||
|
// Down and up again, connection 0 too waits for its book.
|
||||||
|
feed.on_received(Received::Down(0));
|
||||||
|
feed.on_received(Received::Up(0));
|
||||||
|
assert!(feed.from(0, 3, change("M", 12, Side::Buy, 1, 1)).is_empty());
|
||||||
|
assert_eq!(feed.rule.counts.unplaced, 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_fresh_book_replaces_ours_only_at_or_past_it_and_different() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
feed.from(0, 3, change("M", 11, Side::Buy, 400_000, 1_000_000));
|
||||||
|
// Older than ours: connection 1 is only behind.
|
||||||
|
assert!(
|
||||||
|
feed.from(1, 4, book("M", 10, &[(400_000, 5_000_000)], &[]))
|
||||||
|
.is_empty()
|
||||||
|
);
|
||||||
|
// At our clock and the same: nothing.
|
||||||
|
assert!(
|
||||||
|
feed.from(1, 5, book("M", 11, &[(400_000, 6_000_000)], &[]))
|
||||||
|
.is_empty()
|
||||||
|
);
|
||||||
|
// Past ours and the same: nothing either.
|
||||||
|
assert!(
|
||||||
|
feed.from(1, 6, book("M", 15, &[(400_000, 6_000_000)], &[]))
|
||||||
|
.is_empty()
|
||||||
|
);
|
||||||
|
assert_eq!(feed.rule.counts.losses, 0);
|
||||||
|
// Past ours and different: it replaces ours, as a Snapshot.
|
||||||
|
let different = feed.from(1, 7, book("M", 20, &[(400_000, 1_000_000)], &[]));
|
||||||
|
assert_eq!(snapshots(&different), 1);
|
||||||
|
assert_eq!(feed.rule.counts.losses, 1);
|
||||||
|
assert_eq!(feed.shell.symbols["M"].book.bids[&400_000], 1_000_000);
|
||||||
|
// Its changes up to that book are duplicates now.
|
||||||
|
assert!(deltas(&feed.from(0, 8, change("M", 16, Side::Buy, 400_000, 1))).is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_level_below_zero_resyncs_and_the_answer_replaces_ours() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
let effects = feed.from(0, 3, change("M", 11, Side::Buy, 400_000, -6_000_000));
|
||||||
|
assert_eq!(deltas(&effects)[0].quantity, 0);
|
||||||
|
assert_eq!(resyncs(&effects), [(0, "M")]);
|
||||||
|
assert_eq!(feed.rule.counts.below_zero, 1);
|
||||||
|
assert!(!Clock::may_be_quiet(&feed.shell.symbols["M"].rule));
|
||||||
|
// The answer replaces ours even behind it, and differing is a loss.
|
||||||
|
let answer = feed.from(0, 4, book("M", 9, &[(400_000, 9_000_000)], &[]));
|
||||||
|
assert_eq!(snapshots(&answer), 1);
|
||||||
|
assert_eq!(feed.shell.symbols["M"].book.bids[&400_000], 9_000_000);
|
||||||
|
assert_eq!(feed.shell.symbols["M"].rule.asked, None);
|
||||||
|
// The stream that answered goes on from its book.
|
||||||
|
let after = feed.from(0, 5, change("M", 10, Side::Buy, 400_000, 1_000_000));
|
||||||
|
assert_eq!(deltas(&after)[0].quantity, 10_000_000);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The answer to a resync can be behind what the other connection has
|
||||||
|
/// delivered: the book goes back to it, and only a stream that continues
|
||||||
|
/// from there is taken until the other fits again.
|
||||||
|
#[test]
|
||||||
|
fn after_an_answer_behind_ours_a_connection_ahead_waits_for_the_book() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
feed.from(0, 3, change("M", 11, Side::Buy, 400_000, -6_000_000));
|
||||||
|
feed.from(1, 4, change("M", 11, Side::Buy, 400_000, -6_000_000));
|
||||||
|
feed.from(1, 5, change("M", 12, Side::Buy, 400_000, 1_000_000));
|
||||||
|
// Connection 0's answer holds clock 11, not yet 12.
|
||||||
|
let answer = feed.from(0, 6, book("M", 11, &[(400_000, 2_000_000)], &[]));
|
||||||
|
assert_eq!(snapshots(&answer), 1);
|
||||||
|
// Connection 1 is past it: its next change would skip clock 12.
|
||||||
|
let ahead = feed.from(1, 7, change("M", 13, Side::Buy, 400_000, 1_000_000));
|
||||||
|
assert!(deltas(&ahead).is_empty(), "{ahead:?}");
|
||||||
|
let twelve = feed.from(0, 8, change("M", 12, Side::Buy, 400_000, 1_000_000));
|
||||||
|
assert_eq!(deltas(&twelve)[0].quantity, 3_000_000);
|
||||||
|
let thirteen = feed.from(0, 9, change("M", 13, Side::Buy, 400_000, 1_000_000));
|
||||||
|
assert_eq!(deltas(&thirteen)[0].quantity, 4_000_000);
|
||||||
|
// The book has caught up with connection 1: it counts again.
|
||||||
|
let fourteen = feed.from(1, 10, change("M", 14, Side::Buy, 400_000, 1_000_000));
|
||||||
|
assert_eq!(deltas(&fourteen)[0].quantity, 5_000_000);
|
||||||
|
assert_eq!(feed.rule.counts.ahead, 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_first_book_asked_from_one_connection_can_come_from_the_other() {
|
||||||
|
let mut feed = feed_of(&["N"], &[0, 1]);
|
||||||
|
feed.on_received(Received::Up(0));
|
||||||
|
feed.from(0, 5, change("N", 1, Side::Buy, 1, 1));
|
||||||
|
assert_eq!(resyncs(&feed.tick(next_minute(at(10)))), [(0, "N")]);
|
||||||
|
let first = feed.from(1, 61_000, book("N", 20, &[(400_000, 5_000_000)], &[]));
|
||||||
|
assert_eq!(snapshots(&first), 1);
|
||||||
|
// Connection 0's answer, older and different, is only where it stands.
|
||||||
|
let answer = feed.from(0, 61_001, book("N", 19, &[(400_000, 4_000_000)], &[]));
|
||||||
|
assert!(answer.is_empty(), "{answer:?}");
|
||||||
|
assert_eq!(feed.rule.counts.losses, 0);
|
||||||
|
assert_eq!(feed.shell.symbols["N"].book.bids[&400_000], 5_000_000);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_symbol_no_connection_delivers_is_asked_for_at_the_minute() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
feed.on_received(Received::Down(0));
|
||||||
|
feed.on_received(Received::Down(1));
|
||||||
|
feed.on_received(Received::Up(0));
|
||||||
|
feed.on_received(Received::Up(1));
|
||||||
|
// Its books didn't come with the reconnects.
|
||||||
|
assert_eq!(resyncs(&feed.tick(next_minute(at(10)))), [(0, "M")]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_resync_asked_from_a_connection_that_leaves_is_asked_from_another() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
feed.from(0, 3, change("M", 11, Side::Buy, 400_000, -6_000_000));
|
||||||
|
let market = Market { ticker: "M".into() };
|
||||||
|
let moved = feed.move_market(&market, 0, 2, at(4));
|
||||||
|
assert!(resyncs(&moved).is_empty());
|
||||||
|
// Connection 2's book ends the move: connection 0 leaves.
|
||||||
|
let ended = feed.from(2, 5, book("M", 12, &[], &[]));
|
||||||
|
assert!(ended.iter().any(|effect| matches!(
|
||||||
|
effect,
|
||||||
|
Effect::Send(0, Command::Unsubscribe(tokens)) if tokens == &["M".to_string()]
|
||||||
|
)));
|
||||||
|
// It came from connection 2, not the one asked: not the answer.
|
||||||
|
assert_eq!(resyncs(&ended), [(1, "M")]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_fresh_book_still_not_come_is_asked_again_at_the_minute() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
feed.from(0, 3, change("M", 11, Side::Buy, 400_000, -6_000_000));
|
||||||
|
let minute = feed.tick(next_minute(at(10)));
|
||||||
|
assert_eq!(resyncs(&minute), [(0, "M")]);
|
||||||
|
// A symbol with no book at all is asked for one too.
|
||||||
|
let mut none = feed_of(&["N"], &[0]);
|
||||||
|
none.on_received(Received::Up(0));
|
||||||
|
// An event starts the book thread's minutes.
|
||||||
|
none.from(0, 5, change("N", 1, Side::Buy, 1, 1));
|
||||||
|
assert_eq!(resyncs(&none.tick(next_minute(at(10)))), [(0, "N")]);
|
||||||
|
// One in order isn't.
|
||||||
|
let mut fine = both_on("F");
|
||||||
|
assert!(resyncs(&fine.tick(next_minute(at(10)))).is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn no_minute_snapshot_while_no_connection_delivers_the_symbol() {
|
||||||
|
let mut feed = both_on("M");
|
||||||
|
let minute = next_minute(at(10));
|
||||||
|
assert_eq!(snapshots(&feed.tick(minute)), 1);
|
||||||
|
feed.on_received(Received::Down(0));
|
||||||
|
feed.on_received(Received::Down(1));
|
||||||
|
assert_eq!(snapshots(&feed.tick(minute + TimeDelta::minutes(1))), 0);
|
||||||
|
// Back, its book compared: the same, so no Snapshot until the minute.
|
||||||
|
assert!(
|
||||||
|
feed.from(1, 70_000, book("M", 30, &[(400_000, 5_000_000)], &[]))
|
||||||
|
.is_empty()
|
||||||
|
);
|
||||||
|
assert_eq!(snapshots(&feed.tick(minute + TimeDelta::minutes(2))), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The demo's recording on two connections (tests/fixtures/README.md).
|
||||||
|
fn recording() -> Vec<(DateTime<Utc>, usize, Event)> {
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct Line {
|
||||||
|
received_at_us: i64,
|
||||||
|
connection: usize,
|
||||||
|
text: String,
|
||||||
|
}
|
||||||
|
let path = concat!(
|
||||||
|
env!("CARGO_MANIFEST_DIR"),
|
||||||
|
"/tests/fixtures/kalshi-frames-two.jsonl"
|
||||||
|
);
|
||||||
|
std::fs::read_to_string(path)
|
||||||
|
.unwrap()
|
||||||
|
.lines()
|
||||||
|
.filter_map(|line| {
|
||||||
|
let line: Line = serde_json::from_str(line).unwrap();
|
||||||
|
let frame: wire::Frame = serde_json::from_str(&line.text).unwrap();
|
||||||
|
let event = wire::event(&frame).unwrap()?;
|
||||||
|
let received_at = DateTime::from_timestamp_micros(line.received_at_us).unwrap();
|
||||||
|
Some((received_at, line.connection, event))
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn tickers(recording: &[(DateTime<Utc>, usize, Event)]) -> Vec<String> {
|
||||||
|
let mut tickers: Vec<String> = recording
|
||||||
|
.iter()
|
||||||
|
.map(|(_, _, event)| crate::core::Event::market(event).unwrap().to_string())
|
||||||
|
.collect();
|
||||||
|
tickers.sort();
|
||||||
|
tickers.dedup();
|
||||||
|
tickers
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Each symbol's Deltas, as (side, price, quantity, receive time).
|
||||||
|
type Written = HashMap<String, Vec<(Side, i64, i64, DateTime<Utc>)>>;
|
||||||
|
|
||||||
|
/// Replays the connections in `on`.
|
||||||
|
fn replay(on: &[usize]) -> (Feed, Written) {
|
||||||
|
let recording = recording();
|
||||||
|
let tickers = tickers(&recording);
|
||||||
|
let tickers: Vec<&str> = tickers.iter().map(String::as_str).collect();
|
||||||
|
let mut feed = feed_of(&tickers, on);
|
||||||
|
let mut written: HashMap<_, Vec<_>> = HashMap::new();
|
||||||
|
for (received_at, connection, event) in recording {
|
||||||
|
if !on.contains(&connection) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let arrival = Arrival {
|
||||||
|
received_at,
|
||||||
|
connection,
|
||||||
|
bytes: 0,
|
||||||
|
event,
|
||||||
|
};
|
||||||
|
for delta in deltas(&feed.on_event(arrival)) {
|
||||||
|
written.entry(delta.symbol.clone()).or_default().push((
|
||||||
|
delta.side,
|
||||||
|
delta.price_level,
|
||||||
|
delta.quantity,
|
||||||
|
delta.backtest_timestamp,
|
||||||
|
));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(feed, written)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Connection 0 asked for every book again every 3 s: each, compared
|
||||||
|
/// with ours folded from the changes, was the same.
|
||||||
|
#[test]
|
||||||
|
fn the_recorded_changes_fold_into_the_exchanges_books() {
|
||||||
|
let (feed, written) = replay(&[0]);
|
||||||
|
let counts = &feed.rule.counts;
|
||||||
|
assert!(counts.books > 700, "{counts:?}");
|
||||||
|
assert_eq!(
|
||||||
|
(counts.losses, counts.below_zero, counts.duplicates),
|
||||||
|
(0, 0, 0)
|
||||||
|
);
|
||||||
|
assert!(written.values().map(Vec::len).sum::<usize>() > 1_000);
|
||||||
|
// A NO bid arrived as the ask it is: no book is crossed. Taken at
|
||||||
|
// one minus its price again, a third of them would be.
|
||||||
|
let mut two_sided = 0;
|
||||||
|
for (ticker, symbol) in &feed.shell.symbols {
|
||||||
|
let best_bid = symbol.book.bids.keys().next_back();
|
||||||
|
let best_ask = symbol.book.asks.keys().next();
|
||||||
|
if let (Some(bid), Some(ask)) = (best_bid, best_ask) {
|
||||||
|
assert!(bid < ask, "{ticker}: bid {bid}, ask {ask}");
|
||||||
|
two_sided += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert!(two_sided >= 10, "{two_sided}");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The book and writer threads as main runs them, fed the recording:
|
||||||
|
/// every Snapshot is the one before it with the Deltas between.
|
||||||
|
#[test]
|
||||||
|
fn the_output_reconciles() {
|
||||||
|
let recording = recording();
|
||||||
|
let tickers = tickers(&recording);
|
||||||
|
let tickers: Vec<&str> = tickers.iter().map(String::as_str).collect();
|
||||||
|
let feed = feed_of(&tickers, &[0, 1]);
|
||||||
|
let (events_tx, events_rx) = crossbeam::channel::unbounded();
|
||||||
|
let (outputs_tx, outputs_rx) = crossbeam::channel::unbounded();
|
||||||
|
let (connection_tx, _connection_rx) = crossbeam::channel::unbounded();
|
||||||
|
let (_commands_tx, commands_rx) = crossbeam::channel::unbounded();
|
||||||
|
let (reports_tx, _reports_rx) = crossbeam::channel::unbounded();
|
||||||
|
for (received_at, connection, event) in recording {
|
||||||
|
let arrival = Arrival {
|
||||||
|
received_at,
|
||||||
|
connection,
|
||||||
|
bytes: 0,
|
||||||
|
event,
|
||||||
|
};
|
||||||
|
events_tx.send(Received::Event(arrival)).unwrap();
|
||||||
|
}
|
||||||
|
drop(events_tx);
|
||||||
|
let book = std::thread::spawn(move || {
|
||||||
|
feed::serve(
|
||||||
|
feed,
|
||||||
|
vec![connection_tx.clone(), connection_tx],
|
||||||
|
events_rx,
|
||||||
|
commands_rx,
|
||||||
|
outputs_tx,
|
||||||
|
reports_tx,
|
||||||
|
)
|
||||||
|
});
|
||||||
|
let path =
|
||||||
|
std::env::temp_dir().join(format!("kalshi-reconcile-{}.sbe", std::process::id()));
|
||||||
|
let written = crate::write(path.to_str().unwrap(), outputs_rx);
|
||||||
|
book.join().unwrap();
|
||||||
|
written.unwrap();
|
||||||
|
let report = engineering_challenge::reconcile::reconcile(&path, None);
|
||||||
|
std::fs::remove_file(&path).unwrap();
|
||||||
|
|
||||||
|
let report = report.unwrap();
|
||||||
|
// The opening Snapshots, and one per symbol at the minute crossed.
|
||||||
|
assert_eq!(report.snapshots, tickers.len() * 2, "{report:?}");
|
||||||
|
assert!(report.deltas > 1_000, "{report:?}");
|
||||||
|
assert_eq!(report.orphan_deltas, 0);
|
||||||
|
assert_eq!(report.mismatches, []);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Connection 1 left five markets for 5 s: connection 0 carried them.
|
||||||
|
#[test]
|
||||||
|
fn two_recorded_connections_write_the_changes_of_one_each_at_its_first_copy() {
|
||||||
|
let (one, alone) = replay(&[0]);
|
||||||
|
let (both, together) = replay(&[0, 1]);
|
||||||
|
assert_eq!(both.rule.counts.losses, 0);
|
||||||
|
assert!(both.rule.counts.duplicates > 900, "{:?}", both.rule.counts);
|
||||||
|
assert_eq!(alone.keys().len(), together.keys().len());
|
||||||
|
let mut earlier = 0;
|
||||||
|
for (symbol, changes) in &alone {
|
||||||
|
let content = |changes: &[(Side, i64, i64, DateTime<Utc>)]| {
|
||||||
|
changes
|
||||||
|
.iter()
|
||||||
|
.map(|&(side, price, quantity, _)| (side, price, quantity))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
};
|
||||||
|
assert_eq!(content(changes), content(&together[symbol]), "{symbol}");
|
||||||
|
for (single, first) in changes.iter().zip(&together[symbol]) {
|
||||||
|
assert!(first.3 <= single.3);
|
||||||
|
earlier += usize::from(first.3 < single.3);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Connection 1 was first for most changes in the recording.
|
||||||
|
assert!(earlier > 500, "{earlier}");
|
||||||
|
assert_eq!(one.rule.counts.written, both.rule.counts.written);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,334 @@
|
|||||||
|
//! Kalshi's market listing: which markets are live, or about to be. Two fetcher threads page
|
||||||
|
//! it on the orchestrator's request and stream what they read; they hold no
|
||||||
|
//! state and no timer, as Gamma's (roadmap/06). Read without a key.
|
||||||
|
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use chrono::{DateTime, TimeDelta, Utc};
|
||||||
|
use crossbeam::channel::{Receiver, Sender};
|
||||||
|
use serde::Deserialize;
|
||||||
|
|
||||||
|
use super::Market;
|
||||||
|
use crate::core::{Finding, Request, retried};
|
||||||
|
|
||||||
|
/// The listing's largest page.
|
||||||
|
const PAGE_SIZE: &str = "1000";
|
||||||
|
/// For every request, so SIGINT never waits on the listing longer than this.
|
||||||
|
const TIMEOUT: Duration = Duration::from_secs(10);
|
||||||
|
/// A page's tries, and the wait between them.
|
||||||
|
const TRIES: u32 = 5;
|
||||||
|
const RETRY_AFTER: Duration = Duration::from_secs(5);
|
||||||
|
/// A market not yet open is tracked this long before its opening: its book is
|
||||||
|
/// there, empty, from its first order. Three hourly rescans, so every market
|
||||||
|
/// is reached by a scan before it opens even when one scan fails.
|
||||||
|
const OPENING_AHEAD: TimeDelta = TimeDelta::hours(3);
|
||||||
|
|
||||||
|
/// A fetcher thread: answers each request until the orchestrator is gone.
|
||||||
|
pub fn run(listing: &str, requests: Receiver<Request>, findings: Sender<Finding<Market>>) {
|
||||||
|
let agent = ureq::Agent::new_with_config(
|
||||||
|
ureq::Agent::config_builder()
|
||||||
|
.timeout_global(Some(TIMEOUT))
|
||||||
|
.build(),
|
||||||
|
);
|
||||||
|
let markets = format!("{listing}/markets");
|
||||||
|
for request in requests {
|
||||||
|
let sent = match request {
|
||||||
|
Request::Rescan => rescan(&agent, &markets, &findings),
|
||||||
|
Request::Recent { since } => recent(&agent, &markets, since, &findings),
|
||||||
|
};
|
||||||
|
if sent.is_err() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `Err` once the orchestrator is gone.
|
||||||
|
type Sent = Result<(), crossbeam::channel::SendError<Finding<Market>>>;
|
||||||
|
|
||||||
|
fn rescan(agent: &ureq::Agent, markets: &str, findings: &Sender<Finding<Market>>) -> Sent {
|
||||||
|
findings.send(Finding::ScanStarted)?;
|
||||||
|
// Unopened first: a market that opens while the scan runs is then on one
|
||||||
|
// list at least. Open first, it could be on neither, and be removed.
|
||||||
|
for status in ["unopened", "open"] {
|
||||||
|
let query = [("status", status.to_string()), mve_excluded()];
|
||||||
|
let mut cursor = String::new();
|
||||||
|
loop {
|
||||||
|
let page = match retried(TRIES, RETRY_AFTER, || page(agent, markets, &query, &cursor)) {
|
||||||
|
Ok(page) => page,
|
||||||
|
Err(error) => {
|
||||||
|
log!("kalshi: rescan failed: {error}");
|
||||||
|
return findings.send(Finding::ScanFailed);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let last = page.markets.is_empty() || page.cursor.is_empty();
|
||||||
|
findings.send(Finding::ScanPage(tracked(page.markets, Utc::now())))?;
|
||||||
|
if last {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
cursor = page.cursor;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
findings.send(Finding::ScanDone)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn recent(
|
||||||
|
agent: &ureq::Agent,
|
||||||
|
markets: &str,
|
||||||
|
since: DateTime<Utc>,
|
||||||
|
findings: &Sender<Finding<Market>>,
|
||||||
|
) -> Sent {
|
||||||
|
let since = since.timestamp().to_string();
|
||||||
|
// Created, whatever their status: open and unopened alike.
|
||||||
|
let created = [("min_created_ts", since.clone()), mve_excluded()];
|
||||||
|
let closed = [
|
||||||
|
("status", "closed".to_string()),
|
||||||
|
("min_close_ts", since),
|
||||||
|
mve_excluded(),
|
||||||
|
];
|
||||||
|
let result = every_page(agent, markets, &created).and_then(|created| {
|
||||||
|
findings.send(Finding::Created(tracked(created, Utc::now())))?;
|
||||||
|
every_page(agent, markets, &closed)
|
||||||
|
});
|
||||||
|
match result {
|
||||||
|
Ok(closed) => {
|
||||||
|
let tickers = closed.into_iter().map(|market| market.ticker);
|
||||||
|
findings.send(Finding::Closed(tickers.collect()))?;
|
||||||
|
findings.send(Finding::RecentDone)
|
||||||
|
}
|
||||||
|
Err(error) => {
|
||||||
|
log!("kalshi: recent check failed: {error}");
|
||||||
|
findings.send(Finding::RecentFailed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Combos (MVEs) are not in the challenge.
|
||||||
|
fn mve_excluded() -> (&'static str, String) {
|
||||||
|
("mve_filter", "exclude".to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn every_page(
|
||||||
|
agent: &ureq::Agent,
|
||||||
|
markets: &str,
|
||||||
|
query: &[(&str, String)],
|
||||||
|
) -> anyhow::Result<Vec<ListedMarket>> {
|
||||||
|
let mut listed = Vec::new();
|
||||||
|
let mut cursor = String::new();
|
||||||
|
loop {
|
||||||
|
let page = retried(TRIES, RETRY_AFTER, || page(agent, markets, query, &cursor))?;
|
||||||
|
let last = page.markets.is_empty() || page.cursor.is_empty();
|
||||||
|
listed.extend(page.markets);
|
||||||
|
if last {
|
||||||
|
return Ok(listed);
|
||||||
|
}
|
||||||
|
cursor = page.cursor;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct Page {
|
||||||
|
markets: Vec<ListedMarket>,
|
||||||
|
/// Empty on the last page.
|
||||||
|
#[serde(default)]
|
||||||
|
cursor: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Deserialize)]
|
||||||
|
struct ListedMarket {
|
||||||
|
ticker: String,
|
||||||
|
/// `active` once open, `initialized` before.
|
||||||
|
status: String,
|
||||||
|
open_time: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn page(
|
||||||
|
agent: &ureq::Agent,
|
||||||
|
markets: &str,
|
||||||
|
query: &[(&str, String)],
|
||||||
|
cursor: &str,
|
||||||
|
) -> anyhow::Result<Page> {
|
||||||
|
let mut request = agent
|
||||||
|
.get(markets)
|
||||||
|
.query("limit", PAGE_SIZE)
|
||||||
|
.query_pairs(query.iter().map(|(name, value)| (*name, value.as_str())));
|
||||||
|
if !cursor.is_empty() {
|
||||||
|
request = request.query("cursor", cursor);
|
||||||
|
}
|
||||||
|
let body = request.call()?.body_mut().read_to_string()?;
|
||||||
|
Ok(serde_json::from_str(&body)?)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The markets to track among `markets` at `now`: those live, and those
|
||||||
|
/// opening within [`OPENING_AHEAD`].
|
||||||
|
fn tracked(markets: Vec<ListedMarket>, now: DateTime<Utc>) -> Vec<Market> {
|
||||||
|
markets
|
||||||
|
.into_iter()
|
||||||
|
.filter(|market| match market.status.as_str() {
|
||||||
|
"active" => true,
|
||||||
|
"initialized" => market
|
||||||
|
.open_time
|
||||||
|
.as_deref()
|
||||||
|
.and_then(|time| DateTime::parse_from_rfc3339(time).ok())
|
||||||
|
.is_some_and(|opens| opens.to_utc() - now <= OPENING_AHEAD),
|
||||||
|
_ => false,
|
||||||
|
})
|
||||||
|
.map(|market| Market {
|
||||||
|
ticker: market.ticker,
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn fixture(name: &str) -> Page {
|
||||||
|
let path = format!("{}/tests/fixtures/{name}", env!("CARGO_MANIFEST_DIR"));
|
||||||
|
serde_json::from_str(&std::fs::read_to_string(path).unwrap()).unwrap()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn time(text: &str) -> DateTime<Utc> {
|
||||||
|
DateTime::parse_from_rfc3339(text).unwrap().to_utc()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn tickers(markets: &[Market]) -> Vec<&str> {
|
||||||
|
markets
|
||||||
|
.iter()
|
||||||
|
.map(|market| market.ticker.as_str())
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn open_markets_are_tracked_and_the_last_page_has_no_cursor() {
|
||||||
|
let page = fixture("kalshi-open.json");
|
||||||
|
assert!(!page.cursor.is_empty());
|
||||||
|
let live = tracked(page.markets, time("2026-11-11T19:30:00Z"));
|
||||||
|
assert_eq!(live.len(), 3);
|
||||||
|
assert!(live[0].ticker.starts_with("KX"));
|
||||||
|
let last: Page = serde_json::from_str(r#"{"markets":[],"cursor":""}"#).unwrap();
|
||||||
|
assert!(last.cursor.is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The fixture's unopened markets open at 19:19:08.
|
||||||
|
#[test]
|
||||||
|
fn an_unopened_market_is_tracked_from_three_hours_before_its_opening() {
|
||||||
|
let opening = time("2026-11-11T19:19:08Z");
|
||||||
|
let at = |now: DateTime<Utc>| tracked(fixture("kalshi-unopened.json").markets, now);
|
||||||
|
assert!(at(opening - OPENING_AHEAD - TimeDelta::seconds(1)).is_empty());
|
||||||
|
assert_eq!(at(opening - OPENING_AHEAD).len(), 3);
|
||||||
|
// Past its opening and still listed unopened: tracked, empty.
|
||||||
|
assert_eq!(
|
||||||
|
tickers(&at(opening + TimeDelta::minutes(20)))[0],
|
||||||
|
"KXEUROCUPTOTAL-26NOV121400MAXSKY-179"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A local listing: answers each request with the fixture its query asks
|
||||||
|
/// for (an empty last page after a cursor), and tells the queries it got.
|
||||||
|
fn listing(requests: usize) -> (String, std::thread::JoinHandle<Vec<String>>) {
|
||||||
|
use std::io::{BufRead, BufReader, Write};
|
||||||
|
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
|
||||||
|
let url = format!("http://{}", listener.local_addr().unwrap());
|
||||||
|
let server = std::thread::spawn(move || {
|
||||||
|
let mut queries = Vec::new();
|
||||||
|
for _ in 0..requests {
|
||||||
|
let (mut stream, _) = listener.accept().unwrap();
|
||||||
|
let mut line = String::new();
|
||||||
|
BufReader::new(&stream).read_line(&mut line).unwrap();
|
||||||
|
let query = line.split(' ').nth(1).unwrap().to_string();
|
||||||
|
let name = if query.contains("cursor=") {
|
||||||
|
None
|
||||||
|
} else if query.contains("status=unopened") {
|
||||||
|
Some("kalshi-unopened.json")
|
||||||
|
} else if query.contains("status=closed") {
|
||||||
|
Some("kalshi-closed.json")
|
||||||
|
} else {
|
||||||
|
Some("kalshi-open.json")
|
||||||
|
};
|
||||||
|
let body = name.map_or(r#"{"markets":[],"cursor":""}"#.to_string(), |name| {
|
||||||
|
let path = format!("{}/tests/fixtures/{name}", env!("CARGO_MANIFEST_DIR"));
|
||||||
|
std::fs::read_to_string(path).unwrap()
|
||||||
|
});
|
||||||
|
write!(
|
||||||
|
stream,
|
||||||
|
"HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
|
||||||
|
body.len()
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
queries.push(query);
|
||||||
|
}
|
||||||
|
queries
|
||||||
|
});
|
||||||
|
(url, server)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn found(findings: &[Finding<Market>]) -> Vec<String> {
|
||||||
|
findings
|
||||||
|
.iter()
|
||||||
|
.map(|finding| match finding {
|
||||||
|
Finding::ScanPage(markets) => format!("page {}", tickers(markets).join(" ")),
|
||||||
|
Finding::Created(markets) => format!("created {}", tickers(markets).join(" ")),
|
||||||
|
Finding::Closed(closed) => format!("closed {}", closed.join(" ")),
|
||||||
|
other => format!("{other:?}"),
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_rescan_pages_the_unopened_markets_then_the_open_ones() {
|
||||||
|
let (url, server) = listing(4);
|
||||||
|
let (requests_tx, requests) = crossbeam::channel::unbounded();
|
||||||
|
let (findings_tx, findings) = crossbeam::channel::unbounded();
|
||||||
|
requests_tx.send(Request::Rescan).unwrap();
|
||||||
|
drop(requests_tx);
|
||||||
|
run(&url, requests, findings_tx);
|
||||||
|
let queries = server.join().unwrap();
|
||||||
|
// Unopened first: a market opening meanwhile is on one list at least.
|
||||||
|
assert!(queries[0].contains("status=unopened"), "{queries:?}");
|
||||||
|
assert!(queries[1].contains("status=unopened") && queries[1].contains("cursor="));
|
||||||
|
assert!(queries[2].contains("status=open") && !queries[2].contains("cursor="));
|
||||||
|
for query in &queries {
|
||||||
|
assert!(query.contains("mve_filter=exclude") && query.contains("limit=1000"));
|
||||||
|
}
|
||||||
|
let found = found(&findings.try_iter().collect::<Vec<_>>());
|
||||||
|
assert_eq!(found.len(), 6, "{found:?}");
|
||||||
|
assert_eq!(found[0], "ScanStarted");
|
||||||
|
assert!(found[1].starts_with("page KXEUROCUPTOTAL-26NOV121400MAXSKY-179 "));
|
||||||
|
assert_eq!(found[2], "page ");
|
||||||
|
assert!(found[3].starts_with("page KXNCAAWBTOTAL-26NOV11QUINSJU-106 "));
|
||||||
|
assert_eq!(found[5], "ScanDone");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_recent_check_reports_the_markets_created_and_those_closed() {
|
||||||
|
let (url, server) = listing(4);
|
||||||
|
let (requests_tx, requests) = crossbeam::channel::unbounded();
|
||||||
|
let (findings_tx, findings) = crossbeam::channel::unbounded();
|
||||||
|
let since = time("2026-11-11T19:00:00Z");
|
||||||
|
requests_tx.send(Request::Recent { since }).unwrap();
|
||||||
|
drop(requests_tx);
|
||||||
|
run(&url, requests, findings_tx);
|
||||||
|
let queries = server.join().unwrap();
|
||||||
|
let since = format!("={}", since.timestamp());
|
||||||
|
assert!(
|
||||||
|
queries[0].contains(&format!("min_created_ts{since}"))
|
||||||
|
&& !queries[0].contains("status=")
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
queries[2].contains(&format!("min_close_ts{since}"))
|
||||||
|
&& queries[2].contains("status=closed")
|
||||||
|
);
|
||||||
|
let found = found(&findings.try_iter().collect::<Vec<_>>());
|
||||||
|
assert_eq!(found.len(), 3, "{found:?}");
|
||||||
|
assert!(found[0].starts_with("created KXNCAAWBTOTAL-26NOV11QUINSJU-106 "));
|
||||||
|
assert!(found[1].starts_with("closed KXFIBAECUPSPREAD-26NOV111230SZOZIG-SZO23 "));
|
||||||
|
assert_eq!(found[2], "RecentDone");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_closed_market_is_not_tracked() {
|
||||||
|
let closed = fixture("kalshi-closed.json");
|
||||||
|
assert_eq!(closed.markets[0].status, "determined");
|
||||||
|
assert!(tracked(closed.markets, time("2026-11-11T19:30:00Z")).is_empty());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
//! Kalshi: its websocket's `orderbook_delta` channel (the connection's side of
|
||||||
|
//! it, signed with an API key), its market listing, and the clock rule its
|
||||||
|
//! book threads apply (docs/notes.md, Kalshi's websocket; roadmap/06).
|
||||||
|
|
||||||
|
pub mod feed;
|
||||||
|
pub mod listing;
|
||||||
|
pub mod sign;
|
||||||
|
pub mod wire;
|
||||||
|
|
||||||
|
use chrono::Utc;
|
||||||
|
use crossbeam::channel::{Receiver, Sender};
|
||||||
|
use serde_json::json;
|
||||||
|
use tungstenite::client::IntoClientRequest;
|
||||||
|
|
||||||
|
use crate::core::{self, Finding, Request, Undecoded};
|
||||||
|
use sign::Key;
|
||||||
|
|
||||||
|
/// Tickers per subscribe or unsubscribe message.
|
||||||
|
const CHUNK: usize = 500;
|
||||||
|
/// Errors after which the subscription is gone (Kalshi's docs: a channel
|
||||||
|
/// error, a subscription buffer overflow): the connection starts over.
|
||||||
|
const TERMINAL_ERRORS: [u32; 2] = [10, 25];
|
||||||
|
|
||||||
|
/// A tracked market: one book, named by its ticker, which is its id and its
|
||||||
|
/// only symbol.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct Market {
|
||||||
|
pub ticker: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl core::Market for Market {
|
||||||
|
fn id(&self) -> &str {
|
||||||
|
&self.ticker
|
||||||
|
}
|
||||||
|
|
||||||
|
fn symbols(&self) -> &[String] {
|
||||||
|
std::slice::from_ref(&self.ticker)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The exchange: its websocket and listing, and the key that signs the
|
||||||
|
/// websocket's handshake.
|
||||||
|
pub struct Kalshi {
|
||||||
|
pub websocket: String,
|
||||||
|
pub listing: String,
|
||||||
|
pub key: Key,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Kalshi {
|
||||||
|
pub fn production(key: Key) -> Self {
|
||||||
|
Self {
|
||||||
|
websocket: "wss://external-api-ws.kalshi.com/trade-api/ws/v2".into(),
|
||||||
|
listing: "https://external-api.kalshi.com/trade-api/v2".into(),
|
||||||
|
key,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn demo(key: Key) -> Self {
|
||||||
|
Self {
|
||||||
|
websocket: "wss://external-api-ws.demo.kalshi.co/trade-api/ws/v2".into(),
|
||||||
|
listing: "https://external-api.demo.kalshi.co/trade-api/v2".into(),
|
||||||
|
key,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl core::Exchange for Kalshi {
|
||||||
|
type Market = Market;
|
||||||
|
type Event = wire::Event;
|
||||||
|
type Session = Session;
|
||||||
|
type Rule = feed::Clock;
|
||||||
|
|
||||||
|
const NAME: &'static str = "kalshi";
|
||||||
|
|
||||||
|
fn request(&self) -> anyhow::Result<tungstenite::handshake::client::Request> {
|
||||||
|
let mut request = self.websocket.as_str().into_client_request()?;
|
||||||
|
for (name, value) in self.key.headers(Utc::now().timestamp_millis()) {
|
||||||
|
request.headers_mut().insert(name, value.parse()?);
|
||||||
|
}
|
||||||
|
Ok(request)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn session(&self) -> Session {
|
||||||
|
Session::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rule(&self) -> feed::Clock {
|
||||||
|
feed::Clock::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn fetch(&self, requests: Receiver<Request>, findings: Sender<Finding<Market>>) {
|
||||||
|
listing::run(&self.listing, requests, findings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A connection's subscription: one per connection, which every subscribe
|
||||||
|
/// adds to.
|
||||||
|
#[derive(Debug, Default)]
|
||||||
|
pub struct Session {
|
||||||
|
/// The last command's id.
|
||||||
|
command: u64,
|
||||||
|
/// The subscription's id, from its `subscribed` answer.
|
||||||
|
sid: Option<u64>,
|
||||||
|
/// The last message's `seq`: the next one must follow it.
|
||||||
|
seq: Option<u64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Session {
|
||||||
|
fn command(&mut self, cmd: &str, params: serde_json::Value) -> String {
|
||||||
|
self.command += 1;
|
||||||
|
json!({"id": self.command, "cmd": cmd, "params": params}).to_string()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl core::Session for Session {
|
||||||
|
type Event = wire::Event;
|
||||||
|
|
||||||
|
/// Nothing: a subscribe can't name no market, so the first one creates the
|
||||||
|
/// subscription.
|
||||||
|
fn opening(&mut self) -> Vec<String> {
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `use_yes_price` makes a NO bid come as the ask it is; sent, since its
|
||||||
|
/// default is announced to flip.
|
||||||
|
fn subscribe(&mut self, tickers: &[&String]) -> Vec<String> {
|
||||||
|
tickers
|
||||||
|
.chunks(CHUNK)
|
||||||
|
.map(|chunk| {
|
||||||
|
let params = json!({
|
||||||
|
"channels": ["orderbook_delta"],
|
||||||
|
"market_tickers": chunk,
|
||||||
|
"use_yes_price": true,
|
||||||
|
"skip_ticker_ack": true,
|
||||||
|
});
|
||||||
|
self.command("subscribe", params)
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unsubscribe(&mut self, tickers: &[&String]) -> Vec<String> {
|
||||||
|
// Every fresh connection's subscription was 1 (docs/notes.md): an
|
||||||
|
// unsubscribe sent before its answer names it so.
|
||||||
|
let sid = self.sid.unwrap_or(1);
|
||||||
|
tickers
|
||||||
|
.chunks(CHUNK)
|
||||||
|
.map(|chunk| {
|
||||||
|
let params = json!({
|
||||||
|
"sids": [sid],
|
||||||
|
"market_tickers": chunk,
|
||||||
|
"action": "delete_markets",
|
||||||
|
"skip_ticker_ack": true,
|
||||||
|
});
|
||||||
|
self.command("update_subscription", params)
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// None: Kalshi pings a connection it has nothing else to send to, and
|
||||||
|
/// tungstenite answers.
|
||||||
|
fn keepalive(&self) -> Option<&'static str> {
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode(&mut self, text: &str) -> Result<Vec<wire::Event>, Undecoded> {
|
||||||
|
let frame: wire::Frame =
|
||||||
|
serde_json::from_str(text).map_err(|error| Undecoded::Unreadable(error.to_string()))?;
|
||||||
|
if let Some(seq) = frame.seq {
|
||||||
|
let expected = self.seq.map_or(1, |last| last + 1);
|
||||||
|
if seq != expected {
|
||||||
|
return Err(Undecoded::Lost(format!(
|
||||||
|
"message {seq} came, {expected} expected"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
self.seq = Some(seq);
|
||||||
|
}
|
||||||
|
match frame.kind.as_str() {
|
||||||
|
"subscribed" => self.sid = self.sid.or(frame.msg.sid),
|
||||||
|
"error" => {
|
||||||
|
let (code, text) = (frame.msg.code.unwrap_or(0), frame.msg.msg.as_deref());
|
||||||
|
let id = frame.id.unwrap_or(0);
|
||||||
|
if TERMINAL_ERRORS.contains(&code) {
|
||||||
|
return Err(Undecoded::Lost(format!("error {code} ({text:?})")));
|
||||||
|
}
|
||||||
|
log!("kalshi: command {id} refused: error {code} ({text:?})");
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
wire::event(&frame)
|
||||||
|
.map(|event| event.into_iter().collect())
|
||||||
|
.map_err(Undecoded::Unreadable)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::core::Session as _;
|
||||||
|
|
||||||
|
fn decoded(session: &mut Session, text: &str) -> Result<usize, String> {
|
||||||
|
match session.decode(text) {
|
||||||
|
Ok(events) => Ok(events.len()),
|
||||||
|
Err(Undecoded::Lost(why)) => Err(format!("lost: {why}")),
|
||||||
|
Err(Undecoded::Unreadable(why)) => Err(format!("unreadable: {why}")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const DELTA: &str = r#"{"type":"orderbook_delta","sid":1,"seq":SEQ,"msg":{"market_ticker":"M","market_id":"x","clock":5,"price_dollars":"0.5000","delta_fp":"1.00","side":"yes","ts_ms":1}}"#;
|
||||||
|
|
||||||
|
fn delta(seq: u64) -> String {
|
||||||
|
DELTA.replace("SEQ", &seq.to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_skipped_seq_is_a_lost_message() {
|
||||||
|
let mut session = Session::default();
|
||||||
|
let subscribed =
|
||||||
|
r#"{"type":"subscribed","id":1,"msg":{"channel":"orderbook_delta","sid":3}}"#;
|
||||||
|
assert_eq!(decoded(&mut session, subscribed), Ok(0));
|
||||||
|
assert_eq!(session.sid, Some(3));
|
||||||
|
assert_eq!(decoded(&mut session, &delta(1)), Ok(1));
|
||||||
|
// An answer counts too.
|
||||||
|
let ok = r#"{"type":"ok","id":2,"sid":3,"seq":2,"msg":{}}"#;
|
||||||
|
assert_eq!(decoded(&mut session, ok), Ok(0));
|
||||||
|
assert_eq!(decoded(&mut session, &delta(3)), Ok(1));
|
||||||
|
assert_eq!(
|
||||||
|
decoded(&mut session, &delta(5)),
|
||||||
|
Err("lost: message 5 came, 4 expected".into())
|
||||||
|
);
|
||||||
|
// A connection's first message is 1.
|
||||||
|
assert!(decoded(&mut Session::default(), &delta(2)).is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_terminal_error_starts_over_and_others_are_logged() {
|
||||||
|
let error = |code: u32| {
|
||||||
|
format!(r#"{{"type":"error","id":4,"seq":1,"msg":{{"code":{code},"msg":"x"}}}}"#)
|
||||||
|
};
|
||||||
|
assert_eq!(decoded(&mut Session::default(), &error(14)), Ok(0));
|
||||||
|
assert!(
|
||||||
|
decoded(&mut Session::default(), &error(25))
|
||||||
|
.unwrap_err()
|
||||||
|
.starts_with("lost")
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
decoded(&mut Session::default(), "not json")
|
||||||
|
.unwrap_err()
|
||||||
|
.starts_with("unreadable")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn commands_carry_their_ids_and_the_subscription() {
|
||||||
|
let mut session = Session::default();
|
||||||
|
let tickers: Vec<String> = (0..501).map(|n| format!("T{n}")).collect();
|
||||||
|
let refs: Vec<&String> = tickers.iter().collect();
|
||||||
|
let sent: Vec<serde_json::Value> = session
|
||||||
|
.subscribe(&refs)
|
||||||
|
.iter()
|
||||||
|
.map(|text| serde_json::from_str(text).unwrap())
|
||||||
|
.collect();
|
||||||
|
assert_eq!(sent.len(), 2);
|
||||||
|
assert_eq!(sent[0]["id"], 1);
|
||||||
|
assert_eq!(sent[1]["id"], 2);
|
||||||
|
assert_eq!(sent[0]["cmd"], "subscribe");
|
||||||
|
assert_eq!(
|
||||||
|
sent[0]["params"]["market_tickers"]
|
||||||
|
.as_array()
|
||||||
|
.unwrap()
|
||||||
|
.len(),
|
||||||
|
500
|
||||||
|
);
|
||||||
|
assert_eq!(sent[0]["params"]["use_yes_price"], true);
|
||||||
|
session.sid = Some(7);
|
||||||
|
let unsubscribe: serde_json::Value =
|
||||||
|
serde_json::from_str(&session.unsubscribe(&refs[..1])[0]).unwrap();
|
||||||
|
assert_eq!(unsubscribe["id"], 3);
|
||||||
|
assert_eq!(unsubscribe["cmd"], "update_subscription");
|
||||||
|
assert_eq!(unsubscribe["params"]["sids"], json!([7]));
|
||||||
|
assert_eq!(unsubscribe["params"]["action"], "delete_markets");
|
||||||
|
assert_eq!(unsubscribe["params"]["market_tickers"], json!(["T0"]));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod socket_tests;
|
||||||
@@ -0,0 +1,186 @@
|
|||||||
|
//! The key that signs Kalshi's websocket handshake (docs/notes.md, Kalshi's
|
||||||
|
//! websocket): an Ed25519 key, read once from the files that the environment
|
||||||
|
//! names. Neither file's content is ever logged.
|
||||||
|
|
||||||
|
use std::path::Path;
|
||||||
|
|
||||||
|
use base64::Engine;
|
||||||
|
use ring::signature::Ed25519KeyPair;
|
||||||
|
use rustls_pki_types::PrivateKeyDer;
|
||||||
|
use rustls_pki_types::pem::PemObject;
|
||||||
|
|
||||||
|
/// The variables naming the two files.
|
||||||
|
pub const KEY_ID_FILE: &str = "KALSHI_KEY_ID_FILE";
|
||||||
|
pub const PRIVATE_KEY_FILE: &str = "KALSHI_PRIVATE_KEY_FILE";
|
||||||
|
|
||||||
|
/// What the handshake signs, after the time: the method and the path.
|
||||||
|
const SIGNED_PATH: &str = "GET/trade-api/ws/v2";
|
||||||
|
|
||||||
|
pub struct Key {
|
||||||
|
id: String,
|
||||||
|
pair: Ed25519KeyPair,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Key {
|
||||||
|
/// The key from the files [`KEY_ID_FILE`] and [`PRIVATE_KEY_FILE`] name.
|
||||||
|
pub fn from_env() -> anyhow::Result<Self> {
|
||||||
|
let path = |variable: &str| {
|
||||||
|
std::env::var_os(variable)
|
||||||
|
.ok_or_else(|| anyhow::anyhow!("{variable} is not set: it names a file of the key"))
|
||||||
|
};
|
||||||
|
Self::from_files(
|
||||||
|
path(KEY_ID_FILE)?.as_ref(),
|
||||||
|
path(PRIVATE_KEY_FILE)?.as_ref(),
|
||||||
|
)
|
||||||
|
.map_err(|error| {
|
||||||
|
anyhow::anyhow!("{error} (the files {KEY_ID_FILE} and {PRIVATE_KEY_FILE} name)")
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The key id in `id_file`, the private key (a PKCS#8 PEM) in `key_file`.
|
||||||
|
pub fn from_files(id_file: &Path, key_file: &Path) -> anyhow::Result<Self> {
|
||||||
|
let id = std::fs::read_to_string(id_file)
|
||||||
|
.map_err(|error| anyhow::anyhow!("{}: {error}", id_file.display()))?
|
||||||
|
.trim()
|
||||||
|
.to_string();
|
||||||
|
anyhow::ensure!(!id.is_empty(), "{}: no key id in it", id_file.display());
|
||||||
|
let pem = std::fs::read(key_file)
|
||||||
|
.map_err(|error| anyhow::anyhow!("{}: {error}", key_file.display()))?;
|
||||||
|
// The errors name the file, never its content: the PEM reader's own
|
||||||
|
// error quotes what it read, so it is dropped.
|
||||||
|
let der = PrivateKeyDer::from_pem_slice(&pem)
|
||||||
|
.map_err(|_| anyhow::anyhow!("{}: not a PEM private key", key_file.display()))?;
|
||||||
|
let PrivateKeyDer::Pkcs8(der) = der else {
|
||||||
|
anyhow::bail!(
|
||||||
|
"{}: not a PKCS#8 key; Kalshi's Ed25519 keys are",
|
||||||
|
key_file.display()
|
||||||
|
);
|
||||||
|
};
|
||||||
|
let pair = Ed25519KeyPair::from_pkcs8_maybe_unchecked(der.secret_pkcs8_der())
|
||||||
|
.map_err(|_| anyhow::anyhow!("{}: not an Ed25519 key", key_file.display()))?;
|
||||||
|
Ok(Self { id, pair })
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The handshake's three headers at `millis` since the epoch.
|
||||||
|
pub fn headers(&self, millis: i64) -> [(&'static str, String); 3] {
|
||||||
|
let signature = self.pair.sign(format!("{millis}{SIGNED_PATH}").as_bytes());
|
||||||
|
[
|
||||||
|
("KALSHI-ACCESS-KEY", self.id.clone()),
|
||||||
|
("KALSHI-ACCESS-TIMESTAMP", millis.to_string()),
|
||||||
|
(
|
||||||
|
"KALSHI-ACCESS-SIGNATURE",
|
||||||
|
base64::engine::general_purpose::STANDARD.encode(signature.as_ref()),
|
||||||
|
),
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RFC 8032's first test key (section 7.1), as PKCS#8: a known key, so a
|
||||||
|
/// known signature.
|
||||||
|
#[cfg(test)]
|
||||||
|
const TEST_KEY: &str = "-----BEGIN PRIVATE KEY-----
|
||||||
|
MC4CAQAwBQYDK2VwBCIEIJ1hsZ3v/VpguoRK9JLsLMREScVpezJpGXA7rAMcrn9g
|
||||||
|
-----END PRIVATE KEY-----
|
||||||
|
";
|
||||||
|
|
||||||
|
/// [`TEST_KEY`], with the id `a-key-id`.
|
||||||
|
#[cfg(test)]
|
||||||
|
pub fn test_key() -> Key {
|
||||||
|
let der = PrivateKeyDer::from_pem_slice(TEST_KEY.as_bytes()).unwrap();
|
||||||
|
let PrivateKeyDer::Pkcs8(der) = der else {
|
||||||
|
unreachable!("PKCS#8")
|
||||||
|
};
|
||||||
|
Key {
|
||||||
|
id: "a-key-id".into(),
|
||||||
|
pair: Ed25519KeyPair::from_pkcs8_maybe_unchecked(der.secret_pkcs8_der()).unwrap(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn write(dir: &Path, name: &str, text: &str) -> std::path::PathBuf {
|
||||||
|
let path = dir.join(name);
|
||||||
|
std::fs::write(&path, text).unwrap();
|
||||||
|
path
|
||||||
|
}
|
||||||
|
|
||||||
|
fn scratch(name: &str) -> std::path::PathBuf {
|
||||||
|
let dir = std::env::temp_dir().join(format!("kalshi-sign-{name}-{}", std::process::id()));
|
||||||
|
std::fs::create_dir_all(&dir).unwrap();
|
||||||
|
dir
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn signs_the_time_method_and_path() {
|
||||||
|
let dir = scratch("signs");
|
||||||
|
let key = Key::from_files(
|
||||||
|
&write(&dir, "id", "a-key-id\n"),
|
||||||
|
&write(&dir, "key.pem", TEST_KEY),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
let [id, time, signature] = key.headers(1_669_149_841_234);
|
||||||
|
assert_eq!(id, ("KALSHI-ACCESS-KEY", "a-key-id".to_string()));
|
||||||
|
assert_eq!(
|
||||||
|
time,
|
||||||
|
("KALSHI-ACCESS-TIMESTAMP", "1669149841234".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(signature.0, "KALSHI-ACCESS-SIGNATURE");
|
||||||
|
// Checked against the RFC's public key for this one.
|
||||||
|
let public = ring::signature::UnparsedPublicKey::new(
|
||||||
|
&ring::signature::ED25519,
|
||||||
|
[
|
||||||
|
0xd7, 0x5a, 0x98, 0x01, 0x82, 0xb1, 0x0a, 0xb7, 0xd5, 0x4b, 0xfe, 0xd3, 0xc9, 0x64,
|
||||||
|
0x07, 0x3a, 0x0e, 0xe1, 0x72, 0xf3, 0xda, 0xa6, 0x23, 0x25, 0xaf, 0x02, 0x1a, 0x68,
|
||||||
|
0xf7, 0x07, 0x51, 0x1a,
|
||||||
|
],
|
||||||
|
);
|
||||||
|
let bytes = base64::engine::general_purpose::STANDARD
|
||||||
|
.decode(&signature.1)
|
||||||
|
.unwrap();
|
||||||
|
public
|
||||||
|
.verify(b"1669149841234GET/trade-api/ws/v2", &bytes)
|
||||||
|
.unwrap();
|
||||||
|
// Ed25519 is deterministic: the same time, the same signature.
|
||||||
|
assert_eq!(key.headers(1_669_149_841_234)[2], signature);
|
||||||
|
std::fs::remove_dir_all(dir).unwrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_missing_or_wrong_file_says_which_and_never_what_it_holds() {
|
||||||
|
let dir = scratch("errors");
|
||||||
|
let id = write(&dir, "id", "a-key-id");
|
||||||
|
let error = |id: &Path, key: &Path| Key::from_files(id, key).err().unwrap().to_string();
|
||||||
|
|
||||||
|
let missing = error(&id, &dir.join("absent.pem"));
|
||||||
|
assert!(missing.contains("absent.pem"), "{missing}");
|
||||||
|
|
||||||
|
// A key flattened onto one line: the PEM reader's own error quotes
|
||||||
|
// what it read, so ours must not carry it.
|
||||||
|
let flat = write(&dir, "flat.pem", &TEST_KEY.replace('\n', " "));
|
||||||
|
let flattened = error(&id, &flat);
|
||||||
|
assert!(flattened.contains("not a PEM private key"), "{flattened}");
|
||||||
|
assert!(flattened.ends_with("not a PEM private key"), "{flattened}");
|
||||||
|
|
||||||
|
let garbage = write(&dir, "garbage.pem", "not a key at all");
|
||||||
|
let unreadable = error(&id, &garbage);
|
||||||
|
assert!(
|
||||||
|
unreadable.ends_with("not a PEM private key"),
|
||||||
|
"{unreadable}"
|
||||||
|
);
|
||||||
|
|
||||||
|
// An RSA key in PKCS#1, as Kalshi generates its RSA keys: refused.
|
||||||
|
let rsa = write(
|
||||||
|
&dir,
|
||||||
|
"rsa.pem",
|
||||||
|
"-----BEGIN RSA PRIVATE KEY-----\nMAA=\n-----END RSA PRIVATE KEY-----\n",
|
||||||
|
);
|
||||||
|
assert!(error(&id, &rsa).contains("not a PKCS#8 key"));
|
||||||
|
|
||||||
|
let empty = write(&dir, "empty-id", "\n");
|
||||||
|
let key = write(&dir, "key.pem", TEST_KEY);
|
||||||
|
assert!(error(&empty, &key).contains("no key id"));
|
||||||
|
std::fs::remove_dir_all(dir).unwrap();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
//! The connection loop (`src/core/socket.rs`) against a local server, with
|
||||||
|
//! Kalshi's codec: the signed handshake, the server's ping, and a skipped
|
||||||
|
//! `seq`.
|
||||||
|
|
||||||
|
use std::net::TcpListener;
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
use std::thread;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use crossbeam::channel::unbounded;
|
||||||
|
use tungstenite::handshake::server::{Request, Response};
|
||||||
|
use tungstenite::{Bytes, Message};
|
||||||
|
|
||||||
|
use super::{Kalshi, sign};
|
||||||
|
use crate::core::socket::{Command, Notice, SILENT_AFTER, run};
|
||||||
|
use crate::core::{BOOK_THREADS, Received};
|
||||||
|
|
||||||
|
const SNAPSHOT: &str = r#"{"type":"orderbook_snapshot","sid":1,"seq":1,"msg":{"market_ticker":"M","market_id":"x","clock":10,"yes_dollars_fp":[["0.4000","5.00"]]},"sending_ts_ms":1794422918369}"#;
|
||||||
|
|
||||||
|
fn delta(seq: u64) -> String {
|
||||||
|
format!(
|
||||||
|
r#"{{"type":"orderbook_delta","sid":1,"seq":{seq},"msg":{{"market_ticker":"M","market_id":"x","clock":{seq}0,"price_dollars":"0.4000","delta_fp":"1.00","side":"yes","ts_ms":1794422918370}}}}"#
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
// The handshake callback's error type is tungstenite's, not ours.
|
||||||
|
#[allow(clippy::result_large_err)]
|
||||||
|
fn signs_its_handshake_answers_pings_and_reconnects_on_a_skipped_seq() {
|
||||||
|
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||||
|
let url = format!("ws://{}/trade-api/ws/v2", listener.local_addr().unwrap());
|
||||||
|
let server = thread::spawn(move || {
|
||||||
|
let mut seen = Vec::new();
|
||||||
|
for connection in 0..2 {
|
||||||
|
let (stream, _) = listener.accept().unwrap();
|
||||||
|
stream
|
||||||
|
.set_read_timeout(Some(Duration::from_secs(5)))
|
||||||
|
.unwrap();
|
||||||
|
let mut headers = Vec::new();
|
||||||
|
let mut socket =
|
||||||
|
tungstenite::accept_hdr(stream, |request: &Request, response: Response| {
|
||||||
|
for name in [
|
||||||
|
"KALSHI-ACCESS-KEY",
|
||||||
|
"KALSHI-ACCESS-TIMESTAMP",
|
||||||
|
"KALSHI-ACCESS-SIGNATURE",
|
||||||
|
] {
|
||||||
|
let value = request
|
||||||
|
.headers()
|
||||||
|
.get(name)
|
||||||
|
.map(|value| value.to_str().unwrap().to_string());
|
||||||
|
headers.push(value);
|
||||||
|
}
|
||||||
|
Ok(response)
|
||||||
|
})
|
||||||
|
.unwrap();
|
||||||
|
let subscribe: serde_json::Value =
|
||||||
|
serde_json::from_str(socket.read().unwrap().to_text().unwrap()).unwrap();
|
||||||
|
assert_eq!(subscribe["cmd"], "subscribe");
|
||||||
|
assert_eq!(
|
||||||
|
subscribe["params"]["market_tickers"],
|
||||||
|
serde_json::json!(["M"])
|
||||||
|
);
|
||||||
|
socket
|
||||||
|
.send(Message::text(
|
||||||
|
r#"{"type":"subscribed","id":1,"msg":{"channel":"orderbook_delta","sid":1}}"#,
|
||||||
|
))
|
||||||
|
.unwrap();
|
||||||
|
socket.send(Message::text(SNAPSHOT)).unwrap();
|
||||||
|
socket.send(Message::text(delta(2))).unwrap();
|
||||||
|
if connection == 1 {
|
||||||
|
socket.close(None).unwrap();
|
||||||
|
while socket.read().is_ok() {}
|
||||||
|
seen.push((headers, false));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
socket
|
||||||
|
.send(Message::Ping(Bytes::from_static(b"heartbeat")))
|
||||||
|
.unwrap();
|
||||||
|
let ponged = loop {
|
||||||
|
match socket.read().unwrap() {
|
||||||
|
Message::Pong(body) => break body == Bytes::from_static(b"heartbeat"),
|
||||||
|
_ => continue,
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// 3 never comes.
|
||||||
|
socket.send(Message::text(delta(4))).unwrap();
|
||||||
|
while socket.read().is_ok() {}
|
||||||
|
seen.push((headers, ponged));
|
||||||
|
}
|
||||||
|
seen
|
||||||
|
});
|
||||||
|
|
||||||
|
let kalshi = Kalshi {
|
||||||
|
websocket: url,
|
||||||
|
listing: String::new(),
|
||||||
|
key: sign::test_key(),
|
||||||
|
};
|
||||||
|
let (book_threads, events): (Vec<_>, Vec<_>) = (0..BOOK_THREADS).map(|_| unbounded()).unzip();
|
||||||
|
let (commands_tx, commands) = unbounded();
|
||||||
|
commands_tx
|
||||||
|
.send(Command::Subscribe(vec!["M".into()]))
|
||||||
|
.unwrap();
|
||||||
|
let (notices_tx, notices) = unbounded();
|
||||||
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
|
let client_stop = Arc::clone(&stop);
|
||||||
|
let client = thread::spawn(move || {
|
||||||
|
run(
|
||||||
|
&kalshi,
|
||||||
|
0,
|
||||||
|
&book_threads,
|
||||||
|
&commands,
|
||||||
|
¬ices_tx,
|
||||||
|
SILENT_AFTER,
|
||||||
|
&client_stop,
|
||||||
|
)
|
||||||
|
});
|
||||||
|
let seen = server.join().unwrap();
|
||||||
|
stop.store(true, Ordering::Relaxed);
|
||||||
|
client.join().unwrap();
|
||||||
|
|
||||||
|
for (headers, _) in &seen {
|
||||||
|
assert_eq!(headers[0].as_deref(), Some("a-key-id"));
|
||||||
|
assert!(
|
||||||
|
headers[1]
|
||||||
|
.as_deref()
|
||||||
|
.is_some_and(|time| time.parse::<i64>().is_ok())
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
headers[2]
|
||||||
|
.as_deref()
|
||||||
|
.is_some_and(|signature| signature.len() == 88)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assert!(seen[0].1, "the ping got its pong");
|
||||||
|
let notices: Vec<Notice> = notices.try_iter().collect();
|
||||||
|
assert_eq!(
|
||||||
|
notices[..3],
|
||||||
|
[Notice::Up(0), Notice::Down(0), Notice::Up(0)]
|
||||||
|
);
|
||||||
|
// The book and the change before the gap reached the book thread; the
|
||||||
|
// change after it didn't.
|
||||||
|
let received: Vec<_> = events
|
||||||
|
.iter()
|
||||||
|
.flat_map(|receiver| receiver.try_iter())
|
||||||
|
.collect();
|
||||||
|
// Every book thread heard it go down each time: after the skipped
|
||||||
|
// `seq`, and when the server closed the second connection.
|
||||||
|
let downs = received
|
||||||
|
.iter()
|
||||||
|
.filter(|received| matches!(received, Received::Down(0)))
|
||||||
|
.count();
|
||||||
|
assert_eq!(downs, 2 * BOOK_THREADS);
|
||||||
|
let events: Vec<_> = received
|
||||||
|
.iter()
|
||||||
|
.filter(|received| matches!(received, Received::Event(_)))
|
||||||
|
.collect();
|
||||||
|
assert_eq!(events.len(), 4, "two per connection: {events:?}");
|
||||||
|
}
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
//! Kalshi's websocket messages, as the exchange writes them, and how a
|
||||||
|
//! connection reads them: each frame is one message, numbered by `seq` across
|
||||||
|
//! the connection (docs/notes.md, Kalshi's websocket).
|
||||||
|
|
||||||
|
use chrono::{DateTime, Utc};
|
||||||
|
use engineering_challenge::Side;
|
||||||
|
use engineering_challenge::book::parse_fixed;
|
||||||
|
use serde::Deserialize;
|
||||||
|
|
||||||
|
/// A message of the `orderbook_delta` channel, read.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub enum Event {
|
||||||
|
Book(BookMessage),
|
||||||
|
Change(ChangeMessage),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A market's full book: its YES bids are the bids; with `use_yes_price`,
|
||||||
|
/// its NO bids come as the asks they are, at one minus their price.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub struct BookMessage {
|
||||||
|
pub ticker: String,
|
||||||
|
/// The clock of the last change it holds (0 before the market opens).
|
||||||
|
pub clock: u64,
|
||||||
|
pub bids: Vec<(i64, i64)>,
|
||||||
|
pub asks: Vec<(i64, i64)>,
|
||||||
|
/// When the exchange sent it: a book has no time of its own.
|
||||||
|
pub sent_at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A change of one level: its quantity moves by `delta`, which can be
|
||||||
|
/// negative.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
pub struct ChangeMessage {
|
||||||
|
pub ticker: String,
|
||||||
|
pub clock: u64,
|
||||||
|
pub side: Side,
|
||||||
|
pub price: i64,
|
||||||
|
pub delta: i64,
|
||||||
|
pub at: DateTime<Utc>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl crate::core::Event for Event {
|
||||||
|
fn market(&self) -> Option<&str> {
|
||||||
|
Some(match self {
|
||||||
|
Event::Book(book) => &book.ticker,
|
||||||
|
Event::Change(change) => &change.ticker,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_book(&self) -> bool {
|
||||||
|
matches!(self, Event::Book(_))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a frame holds, every field the channel's messages use.
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
pub struct Frame {
|
||||||
|
#[serde(rename = "type")]
|
||||||
|
pub kind: String,
|
||||||
|
/// The command it answers, for an answer.
|
||||||
|
pub id: Option<u64>,
|
||||||
|
/// Counts every message of the connection's subscription but
|
||||||
|
/// `subscribed`, from 1.
|
||||||
|
pub seq: Option<u64>,
|
||||||
|
pub sending_ts_ms: Option<i64>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub msg: Body,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Default, Deserialize)]
|
||||||
|
pub struct Body {
|
||||||
|
pub market_ticker: Option<String>,
|
||||||
|
pub clock: Option<u64>,
|
||||||
|
pub yes_dollars_fp: Option<Vec<(String, String)>>,
|
||||||
|
pub no_dollars_fp: Option<Vec<(String, String)>>,
|
||||||
|
pub price_dollars: Option<String>,
|
||||||
|
pub delta_fp: Option<String>,
|
||||||
|
pub side: Option<String>,
|
||||||
|
pub ts_ms: Option<i64>,
|
||||||
|
/// `subscribed`: the subscription's id.
|
||||||
|
pub sid: Option<u64>,
|
||||||
|
/// `error`: its code and text.
|
||||||
|
pub code: Option<u32>,
|
||||||
|
pub msg: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The event a frame of the book channel holds; none for the others (answers
|
||||||
|
/// to commands), `Err` with what is wrong for a book or change unreadable.
|
||||||
|
pub fn event(frame: &Frame) -> Result<Option<Event>, String> {
|
||||||
|
let body = &frame.msg;
|
||||||
|
let ticker = || body.market_ticker.clone().ok_or("no market_ticker");
|
||||||
|
let clock = || body.clock.ok_or("no clock");
|
||||||
|
match frame.kind.as_str() {
|
||||||
|
"orderbook_snapshot" => {
|
||||||
|
let side = |levels: &Option<Vec<(String, String)>>| {
|
||||||
|
levels
|
||||||
|
.iter()
|
||||||
|
.flatten()
|
||||||
|
.map(|(price, quantity)| Some((parse_fixed(price)?, parse_fixed(quantity)?)))
|
||||||
|
.collect::<Option<Vec<_>>>()
|
||||||
|
.ok_or("a level is not two decimals")
|
||||||
|
};
|
||||||
|
let sent_at = frame
|
||||||
|
.sending_ts_ms
|
||||||
|
.and_then(DateTime::from_timestamp_millis)
|
||||||
|
.ok_or("no sending_ts_ms")?;
|
||||||
|
Ok(Some(Event::Book(BookMessage {
|
||||||
|
ticker: ticker()?,
|
||||||
|
clock: clock()?,
|
||||||
|
bids: side(&body.yes_dollars_fp)?,
|
||||||
|
asks: side(&body.no_dollars_fp)?,
|
||||||
|
sent_at,
|
||||||
|
})))
|
||||||
|
}
|
||||||
|
"orderbook_delta" => {
|
||||||
|
let side = match body.side.as_deref() {
|
||||||
|
Some("yes") => Side::Buy,
|
||||||
|
Some("no") => Side::Sell,
|
||||||
|
_ => return Err("side is neither yes nor no".into()),
|
||||||
|
};
|
||||||
|
let price = body.price_dollars.as_deref().and_then(parse_fixed);
|
||||||
|
let delta = body.delta_fp.as_deref().and_then(parse_signed);
|
||||||
|
let at = body.ts_ms.and_then(DateTime::from_timestamp_millis);
|
||||||
|
let (Some(price), Some(delta), Some(at)) = (price, delta, at) else {
|
||||||
|
return Err("price_dollars, delta_fp or ts_ms unreadable".into());
|
||||||
|
};
|
||||||
|
Ok(Some(Event::Change(ChangeMessage {
|
||||||
|
ticker: ticker()?,
|
||||||
|
clock: clock()?,
|
||||||
|
side,
|
||||||
|
price,
|
||||||
|
delta,
|
||||||
|
at,
|
||||||
|
})))
|
||||||
|
}
|
||||||
|
_ => Ok(None),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A signed decimal (`"-54.00"`, `"20.00"`), exactly.
|
||||||
|
fn parse_signed(text: &str) -> Option<i64> {
|
||||||
|
match text.strip_prefix('-') {
|
||||||
|
Some(magnitude) => parse_fixed(magnitude).map(|value| -value),
|
||||||
|
None => parse_fixed(text),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn read(text: &str) -> Result<Option<Event>, String> {
|
||||||
|
event(&serde_json::from_str(text).unwrap())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn reads_a_book_and_a_change_as_recorded() {
|
||||||
|
let book = r#"{"type":"orderbook_snapshot","sid":1,"seq":2,"msg":{"market_ticker":"KXHIGHNY-26NOV12-T59","market_id":"01a1","clock":56884934192,"yes_dollars_fp":[["0.4800","80.00"],["0.6500","10.00"]],"no_dollars_fp":[["0.9100","80.00"],["0.6600","7.00"]]},"sending_ts_ms":1794422918369}"#;
|
||||||
|
assert_eq!(
|
||||||
|
read(book).unwrap(),
|
||||||
|
Some(Event::Book(BookMessage {
|
||||||
|
ticker: "KXHIGHNY-26NOV12-T59".into(),
|
||||||
|
clock: 56_884_934_192,
|
||||||
|
bids: vec![(480_000, 80_000_000), (650_000, 10_000_000)],
|
||||||
|
asks: vec![(910_000, 80_000_000), (660_000, 7_000_000)],
|
||||||
|
sent_at: DateTime::from_timestamp_millis(1_794_422_918_369).unwrap(),
|
||||||
|
}))
|
||||||
|
);
|
||||||
|
let change = r#"{"type":"orderbook_delta","sid":1,"seq":9,"msg":{"market_ticker":"M","market_id":"01a1","clock":56885842636,"price_dollars":"0.8100","delta_fp":"-1.50","side":"no","ts":"2026-11-11T18:36:37.012Z","ts_ms":1794422197012},"sending_ts_ms":1794422197023}"#;
|
||||||
|
assert_eq!(
|
||||||
|
read(change).unwrap(),
|
||||||
|
Some(Event::Change(ChangeMessage {
|
||||||
|
ticker: "M".into(),
|
||||||
|
clock: 56_885_842_636,
|
||||||
|
side: Side::Sell,
|
||||||
|
price: 810_000,
|
||||||
|
delta: -1_500_000,
|
||||||
|
at: DateTime::from_timestamp_millis(1_794_422_197_012).unwrap(),
|
||||||
|
}))
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_empty_book_has_no_levels_and_answers_are_no_events() {
|
||||||
|
// An unopened market's book, sides omitted.
|
||||||
|
let empty = r#"{"type":"orderbook_snapshot","sid":1,"seq":2,"msg":{"market_ticker":"KXQUICKSETTLE-11NOV26H1400-2","market_id":"","clock":0},"sending_ts_ms":1794422918369}"#;
|
||||||
|
let Some(Event::Book(book)) = read(empty).unwrap() else {
|
||||||
|
panic!("not a book");
|
||||||
|
};
|
||||||
|
assert!(book.bids.is_empty() && book.asks.is_empty());
|
||||||
|
assert_eq!(book.clock, 0);
|
||||||
|
for answer in [
|
||||||
|
r#"{"type":"subscribed","id":1,"msg":{"channel":"orderbook_delta","sid":1},"sending_ts_ms":1}"#,
|
||||||
|
r#"{"type":"ok","id":2,"sid":1,"seq":6,"sending_ts_ms":1,"msg":{}}"#,
|
||||||
|
r#"{"type":"error","id":2,"sid":1,"seq":860,"msg":{"code":14,"msg":"Market Ticker required"}}"#,
|
||||||
|
] {
|
||||||
|
assert_eq!(read(answer).unwrap(), None, "{answer}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_unreadable_change_says_so() {
|
||||||
|
for bad in [
|
||||||
|
r#"{"type":"orderbook_delta","seq":1,"msg":{"market_ticker":"M","clock":1,"price_dollars":"0.81","delta_fp":"1e3","side":"no","ts_ms":1}}"#,
|
||||||
|
r#"{"type":"orderbook_delta","seq":1,"msg":{"market_ticker":"M","clock":1,"price_dollars":"0.81","delta_fp":"1.00","side":"maybe","ts_ms":1}}"#,
|
||||||
|
r#"{"type":"orderbook_delta","seq":1,"msg":{"clock":1,"price_dollars":"0.81","delta_fp":"1.00","side":"no","ts_ms":1}}"#,
|
||||||
|
] {
|
||||||
|
assert!(read(bad).is_err(), "{bad}");
|
||||||
|
}
|
||||||
|
assert_eq!(parse_signed("-0.01"), Some(-10_000));
|
||||||
|
assert_eq!(parse_signed("--1"), None);
|
||||||
|
}
|
||||||
|
}
|
||||||
+33
-10
@@ -1,8 +1,9 @@
|
|||||||
//! The collector (roadmap/04): an orchestrator keeps the tracked markets equal
|
//! The collector (roadmap/04, roadmap/06): an orchestrator keeps the tracked
|
||||||
//! to Gamma's live ones and places them on connections (read, stamp, decode,
|
//! markets equal to the exchange's live ones and places them on connections
|
||||||
//! route), which feed the book threads (fold, check, resync), all writing
|
//! (read, stamp, decode, route), which feed the book threads (fold, check,
|
||||||
//! through one writer (SBE).
|
//! resync), all writing through one writer (SBE).
|
||||||
//! Usage: `engineering-challenge [OUTPUT]`, OUTPUT defaulting to stdout; logs
|
//! Usage: `engineering-challenge EXCHANGE [OUTPUT]`, EXCHANGE one of
|
||||||
|
//! `polymarket`, `kalshi` or `kalshi-demo`, OUTPUT defaulting to stdout; logs
|
||||||
//! go to stderr.
|
//! go to stderr.
|
||||||
|
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
@@ -19,6 +20,8 @@ macro_rules! log {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
mod core;
|
||||||
|
mod kalshi;
|
||||||
mod polymarket;
|
mod polymarket;
|
||||||
|
|
||||||
use engineering_challenge::book::Output;
|
use engineering_challenge::book::Output;
|
||||||
@@ -26,11 +29,31 @@ use engineering_challenge::book::Output;
|
|||||||
/// `AlignedSbeReader::open` reads nothing else.
|
/// `AlignedSbeReader::open` reads nothing else.
|
||||||
const CHUNK: usize = 64 << 20;
|
const CHUNK: usize = 64 << 20;
|
||||||
|
|
||||||
fn main() -> anyhow::Result<()> {
|
const USAGE: &str = "usage: engineering-challenge <polymarket|kalshi|kalshi-demo> [OUTPUT]";
|
||||||
let output = std::env::args()
|
|
||||||
.nth(1)
|
|
||||||
.unwrap_or_else(|| "/dev/stdout".to_string());
|
|
||||||
|
|
||||||
|
fn main() -> anyhow::Result<()> {
|
||||||
|
let mut args = std::env::args().skip(1);
|
||||||
|
let exchange = args.next().ok_or_else(|| anyhow::anyhow!(USAGE))?;
|
||||||
|
let output = args.next().unwrap_or_else(|| "/dev/stdout".to_string());
|
||||||
|
// Before anything starts: a key that can't be read stops the run here.
|
||||||
|
match exchange.as_str() {
|
||||||
|
"polymarket" => collect(
|
||||||
|
polymarket::Polymarket {
|
||||||
|
url: polymarket::MARKET_CHANNEL_URL.to_string(),
|
||||||
|
},
|
||||||
|
output,
|
||||||
|
),
|
||||||
|
"kalshi" => collect(
|
||||||
|
kalshi::Kalshi::production(kalshi::sign::Key::from_env()?),
|
||||||
|
output,
|
||||||
|
),
|
||||||
|
"kalshi-demo" => collect(kalshi::Kalshi::demo(kalshi::sign::Key::from_env()?), output),
|
||||||
|
_ => anyhow::bail!("unknown exchange {exchange:?}; {USAGE}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Collects `exchange` into `output` until SIGINT.
|
||||||
|
fn collect<E: core::Exchange>(exchange: E, output: String) -> anyhow::Result<()> {
|
||||||
let stop = Arc::new(AtomicBool::new(false));
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
let on_sigint = Arc::clone(&stop);
|
let on_sigint = Arc::clone(&stop);
|
||||||
ctrlc::set_handler(move || on_sigint.store(true, Ordering::Relaxed))?;
|
ctrlc::set_handler(move || on_sigint.store(true, Ordering::Relaxed))?;
|
||||||
@@ -47,7 +70,7 @@ fn main() -> anyhow::Result<()> {
|
|||||||
// connections; the book threads stop once every connection has, and the
|
// connections; the book threads stop once every connection has, and the
|
||||||
// writer flushes once they all have, even after a panic upstream (the run
|
// writer flushes once they all have, even after a panic upstream (the run
|
||||||
// still fails then).
|
// still fails then).
|
||||||
let failed = polymarket::orchestrator::run(outputs_tx, &stop);
|
let failed = core::orchestrator::run(Arc::new(exchange), outputs_tx, &stop);
|
||||||
let written = writer
|
let written = writer
|
||||||
.join()
|
.join()
|
||||||
.map_err(|_| anyhow::anyhow!("the writer thread panicked"))?;
|
.map_err(|_| anyhow::anyhow!("the writer thread panicked"))?;
|
||||||
|
|||||||
+541
-966
File diff suppressed because it is too large.
Load diff
+8
-43
@@ -2,7 +2,6 @@
|
|||||||
//! threads page it on the orchestrator's request and stream what they read;
|
//! threads page it on the orchestrator's request and stream what they read;
|
||||||
//! they hold no state and no timer (roadmap/03).
|
//! they hold no state and no timer (roadmap/03).
|
||||||
|
|
||||||
use std::thread;
|
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use chrono::{DateTime, Utc};
|
use chrono::{DateTime, Utc};
|
||||||
@@ -10,6 +9,7 @@ use crossbeam::channel::{Receiver, Sender};
|
|||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
|
|
||||||
use super::Market;
|
use super::Market;
|
||||||
|
use crate::core::{Finding, Request, retried};
|
||||||
|
|
||||||
const KEYSET_URL: &str = "https://gamma-api.polymarket.com/markets/keyset";
|
const KEYSET_URL: &str = "https://gamma-api.polymarket.com/markets/keyset";
|
||||||
/// Gamma's page size, whatever a larger `limit` asks for.
|
/// Gamma's page size, whatever a larger `limit` asks for.
|
||||||
@@ -23,31 +23,7 @@ const SCAN_RETRY_AFTER: Duration = Duration::from_secs(5);
|
|||||||
/// null (checked on 4 such tokens, 2026-11-04).
|
/// null (checked on 4 such tokens, 2026-11-04).
|
||||||
const MISSING_MIN_ORDER_SIZE: &str = "0";
|
const MISSING_MIN_ORDER_SIZE: &str = "0";
|
||||||
|
|
||||||
pub enum Request {
|
pub fn run(requests: Receiver<Request>, findings: Sender<Finding<Market>>) {
|
||||||
/// Every live market.
|
|
||||||
Rescan,
|
|
||||||
/// The live markets created and the markets closed since then.
|
|
||||||
Recent { since: DateTime<Utc> },
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug)]
|
|
||||||
pub enum Finding {
|
|
||||||
/// A rescan began: once it is done, a market known now and not seen in it
|
|
||||||
/// is gone.
|
|
||||||
ScanStarted,
|
|
||||||
ScanPage(Vec<Market>),
|
|
||||||
ScanDone,
|
|
||||||
ScanFailed,
|
|
||||||
/// Live markets created since the request's time.
|
|
||||||
Created(Vec<Market>),
|
|
||||||
/// The `conditionId`s of markets closed since the request's time.
|
|
||||||
Closed(Vec<String>),
|
|
||||||
RecentDone,
|
|
||||||
RecentFailed,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A fetcher thread: answers each request until the orchestrator is gone.
|
|
||||||
pub fn run(requests: Receiver<Request>, findings: Sender<Finding>) {
|
|
||||||
let agent = ureq::Agent::new_with_config(
|
let agent = ureq::Agent::new_with_config(
|
||||||
ureq::Agent::config_builder()
|
ureq::Agent::config_builder()
|
||||||
.timeout_global(Some(TIMEOUT))
|
.timeout_global(Some(TIMEOUT))
|
||||||
@@ -65,14 +41,16 @@ pub fn run(requests: Receiver<Request>, findings: Sender<Finding>) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// `Err` once the orchestrator is gone.
|
/// `Err` once the orchestrator is gone.
|
||||||
type Sent = Result<(), crossbeam::channel::SendError<Finding>>;
|
type Sent = Result<(), crossbeam::channel::SendError<Finding<Market>>>;
|
||||||
|
|
||||||
fn rescan(agent: &ureq::Agent, findings: &Sender<Finding>) -> Sent {
|
fn rescan(agent: &ureq::Agent, findings: &Sender<Finding<Market>>) -> Sent {
|
||||||
findings.send(Finding::ScanStarted)?;
|
findings.send(Finding::ScanStarted)?;
|
||||||
let query = [("active", "true"), ("closed", "false")];
|
let query = [("active", "true"), ("closed", "false")];
|
||||||
let mut cursor = None;
|
let mut cursor = None;
|
||||||
loop {
|
loop {
|
||||||
let page = match with_retries(|| page(agent, &query, cursor.as_deref())) {
|
let page = match retried(SCAN_TRIES, SCAN_RETRY_AFTER, || {
|
||||||
|
page(agent, &query, cursor.as_deref())
|
||||||
|
}) {
|
||||||
Ok(page) => page,
|
Ok(page) => page,
|
||||||
Err(error) => {
|
Err(error) => {
|
||||||
log!("polymarket: rescan failed: {error}");
|
log!("polymarket: rescan failed: {error}");
|
||||||
@@ -88,20 +66,7 @@ fn rescan(agent: &ureq::Agent, findings: &Sender<Finding>) -> Sent {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn with_retries(mut fetch: impl FnMut() -> anyhow::Result<Page>) -> anyhow::Result<Page> {
|
fn recent(agent: &ureq::Agent, since: DateTime<Utc>, findings: &Sender<Finding<Market>>) -> Sent {
|
||||||
let mut tries = 1;
|
|
||||||
loop {
|
|
||||||
match fetch() {
|
|
||||||
Err(_) if tries < SCAN_TRIES => {
|
|
||||||
tries += 1;
|
|
||||||
thread::sleep(SCAN_RETRY_AFTER);
|
|
||||||
}
|
|
||||||
result => return result,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn recent(agent: &ureq::Agent, since: DateTime<Utc>, findings: &Sender<Finding>) -> Sent {
|
|
||||||
let created = [
|
let created = [
|
||||||
("active", "true"),
|
("active", "true"),
|
||||||
("closed", "false"),
|
("closed", "false"),
|
||||||
|
|||||||
+91
-33
@@ -1,25 +1,21 @@
|
|||||||
//! Polymarket: which markets are live (Gamma), the orchestrator that places
|
//! Polymarket: its market channel (the connection's side of it), Gamma (which
|
||||||
//! them on connections, the connections, and the book threads they feed
|
//! markets are live), and the hash rule its book threads apply
|
||||||
//! (docs/adr/0003-first-one-wins.md).
|
//! (docs/adr/0003-first-one-wins.md).
|
||||||
|
|
||||||
pub mod feed;
|
pub mod feed;
|
||||||
pub mod gamma;
|
pub mod gamma;
|
||||||
pub mod orchestrator;
|
|
||||||
pub mod socket;
|
|
||||||
pub mod wire;
|
pub mod wire;
|
||||||
|
|
||||||
/// How many book threads own the books. One handled ~30,000 frames a second
|
use crossbeam::channel::{Receiver, Sender};
|
||||||
/// in sprint 02.
|
use serde_json::json;
|
||||||
pub const BOOK_THREADS: usize = 16;
|
use tungstenite::client::IntoClientRequest;
|
||||||
|
|
||||||
/// The book thread that owns a market's books, from its `conditionId` (`0x`
|
use crate::core::{self, Finding, Request, Undecoded};
|
||||||
/// and 64 hex digits, a hash: any 16 of them spread evenly). Computed, so the
|
|
||||||
/// orchestrator and every connection agree without a shared table.
|
pub const MARKET_CHANNEL_URL: &str = "wss://ws-subscriptions-clob.polymarket.com/ws/market";
|
||||||
pub fn book_thread(condition_id: &str) -> usize {
|
/// Tokens per subscribe or unsubscribe message: a first message of more than
|
||||||
let digits = condition_id.get(2..18).unwrap_or_default();
|
/// 500 gets no opening books (docs/notes.md).
|
||||||
// An id of another shape still gets a thread, always the same one.
|
const CHUNK: usize = 500;
|
||||||
u64::from_str_radix(digits, 16).unwrap_or(0) as usize % BOOK_THREADS
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A live market: its symbols and the two inputs of their book hash that the
|
/// A live market: its symbols and the two inputs of their book hash that the
|
||||||
/// websocket doesn't carry (docs/notes.md).
|
/// websocket doesn't carry (docs/notes.md).
|
||||||
@@ -34,27 +30,89 @@ pub struct Market {
|
|||||||
pub neg_risk: bool,
|
pub neg_risk: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl core::Market for Market {
|
||||||
|
fn id(&self) -> &str {
|
||||||
|
&self.condition_id
|
||||||
|
}
|
||||||
|
|
||||||
|
fn symbols(&self) -> &[String] {
|
||||||
|
&self.tokens
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The exchange, at its market channel's URL (the tests' local one in tests).
|
||||||
|
pub struct Polymarket {
|
||||||
|
pub url: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl core::Exchange for Polymarket {
|
||||||
|
type Market = Market;
|
||||||
|
type Event = wire::Event;
|
||||||
|
type Session = Session;
|
||||||
|
type Rule = feed::Chain;
|
||||||
|
|
||||||
|
const NAME: &'static str = "polymarket";
|
||||||
|
|
||||||
|
fn request(&self) -> anyhow::Result<tungstenite::handshake::client::Request> {
|
||||||
|
Ok(self.url.as_str().into_client_request()?)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn session(&self) -> Session {
|
||||||
|
Session
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rule(&self) -> feed::Chain {
|
||||||
|
feed::Chain::default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn fetch(&self, requests: Receiver<Request>, findings: Sender<Finding<Market>>) {
|
||||||
|
gamma::run(requests, findings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The market channel keeps no state of its own per connection.
|
||||||
|
pub struct Session;
|
||||||
|
|
||||||
|
impl Session {
|
||||||
|
/// The messages that subscribe or unsubscribe `tokens` on an open
|
||||||
|
/// connection.
|
||||||
|
fn operations(operation: &str, tokens: &[&String]) -> Vec<String> {
|
||||||
|
tokens
|
||||||
|
.chunks(CHUNK)
|
||||||
|
.map(|chunk| json!({"assets_ids": chunk, "operation": operation}).to_string())
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl core::Session for Session {
|
||||||
|
type Event = wire::Event;
|
||||||
|
|
||||||
|
/// Opened empty, then subscribed in chunks: one rule for every token.
|
||||||
|
fn opening(&mut self) -> Vec<String> {
|
||||||
|
vec![json!({"type": "market", "assets_ids": []}).to_string()]
|
||||||
|
}
|
||||||
|
|
||||||
|
fn subscribe(&mut self, tokens: &[&String]) -> Vec<String> {
|
||||||
|
Self::operations("subscribe", tokens)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn unsubscribe(&mut self, tokens: &[&String]) -> Vec<String> {
|
||||||
|
Self::operations("unsubscribe", tokens)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn keepalive(&self) -> Option<&'static str> {
|
||||||
|
Some("PING")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode(&mut self, text: &str) -> Result<Vec<wire::Event>, Undecoded> {
|
||||||
|
wire::decode(text).map_err(|error| Undecoded::Unreadable(error.to_string()))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
pub(crate) fn fixture_markets() -> Vec<Market> {
|
pub(crate) fn fixture_markets() -> Vec<Market> {
|
||||||
gamma::parse_markets(include_str!("../../tests/fixtures/gamma-markets.json"))
|
gamma::parse_markets(include_str!("../../tests/fixtures/gamma-markets.json"))
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod socket_tests;
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_market_always_gets_the_same_book_thread_and_markets_spread() {
|
|
||||||
let markets = fixture_markets();
|
|
||||||
let threads: std::collections::BTreeSet<usize> = markets
|
|
||||||
.iter()
|
|
||||||
.map(|market| book_thread(&market.condition_id))
|
|
||||||
.collect();
|
|
||||||
assert!(threads.iter().all(|&thread| thread < BOOK_THREADS));
|
|
||||||
// 20 markets over 16 threads: several threads, not one.
|
|
||||||
assert!(threads.len() > 5, "{threads:?}");
|
|
||||||
let id = &markets[0].condition_id;
|
|
||||||
assert_eq!(book_thread(id), book_thread(&id.clone()));
|
|
||||||
assert_eq!(book_thread("not an id"), 0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,600 +0,0 @@
|
|||||||
//! A connection: reads the market channel, stamps each frame's receive time,
|
|
||||||
//! decodes it and sends each event to the book thread of its market; and
|
|
||||||
//! sends the exchange what the book threads ask for (subscriptions, resyncs).
|
|
||||||
//! It reconnects when it drops, falls silent, or is recycled.
|
|
||||||
|
|
||||||
use std::collections::BTreeSet;
|
|
||||||
use std::io::ErrorKind;
|
|
||||||
use std::net::{TcpStream, ToSocketAddrs};
|
|
||||||
use std::sync::atomic::{AtomicBool, Ordering};
|
|
||||||
use std::thread;
|
|
||||||
use std::time::{Duration, Instant};
|
|
||||||
|
|
||||||
use chrono::{DateTime, Utc};
|
|
||||||
use crossbeam::channel::{Receiver, Sender};
|
|
||||||
use serde_json::json;
|
|
||||||
use tungstenite::http::Uri;
|
|
||||||
use tungstenite::protocol::frame::coding::CloseCode;
|
|
||||||
use tungstenite::stream::MaybeTlsStream;
|
|
||||||
use tungstenite::{Message, WebSocket};
|
|
||||||
|
|
||||||
use super::book_thread;
|
|
||||||
use super::wire::{self, Arrival, Received};
|
|
||||||
|
|
||||||
pub const MARKET_CHANNEL_URL: &str = "wss://ws-subscriptions-clob.polymarket.com/ws/market";
|
|
||||||
/// For the TCP connect and the TLS + websocket handshake, and for every send.
|
|
||||||
const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
|
|
||||||
/// How often the loop wakes when no frame comes: tungstenite has no `split`,
|
|
||||||
/// so the reads and the writes below (PING, commands, close) share this thread.
|
|
||||||
const READ_TIMEOUT: Duration = Duration::from_millis(200);
|
|
||||||
const PING_EVERY: Duration = Duration::from_secs(10);
|
|
||||||
const RECONNECT_AFTER: Duration = Duration::from_secs(1);
|
|
||||||
const CLOSE_TIMEOUT: Duration = Duration::from_secs(2);
|
|
||||||
/// Nothing received this long, not even the answer to three keepalives: the
|
|
||||||
/// connection is silent, and reconnects.
|
|
||||||
pub const SILENT_AFTER: Duration = Duration::from_secs(30);
|
|
||||||
/// Tokens per subscribe or unsubscribe message: a first message of more than
|
|
||||||
/// 500 gets no opening books (docs/notes.md).
|
|
||||||
const CHUNK: usize = 500;
|
|
||||||
|
|
||||||
/// What a book thread asks a connection for.
|
|
||||||
#[derive(Debug, PartialEq, Eq)]
|
|
||||||
pub enum Command {
|
|
||||||
Subscribe(Vec<String>),
|
|
||||||
Unsubscribe(Vec<String>),
|
|
||||||
/// Unsubscribe and subscribe again: the server then sends a fresh book.
|
|
||||||
/// A resync asks for that book; a quiet subscription only for the new
|
|
||||||
/// subscription.
|
|
||||||
Resubscribe(String),
|
|
||||||
/// Close and reconnect, subscribing every token again.
|
|
||||||
Recycle,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// What a connection tells the orchestrator.
|
|
||||||
#[derive(Debug, PartialEq, Eq)]
|
|
||||||
pub enum Notice {
|
|
||||||
/// Connected, its tokens subscribed.
|
|
||||||
Up(usize),
|
|
||||||
Down(usize),
|
|
||||||
/// Closed by the server with 1013: it couldn't send to us fast enough.
|
|
||||||
TooSlow(usize),
|
|
||||||
/// Closed by us: nothing came for [`SILENT_AFTER`].
|
|
||||||
Silent(usize),
|
|
||||||
}
|
|
||||||
|
|
||||||
type Socket = WebSocket<MaybeTlsStream<TcpStream>>;
|
|
||||||
|
|
||||||
/// Runs until `stop` is set or a book thread is gone, reconnecting (and
|
|
||||||
/// subscribing its tokens again) whenever the connection drops, falls silent
|
|
||||||
/// for `silent_after` or is recycled, and telling every book thread and the
|
|
||||||
/// orchestrator (`notices`) when it comes up and goes down. `book_threads`
|
|
||||||
/// holds every book thread's channel, by number.
|
|
||||||
pub fn run(
|
|
||||||
url: &str,
|
|
||||||
number: usize,
|
|
||||||
book_threads: &[Sender<Received>],
|
|
||||||
commands: &Receiver<Command>,
|
|
||||||
notices: &Sender<Notice>,
|
|
||||||
silent_after: Duration,
|
|
||||||
stop: &AtomicBool,
|
|
||||||
) {
|
|
||||||
let tell = |received: fn(usize) -> Received, notice: fn(usize) -> Notice| {
|
|
||||||
for book_thread in book_threads {
|
|
||||||
_ = book_thread.send(received(number));
|
|
||||||
}
|
|
||||||
_ = notices.send(notice(number));
|
|
||||||
};
|
|
||||||
let mut tokens = BTreeSet::new();
|
|
||||||
while !stop.load(Ordering::Relaxed) {
|
|
||||||
// Asked for while disconnected: the subscribe on connect covers them,
|
|
||||||
// and its books are fresh, so resyncs have nothing left to do.
|
|
||||||
for command in commands.try_iter() {
|
|
||||||
_ = apply(&mut tokens, &command);
|
|
||||||
}
|
|
||||||
match connect(url, &tokens) {
|
|
||||||
Ok(mut socket) => {
|
|
||||||
log!(
|
|
||||||
"polymarket: connection {number}: connected, {} symbols",
|
|
||||||
tokens.len()
|
|
||||||
);
|
|
||||||
tell(Received::Up, Notice::Up);
|
|
||||||
let ended = read(
|
|
||||||
&mut socket,
|
|
||||||
number,
|
|
||||||
&mut tokens,
|
|
||||||
book_threads,
|
|
||||||
commands,
|
|
||||||
notices,
|
|
||||||
silent_after,
|
|
||||||
stop,
|
|
||||||
);
|
|
||||||
if !matches!(ended, Ok(Ended::Stopped)) {
|
|
||||||
tell(Received::Down, Notice::Down);
|
|
||||||
}
|
|
||||||
match ended {
|
|
||||||
Ok(Ended::Stopped) => return close(&mut socket, number, book_threads),
|
|
||||||
Ok(Ended::Recycled) => log!("polymarket: connection {number}: recycled"),
|
|
||||||
Ok(Ended::BookThreadGone) => {
|
|
||||||
// Only a panic ends a book thread before us: its
|
|
||||||
// markets would go unserved, so the run stops.
|
|
||||||
log!("polymarket: connection {number}: a book thread is gone, stopping");
|
|
||||||
return stop.store(true, Ordering::Relaxed);
|
|
||||||
}
|
|
||||||
Err(error) => {
|
|
||||||
log!("polymarket: connection {number}: connection lost: {error}")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Err(error) => log!("polymarket: connection {number}: connect failed: {error}"),
|
|
||||||
}
|
|
||||||
thread::sleep(RECONNECT_AFTER);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
enum Ended {
|
|
||||||
Stopped,
|
|
||||||
BookThreadGone,
|
|
||||||
Recycled,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Updates the token set, and returns the messages that carry `command` out
|
|
||||||
/// on an open connection: none for a resync of a token not in the set.
|
|
||||||
#[must_use]
|
|
||||||
fn apply(tokens: &mut BTreeSet<String>, command: &Command) -> Vec<String> {
|
|
||||||
match command {
|
|
||||||
Command::Subscribe(added) => {
|
|
||||||
tokens.extend(added.iter().cloned());
|
|
||||||
operations("subscribe", added)
|
|
||||||
}
|
|
||||||
Command::Unsubscribe(removed) => {
|
|
||||||
removed.iter().for_each(|token| _ = tokens.remove(token));
|
|
||||||
operations("unsubscribe", removed)
|
|
||||||
}
|
|
||||||
// Subscribing a token already subscribed sends nothing back but
|
|
||||||
// `NO NEW ASSETS`: unsubscribing first makes the server send a fresh
|
|
||||||
// book.
|
|
||||||
Command::Resubscribe(token) if tokens.contains(token) => [
|
|
||||||
operations("unsubscribe", [token]),
|
|
||||||
operations("subscribe", [token]),
|
|
||||||
]
|
|
||||||
.concat(),
|
|
||||||
Command::Resubscribe(_) => Vec::new(),
|
|
||||||
// The read loop acts on it; disconnected, the reconnect does.
|
|
||||||
Command::Recycle => Vec::new(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The messages that subscribe or unsubscribe `tokens` on an open connection.
|
|
||||||
fn operations<'a>(operation: &str, tokens: impl IntoIterator<Item = &'a String>) -> Vec<String> {
|
|
||||||
let tokens: Vec<_> = tokens.into_iter().collect();
|
|
||||||
tokens
|
|
||||||
.chunks(CHUNK)
|
|
||||||
.map(|chunk| json!({"assets_ids": chunk, "operation": operation}).to_string())
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
fn connect(url: &str, tokens: &BTreeSet<String>) -> anyhow::Result<Socket> {
|
|
||||||
let uri: Uri = url.parse()?;
|
|
||||||
let host = uri
|
|
||||||
.host()
|
|
||||||
.ok_or_else(|| anyhow::anyhow!("{url} has no host"))?;
|
|
||||||
// The market channel is `wss://` on TLS's port; the tests name theirs.
|
|
||||||
let port = uri.port_u16().unwrap_or(443);
|
|
||||||
// Timed, unlike `tungstenite::connect`, so a stalled connect or handshake
|
|
||||||
// can't keep SIGINT waiting. The name lookup itself has no timeout.
|
|
||||||
let address = (host, port)
|
|
||||||
.to_socket_addrs()?
|
|
||||||
.next()
|
|
||||||
.ok_or_else(|| anyhow::anyhow!("{host} has no address"))?;
|
|
||||||
let stream = TcpStream::connect_timeout(&address, CONNECT_TIMEOUT)?;
|
|
||||||
stream.set_read_timeout(Some(CONNECT_TIMEOUT))?;
|
|
||||||
stream.set_write_timeout(Some(CONNECT_TIMEOUT))?;
|
|
||||||
let (mut socket, _response) = tungstenite::client_tls(url, stream)
|
|
||||||
.map_err(|error| anyhow::anyhow!("handshake failed: {error}"))?;
|
|
||||||
let stream = match socket.get_ref() {
|
|
||||||
MaybeTlsStream::Plain(stream) => stream,
|
|
||||||
MaybeTlsStream::Rustls(stream) => stream.get_ref(),
|
|
||||||
_ => anyhow::bail!("unexpected TLS stream"),
|
|
||||||
};
|
|
||||||
stream.set_read_timeout(Some(READ_TIMEOUT))?;
|
|
||||||
// Opened empty, then subscribed in chunks: one rule for every token.
|
|
||||||
socket.send(Message::text(
|
|
||||||
json!({"type": "market", "assets_ids": []}).to_string(),
|
|
||||||
))?;
|
|
||||||
for message in operations("subscribe", tokens) {
|
|
||||||
socket.send(Message::text(message))?;
|
|
||||||
}
|
|
||||||
Ok(socket)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Decodes a frame and sends each event to the book thread of its market;
|
|
||||||
/// `false` once a book thread is gone.
|
|
||||||
fn route(
|
|
||||||
received_at: DateTime<Utc>,
|
|
||||||
text: &str,
|
|
||||||
number: usize,
|
|
||||||
book_threads: &[Sender<Received>],
|
|
||||||
) -> bool {
|
|
||||||
let events = match wire::decode(text) {
|
|
||||||
Ok(events) => events,
|
|
||||||
Err(error) => {
|
|
||||||
log!("polymarket: unreadable frame ({error}): {}", truncate(text));
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
let bytes = (text.len() / events.len().max(1)) as u64;
|
|
||||||
for event in events {
|
|
||||||
// An event of no market (none we ask for) has no book thread.
|
|
||||||
let Some(thread) = event.market().map(book_thread) else {
|
|
||||||
continue;
|
|
||||||
};
|
|
||||||
let arrival = Arrival {
|
|
||||||
received_at,
|
|
||||||
connection: number,
|
|
||||||
bytes,
|
|
||||||
event,
|
|
||||||
};
|
|
||||||
if book_threads[thread].send(Received::Event(arrival)).is_err() {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
true
|
|
||||||
}
|
|
||||||
|
|
||||||
fn truncate(text: &str) -> &str {
|
|
||||||
text.char_indices()
|
|
||||||
.nth(200)
|
|
||||||
.map_or(text, |(end, _)| &text[..end])
|
|
||||||
}
|
|
||||||
|
|
||||||
// The connection's whole context, each a different part of it.
|
|
||||||
#[allow(clippy::too_many_arguments)]
|
|
||||||
fn read(
|
|
||||||
socket: &mut Socket,
|
|
||||||
number: usize,
|
|
||||||
tokens: &mut BTreeSet<String>,
|
|
||||||
book_threads: &[Sender<Received>],
|
|
||||||
commands: &Receiver<Command>,
|
|
||||||
notices: &Sender<Notice>,
|
|
||||||
silent_after: Duration,
|
|
||||||
stop: &AtomicBool,
|
|
||||||
) -> anyhow::Result<Ended> {
|
|
||||||
let mut last_ping = Instant::now();
|
|
||||||
let mut last_heard = Instant::now();
|
|
||||||
loop {
|
|
||||||
let message = socket.read();
|
|
||||||
// Stamp as soon as the frame is off the wire, before any processing.
|
|
||||||
let received_at = Utc::now();
|
|
||||||
if message.is_ok() {
|
|
||||||
last_heard = Instant::now();
|
|
||||||
}
|
|
||||||
match message {
|
|
||||||
Ok(Message::Text(text)) => {
|
|
||||||
if !route(received_at, text.as_str(), number, book_threads) {
|
|
||||||
return Ok(Ended::BookThreadGone);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Ok(Message::Close(frame)) => {
|
|
||||||
// 1013: the server couldn't send to us fast enough. The
|
|
||||||
// orchestrator splits a connection closed so twice.
|
|
||||||
if frame
|
|
||||||
.as_ref()
|
|
||||||
.is_some_and(|frame| frame.code == CloseCode::Again)
|
|
||||||
{
|
|
||||||
_ = notices.send(Notice::TooSlow(number));
|
|
||||||
}
|
|
||||||
anyhow::bail!("closed by the server: {frame:?}")
|
|
||||||
}
|
|
||||||
Ok(_) => {}
|
|
||||||
Err(tungstenite::Error::Io(error)) if timed_out(&error) => {}
|
|
||||||
Err(error) => return Err(error.into()),
|
|
||||||
}
|
|
||||||
|
|
||||||
if stop.load(Ordering::Relaxed) {
|
|
||||||
return Ok(Ended::Stopped);
|
|
||||||
}
|
|
||||||
if last_heard.elapsed() >= silent_after {
|
|
||||||
_ = notices.send(Notice::Silent(number));
|
|
||||||
anyhow::bail!("silent for {} s", silent_after.as_secs_f64());
|
|
||||||
}
|
|
||||||
if last_ping.elapsed() >= PING_EVERY {
|
|
||||||
socket.send(Message::text("PING"))?;
|
|
||||||
last_ping = Instant::now();
|
|
||||||
}
|
|
||||||
for command in commands.try_iter() {
|
|
||||||
if command == Command::Recycle {
|
|
||||||
// Not waited on: whatever still comes would arrive from a
|
|
||||||
// connection already down for the book threads. The commands
|
|
||||||
// behind it are applied before the reconnect.
|
|
||||||
_ = socket.close(None);
|
|
||||||
_ = socket.flush();
|
|
||||||
return Ok(Ended::Recycled);
|
|
||||||
}
|
|
||||||
for message in apply(tokens, &command) {
|
|
||||||
socket.send(Message::text(message))?;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Closes the connection, forwarding what still arrives until the server
|
|
||||||
/// confirms or [`CLOSE_TIMEOUT`] passes.
|
|
||||||
fn close(socket: &mut Socket, number: usize, book_threads: &[Sender<Received>]) {
|
|
||||||
if let Err(error) = socket.close(None) {
|
|
||||||
return log!("polymarket: connection {number}: close failed: {error}");
|
|
||||||
}
|
|
||||||
let deadline = Instant::now() + CLOSE_TIMEOUT;
|
|
||||||
while Instant::now() < deadline {
|
|
||||||
let message = socket.read();
|
|
||||||
let received_at = Utc::now();
|
|
||||||
match message {
|
|
||||||
Ok(Message::Text(text)) => _ = route(received_at, text.as_str(), number, book_threads),
|
|
||||||
Ok(_) => {}
|
|
||||||
Err(tungstenite::Error::Io(error)) if timed_out(&error) => {}
|
|
||||||
// ConnectionClosed: the close handshake is done.
|
|
||||||
Err(_) => break,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A read timeout shows up as `WouldBlock` on macOS and `TimedOut` elsewhere;
|
|
||||||
/// tungstenite keeps a partly read frame, so the next read resumes it.
|
|
||||||
fn timed_out(error: &std::io::Error) -> bool {
|
|
||||||
matches!(error.kind(), ErrorKind::WouldBlock | ErrorKind::TimedOut)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use std::net::TcpListener;
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
use crossbeam::channel::unbounded;
|
|
||||||
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
/// The tokens each message of one connection subscribed, after the first.
|
|
||||||
fn subscribed(socket: &mut WebSocket<TcpStream>, tokens: usize) -> Vec<Vec<String>> {
|
|
||||||
let first = socket.read().unwrap().into_text().unwrap();
|
|
||||||
let first: serde_json::Value = serde_json::from_str(&first).unwrap();
|
|
||||||
assert_eq!(first, json!({"type": "market", "assets_ids": []}));
|
|
||||||
let mut chunks = Vec::new();
|
|
||||||
while chunks.iter().map(Vec::len).sum::<usize>() < tokens {
|
|
||||||
let text = socket.read().unwrap().into_text().unwrap();
|
|
||||||
let message: serde_json::Value = serde_json::from_str(&text).unwrap();
|
|
||||||
assert_eq!(message["operation"], "subscribe");
|
|
||||||
chunks.push(serde_json::from_value(message["assets_ids"].clone()).unwrap());
|
|
||||||
}
|
|
||||||
chunks
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_reconnect_subscribes_every_token_again_in_chunks() {
|
|
||||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
|
||||||
let url = format!("ws://{}/ws/market", listener.local_addr().unwrap());
|
|
||||||
let tokens: Vec<String> = (0..1_200).map(|token| token.to_string()).collect();
|
|
||||||
let total = tokens.len();
|
|
||||||
let server = thread::spawn(move || {
|
|
||||||
let mut connections = Vec::new();
|
|
||||||
for _ in 0..2 {
|
|
||||||
let (stream, _) = listener.accept().unwrap();
|
|
||||||
// A client that never sends fails the test rather than hangs it.
|
|
||||||
stream
|
|
||||||
.set_read_timeout(Some(Duration::from_secs(5)))
|
|
||||||
.unwrap();
|
|
||||||
let mut socket = tungstenite::accept(stream).unwrap();
|
|
||||||
connections.push(subscribed(&mut socket, total));
|
|
||||||
// The client reconnects after this close; after the second,
|
|
||||||
// it finds no server and stops.
|
|
||||||
socket.close(None).unwrap();
|
|
||||||
while socket.read().is_ok() {}
|
|
||||||
}
|
|
||||||
connections
|
|
||||||
});
|
|
||||||
|
|
||||||
let (events_tx, _events_rx) = unbounded();
|
|
||||||
let (commands_tx, commands_rx) = unbounded();
|
|
||||||
commands_tx
|
|
||||||
.send(Command::Subscribe(tokens.clone()))
|
|
||||||
.unwrap();
|
|
||||||
let stop = Arc::new(AtomicBool::new(false));
|
|
||||||
let client_stop = Arc::clone(&stop);
|
|
||||||
let (notices, _notices_rx) = unbounded();
|
|
||||||
let client = thread::spawn(move || {
|
|
||||||
run(
|
|
||||||
&url,
|
|
||||||
0,
|
|
||||||
&[events_tx],
|
|
||||||
&commands_rx,
|
|
||||||
¬ices,
|
|
||||||
SILENT_AFTER,
|
|
||||||
&client_stop,
|
|
||||||
)
|
|
||||||
});
|
|
||||||
|
|
||||||
let connections = server.join().unwrap();
|
|
||||||
stop.store(true, Ordering::Relaxed);
|
|
||||||
client.join().unwrap();
|
|
||||||
let mut tokens = tokens;
|
|
||||||
tokens.sort();
|
|
||||||
for chunks in connections {
|
|
||||||
assert!(chunks.iter().all(|chunk| chunk.len() <= CHUNK));
|
|
||||||
let mut subscribed = chunks.concat();
|
|
||||||
subscribed.sort();
|
|
||||||
assert_eq!(subscribed, tokens);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A local server that accepts two connections of `total` tokens, each
|
|
||||||
/// handled by `serve` once subscribed, and answers what the client
|
|
||||||
/// subscribed on each.
|
|
||||||
fn serving_twice(
|
|
||||||
total: usize,
|
|
||||||
serve: fn(&mut WebSocket<TcpStream>),
|
|
||||||
) -> (String, thread::JoinHandle<Vec<Vec<Vec<String>>>>) {
|
|
||||||
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
|
||||||
let url = format!("ws://{}/ws/market", listener.local_addr().unwrap());
|
|
||||||
let server = thread::spawn(move || {
|
|
||||||
(0..2)
|
|
||||||
.map(|_| {
|
|
||||||
let (stream, _) = listener.accept().unwrap();
|
|
||||||
stream
|
|
||||||
.set_read_timeout(Some(Duration::from_secs(5)))
|
|
||||||
.unwrap();
|
|
||||||
let mut socket = tungstenite::accept(stream).unwrap();
|
|
||||||
let chunks = subscribed(&mut socket, total);
|
|
||||||
serve(&mut socket);
|
|
||||||
chunks
|
|
||||||
})
|
|
||||||
.collect()
|
|
||||||
});
|
|
||||||
(url, server)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Runs a connection of `tokens` against `url` until `until` returns,
|
|
||||||
/// then stops it: the notices it sent.
|
|
||||||
fn client(
|
|
||||||
url: String,
|
|
||||||
tokens: Vec<String>,
|
|
||||||
silent_after: Duration,
|
|
||||||
until: impl FnOnce(&Sender<Command>),
|
|
||||||
) -> Vec<Notice> {
|
|
||||||
let (events_tx, _events_rx) = unbounded();
|
|
||||||
let (commands_tx, commands_rx) = unbounded();
|
|
||||||
commands_tx.send(Command::Subscribe(tokens)).unwrap();
|
|
||||||
let stop = Arc::new(AtomicBool::new(false));
|
|
||||||
let client_stop = Arc::clone(&stop);
|
|
||||||
let (notices, notices_rx) = unbounded();
|
|
||||||
let client = thread::spawn(move || {
|
|
||||||
run(
|
|
||||||
&url,
|
|
||||||
0,
|
|
||||||
&[events_tx],
|
|
||||||
&commands_rx,
|
|
||||||
¬ices,
|
|
||||||
silent_after,
|
|
||||||
&client_stop,
|
|
||||||
)
|
|
||||||
});
|
|
||||||
until(&commands_tx);
|
|
||||||
stop.store(true, Ordering::Relaxed);
|
|
||||||
client.join().unwrap();
|
|
||||||
notices_rx.try_iter().collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_silent_connection_reconnects_with_its_tokens() {
|
|
||||||
let tokens: Vec<String> = (0..10).map(|token| token.to_string()).collect();
|
|
||||||
// Silent: reads what comes (the keepalives) and answers nothing,
|
|
||||||
// until the client closes.
|
|
||||||
let (url, server) = serving_twice(tokens.len(), |socket| while socket.read().is_ok() {});
|
|
||||||
let notices = client(url, tokens.clone(), Duration::from_millis(500), |_| {
|
|
||||||
thread::sleep(Duration::from_millis(2_500));
|
|
||||||
});
|
|
||||||
let connections = server.join().unwrap();
|
|
||||||
assert_eq!(connections.len(), 2);
|
|
||||||
assert!(connections.iter().all(|chunks| chunks.concat() == tokens));
|
|
||||||
assert_eq!(
|
|
||||||
notices[..3],
|
|
||||||
[Notice::Up(0), Notice::Silent(0), Notice::Down(0)]
|
|
||||||
);
|
|
||||||
assert_eq!(notices[3], Notice::Up(0));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn a_recycle_closes_and_subscribes_every_token_again() {
|
|
||||||
let tokens: Vec<String> = (0..10).map(|token| token.to_string()).collect();
|
|
||||||
// Answers the keepalives, as the exchange does, until closed.
|
|
||||||
let (url, server) = serving_twice(tokens.len(), |socket| {
|
|
||||||
while let Ok(message) = socket.read() {
|
|
||||||
if message.is_text() {
|
|
||||||
_ = socket.send(Message::text("PONG"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
});
|
|
||||||
let notices = client(url, tokens.clone(), SILENT_AFTER, |commands| {
|
|
||||||
thread::sleep(Duration::from_millis(300));
|
|
||||||
commands.send(Command::Recycle).unwrap();
|
|
||||||
thread::sleep(Duration::from_millis(1_800));
|
|
||||||
});
|
|
||||||
let connections = server.join().unwrap();
|
|
||||||
assert!(connections.iter().all(|chunks| chunks.concat() == tokens));
|
|
||||||
assert_eq!(notices, [Notice::Up(0), Notice::Down(0), Notice::Up(0)]);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The recording's first frame holds the opening books of 40 tokens, 20
|
|
||||||
/// markets: each book goes to its market's book thread, with its share of
|
|
||||||
/// the frame's bytes.
|
|
||||||
#[test]
|
|
||||||
fn a_frame_of_several_markets_reaches_each_markets_book_thread() {
|
|
||||||
#[derive(serde::Deserialize)]
|
|
||||||
struct Line {
|
|
||||||
text: String,
|
|
||||||
}
|
|
||||||
let path = concat!(
|
|
||||||
env!("CARGO_MANIFEST_DIR"),
|
|
||||||
"/tests/fixtures/polymarket-frames.jsonl"
|
|
||||||
);
|
|
||||||
let recording = std::fs::read_to_string(path).unwrap();
|
|
||||||
let line: Line = serde_json::from_str(recording.lines().next().unwrap()).unwrap();
|
|
||||||
let (senders, receivers): (Vec<_>, Vec<_>) = (0..crate::polymarket::BOOK_THREADS)
|
|
||||||
.map(|_| unbounded())
|
|
||||||
.unzip();
|
|
||||||
|
|
||||||
assert!(route(Utc::now(), &line.text, 3, &senders));
|
|
||||||
let mut books = 0;
|
|
||||||
for (thread, receiver) in receivers.iter().enumerate() {
|
|
||||||
for received in receiver.try_iter() {
|
|
||||||
let Received::Event(arrival) = received else {
|
|
||||||
panic!("not an event: {received:?}");
|
|
||||||
};
|
|
||||||
let market = arrival.event.market().unwrap().to_string();
|
|
||||||
assert_eq!(book_thread(&market), thread);
|
|
||||||
assert_eq!(arrival.connection, 3);
|
|
||||||
assert_eq!(arrival.bytes, line.text.len() as u64 / 40);
|
|
||||||
books += 1;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert_eq!(books, 40);
|
|
||||||
// Neither a keepalive's answer nor an unreadable frame reaches one.
|
|
||||||
assert!(route(Utc::now(), "PONG", 3, &senders));
|
|
||||||
assert!(route(Utc::now(), "not json", 3, &senders));
|
|
||||||
assert!(receivers.iter().all(|receiver| receiver.is_empty()));
|
|
||||||
// A book thread gone stops the connection.
|
|
||||||
drop(receivers);
|
|
||||||
assert!(!route(Utc::now(), &line.text, 3, &senders));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn commands_update_the_token_set_and_say_what_to_send() {
|
|
||||||
let mut tokens = BTreeSet::new();
|
|
||||||
let operation = |message: &String| -> (String, Vec<String>) {
|
|
||||||
let message: serde_json::Value = serde_json::from_str(message).unwrap();
|
|
||||||
let tokens = serde_json::from_value(message["assets_ids"].clone()).unwrap();
|
|
||||||
(message["operation"].as_str().unwrap().to_string(), tokens)
|
|
||||||
};
|
|
||||||
let sent = |messages: Vec<String>| messages.iter().map(operation).collect::<Vec<_>>();
|
|
||||||
let op = |name: &str, tokens: &[&str]| {
|
|
||||||
(
|
|
||||||
name.to_string(),
|
|
||||||
tokens.iter().map(|token| token.to_string()).collect(),
|
|
||||||
)
|
|
||||||
};
|
|
||||||
|
|
||||||
let subscribe = Command::Subscribe(vec!["a".into(), "b".into()]);
|
|
||||||
assert_eq!(
|
|
||||||
sent(apply(&mut tokens, &subscribe)),
|
|
||||||
[op("subscribe", &["a", "b"])]
|
|
||||||
);
|
|
||||||
assert_eq!(
|
|
||||||
sent(apply(&mut tokens, &Command::Resubscribe("a".into()))),
|
|
||||||
[op("unsubscribe", &["a"]), op("subscribe", &["a"])]
|
|
||||||
);
|
|
||||||
let unsubscribe = Command::Unsubscribe(vec!["a".into()]);
|
|
||||||
assert_eq!(
|
|
||||||
sent(apply(&mut tokens, &unsubscribe)),
|
|
||||||
[op("unsubscribe", &["a"])]
|
|
||||||
);
|
|
||||||
assert!(apply(&mut tokens, &Command::Resubscribe("a".into())).is_empty());
|
|
||||||
assert_eq!(tokens.into_iter().collect::<Vec<_>>(), ["b"]);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,272 @@
|
|||||||
|
//! The connection loop (`src/core/socket.rs`) against a local server, with
|
||||||
|
//! Polymarket's codec.
|
||||||
|
|
||||||
|
use std::collections::BTreeSet;
|
||||||
|
use std::net::{TcpListener, TcpStream};
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
use std::thread;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use chrono::Utc;
|
||||||
|
use crossbeam::channel::{Sender, unbounded};
|
||||||
|
use serde_json::json;
|
||||||
|
use tungstenite::{Message, WebSocket};
|
||||||
|
|
||||||
|
use super::{CHUNK, Polymarket, Session};
|
||||||
|
use crate::core::Event;
|
||||||
|
use crate::core::socket::{Command, Notice, Routed, SILENT_AFTER, apply, route, run};
|
||||||
|
use crate::core::{BOOK_THREADS, Received, book_thread};
|
||||||
|
|
||||||
|
/// The tokens each message of one connection subscribed, after the first.
|
||||||
|
fn subscribed(socket: &mut WebSocket<TcpStream>, tokens: usize) -> Vec<Vec<String>> {
|
||||||
|
let first = socket.read().unwrap().into_text().unwrap();
|
||||||
|
let first: serde_json::Value = serde_json::from_str(&first).unwrap();
|
||||||
|
assert_eq!(first, json!({"type": "market", "assets_ids": []}));
|
||||||
|
let mut chunks = Vec::new();
|
||||||
|
while chunks.iter().map(Vec::len).sum::<usize>() < tokens {
|
||||||
|
let text = socket.read().unwrap().into_text().unwrap();
|
||||||
|
let message: serde_json::Value = serde_json::from_str(&text).unwrap();
|
||||||
|
assert_eq!(message["operation"], "subscribe");
|
||||||
|
chunks.push(serde_json::from_value(message["assets_ids"].clone()).unwrap());
|
||||||
|
}
|
||||||
|
chunks
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_reconnect_subscribes_every_token_again_in_chunks() {
|
||||||
|
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||||
|
let url = format!("ws://{}/ws/market", listener.local_addr().unwrap());
|
||||||
|
let tokens: Vec<String> = (0..1_200).map(|token| token.to_string()).collect();
|
||||||
|
let total = tokens.len();
|
||||||
|
let server = thread::spawn(move || {
|
||||||
|
let mut connections = Vec::new();
|
||||||
|
for _ in 0..2 {
|
||||||
|
let (stream, _) = listener.accept().unwrap();
|
||||||
|
// A client that never sends fails the test rather than hangs it.
|
||||||
|
stream
|
||||||
|
.set_read_timeout(Some(Duration::from_secs(5)))
|
||||||
|
.unwrap();
|
||||||
|
let mut socket = tungstenite::accept(stream).unwrap();
|
||||||
|
connections.push(subscribed(&mut socket, total));
|
||||||
|
// The client reconnects after this close; after the second,
|
||||||
|
// it finds no server and stops.
|
||||||
|
socket.close(None).unwrap();
|
||||||
|
while socket.read().is_ok() {}
|
||||||
|
}
|
||||||
|
connections
|
||||||
|
});
|
||||||
|
|
||||||
|
let (events_tx, _events_rx) = unbounded();
|
||||||
|
let (commands_tx, commands_rx) = unbounded();
|
||||||
|
commands_tx
|
||||||
|
.send(Command::Subscribe(tokens.clone()))
|
||||||
|
.unwrap();
|
||||||
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
|
let client_stop = Arc::clone(&stop);
|
||||||
|
let (notices, _notices_rx) = unbounded();
|
||||||
|
let client = thread::spawn(move || {
|
||||||
|
run(
|
||||||
|
&Polymarket { url },
|
||||||
|
0,
|
||||||
|
&[events_tx],
|
||||||
|
&commands_rx,
|
||||||
|
¬ices,
|
||||||
|
SILENT_AFTER,
|
||||||
|
&client_stop,
|
||||||
|
)
|
||||||
|
});
|
||||||
|
|
||||||
|
let connections = server.join().unwrap();
|
||||||
|
stop.store(true, Ordering::Relaxed);
|
||||||
|
client.join().unwrap();
|
||||||
|
let mut tokens = tokens;
|
||||||
|
tokens.sort();
|
||||||
|
for chunks in connections {
|
||||||
|
assert!(chunks.iter().all(|chunk| chunk.len() <= CHUNK));
|
||||||
|
let mut subscribed = chunks.concat();
|
||||||
|
subscribed.sort();
|
||||||
|
assert_eq!(subscribed, tokens);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A local server that accepts two connections of `total` tokens, each
|
||||||
|
/// handled by `serve` once subscribed, and answers what the client
|
||||||
|
/// subscribed on each.
|
||||||
|
fn serving_twice(
|
||||||
|
total: usize,
|
||||||
|
serve: fn(&mut WebSocket<TcpStream>),
|
||||||
|
) -> (String, thread::JoinHandle<Vec<Vec<Vec<String>>>>) {
|
||||||
|
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
|
||||||
|
let url = format!("ws://{}/ws/market", listener.local_addr().unwrap());
|
||||||
|
let server = thread::spawn(move || {
|
||||||
|
(0..2)
|
||||||
|
.map(|_| {
|
||||||
|
let (stream, _) = listener.accept().unwrap();
|
||||||
|
stream
|
||||||
|
.set_read_timeout(Some(Duration::from_secs(5)))
|
||||||
|
.unwrap();
|
||||||
|
let mut socket = tungstenite::accept(stream).unwrap();
|
||||||
|
let chunks = subscribed(&mut socket, total);
|
||||||
|
serve(&mut socket);
|
||||||
|
chunks
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
});
|
||||||
|
(url, server)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Runs a connection of `tokens` against `url` until `until` returns,
|
||||||
|
/// then stops it: the notices it sent.
|
||||||
|
fn client(
|
||||||
|
url: String,
|
||||||
|
tokens: Vec<String>,
|
||||||
|
silent_after: Duration,
|
||||||
|
until: impl FnOnce(&Sender<Command>),
|
||||||
|
) -> Vec<Notice> {
|
||||||
|
let (events_tx, _events_rx) = unbounded();
|
||||||
|
let (commands_tx, commands_rx) = unbounded();
|
||||||
|
commands_tx.send(Command::Subscribe(tokens)).unwrap();
|
||||||
|
let stop = Arc::new(AtomicBool::new(false));
|
||||||
|
let client_stop = Arc::clone(&stop);
|
||||||
|
let (notices, notices_rx) = unbounded();
|
||||||
|
let client = thread::spawn(move || {
|
||||||
|
run(
|
||||||
|
&Polymarket { url },
|
||||||
|
0,
|
||||||
|
&[events_tx],
|
||||||
|
&commands_rx,
|
||||||
|
¬ices,
|
||||||
|
silent_after,
|
||||||
|
&client_stop,
|
||||||
|
)
|
||||||
|
});
|
||||||
|
until(&commands_tx);
|
||||||
|
stop.store(true, Ordering::Relaxed);
|
||||||
|
client.join().unwrap();
|
||||||
|
notices_rx.try_iter().collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_silent_connection_reconnects_with_its_tokens() {
|
||||||
|
let tokens: Vec<String> = (0..10).map(|token| token.to_string()).collect();
|
||||||
|
// Silent: reads what comes (the keepalives) and answers nothing,
|
||||||
|
// until the client closes.
|
||||||
|
let (url, server) = serving_twice(tokens.len(), |socket| while socket.read().is_ok() {});
|
||||||
|
let notices = client(url, tokens.clone(), Duration::from_millis(500), |_| {
|
||||||
|
thread::sleep(Duration::from_millis(2_500));
|
||||||
|
});
|
||||||
|
let connections = server.join().unwrap();
|
||||||
|
assert_eq!(connections.len(), 2);
|
||||||
|
assert!(connections.iter().all(|chunks| chunks.concat() == tokens));
|
||||||
|
assert_eq!(
|
||||||
|
notices[..3],
|
||||||
|
[Notice::Up(0), Notice::Silent(0), Notice::Down(0)]
|
||||||
|
);
|
||||||
|
assert_eq!(notices[3], Notice::Up(0));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_recycle_closes_and_subscribes_every_token_again() {
|
||||||
|
let tokens: Vec<String> = (0..10).map(|token| token.to_string()).collect();
|
||||||
|
// Answers the keepalives, as the exchange does, until closed.
|
||||||
|
let (url, server) = serving_twice(tokens.len(), |socket| {
|
||||||
|
while let Ok(message) = socket.read() {
|
||||||
|
if message.is_text() {
|
||||||
|
_ = socket.send(Message::text("PONG"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
let notices = client(url, tokens.clone(), SILENT_AFTER, |commands| {
|
||||||
|
thread::sleep(Duration::from_millis(300));
|
||||||
|
commands.send(Command::Recycle).unwrap();
|
||||||
|
thread::sleep(Duration::from_millis(1_800));
|
||||||
|
});
|
||||||
|
let connections = server.join().unwrap();
|
||||||
|
assert!(connections.iter().all(|chunks| chunks.concat() == tokens));
|
||||||
|
assert_eq!(notices, [Notice::Up(0), Notice::Down(0), Notice::Up(0)]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The recording's first frame holds the opening books of 40 tokens, 20
|
||||||
|
/// markets: each book goes to its market's book thread, with its share of
|
||||||
|
/// the frame's bytes.
|
||||||
|
#[test]
|
||||||
|
fn a_frame_of_several_markets_reaches_each_markets_book_thread() {
|
||||||
|
#[derive(serde::Deserialize)]
|
||||||
|
struct Line {
|
||||||
|
text: String,
|
||||||
|
}
|
||||||
|
let path = concat!(
|
||||||
|
env!("CARGO_MANIFEST_DIR"),
|
||||||
|
"/tests/fixtures/polymarket-frames.jsonl"
|
||||||
|
);
|
||||||
|
let recording = std::fs::read_to_string(path).unwrap();
|
||||||
|
let line: Line = serde_json::from_str(recording.lines().next().unwrap()).unwrap();
|
||||||
|
let (senders, receivers): (Vec<_>, Vec<_>) = (0..BOOK_THREADS).map(|_| unbounded()).unzip();
|
||||||
|
let mut session = Session;
|
||||||
|
let mut routed = |text: &str, senders: &[Sender<Received<_>>]| {
|
||||||
|
route(&mut session, Utc::now(), text, 3, senders, "polymarket")
|
||||||
|
};
|
||||||
|
|
||||||
|
assert_eq!(routed(&line.text, &senders), Routed::Sent);
|
||||||
|
let mut books = 0;
|
||||||
|
for (thread, receiver) in receivers.iter().enumerate() {
|
||||||
|
for received in receiver.try_iter() {
|
||||||
|
let Received::Event(arrival) = received else {
|
||||||
|
panic!("not an event: {received:?}");
|
||||||
|
};
|
||||||
|
let market = arrival.event.market().unwrap().to_string();
|
||||||
|
assert_eq!(book_thread(&market), thread);
|
||||||
|
assert_eq!(arrival.connection, 3);
|
||||||
|
assert_eq!(arrival.bytes, line.text.len() as u64 / 40);
|
||||||
|
books += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert_eq!(books, 40);
|
||||||
|
// Neither a keepalive's answer nor an unreadable frame reaches one.
|
||||||
|
assert_eq!(routed("PONG", &senders), Routed::Sent);
|
||||||
|
assert_eq!(routed("not json", &senders), Routed::Sent);
|
||||||
|
assert!(receivers.iter().all(|receiver| receiver.is_empty()));
|
||||||
|
// A book thread gone stops the connection.
|
||||||
|
drop(receivers);
|
||||||
|
assert_eq!(routed(&line.text, &senders), Routed::BookThreadGone);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn commands_update_the_token_set_and_say_what_to_send() {
|
||||||
|
let mut tokens = BTreeSet::new();
|
||||||
|
let mut session = Session;
|
||||||
|
let operation = |message: &String| -> (String, Vec<String>) {
|
||||||
|
let message: serde_json::Value = serde_json::from_str(message).unwrap();
|
||||||
|
let tokens = serde_json::from_value(message["assets_ids"].clone()).unwrap();
|
||||||
|
(message["operation"].as_str().unwrap().to_string(), tokens)
|
||||||
|
};
|
||||||
|
let sent = |messages: Vec<String>| messages.iter().map(operation).collect::<Vec<_>>();
|
||||||
|
let op = |name: &str, tokens: &[&str]| {
|
||||||
|
(
|
||||||
|
name.to_string(),
|
||||||
|
tokens.iter().map(|token| token.to_string()).collect(),
|
||||||
|
)
|
||||||
|
};
|
||||||
|
|
||||||
|
let subscribe = Command::Subscribe(vec!["a".into(), "b".into()]);
|
||||||
|
assert_eq!(
|
||||||
|
sent(apply(&mut session, &mut tokens, &subscribe)),
|
||||||
|
[op("subscribe", &["a", "b"])]
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
sent(apply(
|
||||||
|
&mut session,
|
||||||
|
&mut tokens,
|
||||||
|
&Command::Resubscribe("a".into())
|
||||||
|
)),
|
||||||
|
[op("unsubscribe", &["a"]), op("subscribe", &["a"])]
|
||||||
|
);
|
||||||
|
let unsubscribe = Command::Unsubscribe(vec!["a".into()]);
|
||||||
|
assert_eq!(
|
||||||
|
sent(apply(&mut session, &mut tokens, &unsubscribe)),
|
||||||
|
[op("unsubscribe", &["a"])]
|
||||||
|
);
|
||||||
|
assert!(apply(&mut session, &mut tokens, &Command::Resubscribe("a".into())).is_empty());
|
||||||
|
assert_eq!(tokens.into_iter().collect::<Vec<_>>(), ["b"]);
|
||||||
|
}
|
||||||
+6
-30
@@ -1,35 +1,8 @@
|
|||||||
//! The market channel's messages, as the exchange writes them, and how a
|
//! The market channel's messages, as the exchange writes them, and how a
|
||||||
//! frame's text becomes events (docs/notes.md).
|
//! frame's text becomes events (docs/notes.md).
|
||||||
|
|
||||||
use chrono::{DateTime, Utc};
|
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
|
|
||||||
/// What a connection sends a book thread.
|
|
||||||
// Nearly every message is an event: boxing it to shrink the two rare markers
|
|
||||||
// would cost an allocation per event for nothing.
|
|
||||||
#[allow(clippy::large_enum_variant)]
|
|
||||||
#[derive(Debug)]
|
|
||||||
pub enum Received {
|
|
||||||
Event(Arrival),
|
|
||||||
/// The connection came up: its subscriptions are being sent.
|
|
||||||
Up(usize),
|
|
||||||
/// The connection went down: nothing more comes from it until it is up.
|
|
||||||
Down(usize),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// One event of a frame on its way to its market's book thread.
|
|
||||||
#[derive(Debug)]
|
|
||||||
pub struct Arrival {
|
|
||||||
/// The frame's receive time, stamped right after the socket read.
|
|
||||||
pub received_at: DateTime<Utc>,
|
|
||||||
/// The connection that delivered it.
|
|
||||||
pub connection: usize,
|
|
||||||
/// Its share of the frame's bytes: a frame is one market's change, or
|
|
||||||
/// books of several, each taking its share.
|
|
||||||
pub bytes: u64,
|
|
||||||
pub event: Event,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The events of a frame's text; none for the keepalive's `PONG` and for
|
/// The events of a frame's text; none for the keepalive's `PONG` and for
|
||||||
/// `NO NEW ASSETS`.
|
/// `NO NEW ASSETS`.
|
||||||
pub fn decode(text: &str) -> Result<Vec<Event>, serde_json::Error> {
|
pub fn decode(text: &str) -> Result<Vec<Event>, serde_json::Error> {
|
||||||
@@ -42,9 +15,8 @@ pub fn decode(text: &str) -> Result<Vec<Event>, serde_json::Error> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Event {
|
impl crate::core::Event for Event {
|
||||||
/// The market it belongs to, which decides its book thread.
|
fn market(&self) -> Option<&str> {
|
||||||
pub fn market(&self) -> Option<&str> {
|
|
||||||
match self {
|
match self {
|
||||||
Event::Book(book) => Some(&book.market),
|
Event::Book(book) => Some(&book.market),
|
||||||
Event::PriceChange(change) => Some(&change.market),
|
Event::PriceChange(change) => Some(&change.market),
|
||||||
@@ -54,6 +26,10 @@ impl Event {
|
|||||||
Event::Other => None,
|
Event::Other => None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn is_book(&self) -> bool {
|
||||||
|
matches!(self, Event::Book(_))
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Deserialize)]
|
#[derive(Debug, Deserialize)]
|
||||||
|
|||||||
Vendored
+15
-2
@@ -1,8 +1,10 @@
|
|||||||
# Test fixtures
|
# Test fixtures
|
||||||
|
|
||||||
Two recordings of Polymarket's market channel, replayed by the tests in
|
Two recordings of Polymarket's market channel, replayed by the tests in
|
||||||
`src/polymarket/feed.rs` (the first also by `socket.rs`'s routing test), and
|
`src/polymarket/feed.rs` (the first also by `socket_tests.rs`'s routing
|
||||||
Gamma pages read by `src/polymarket/gamma.rs` (no network in tests).
|
test), and Gamma pages read by `src/polymarket/gamma.rs`; one recording of
|
||||||
|
Kalshi's demo, replayed by `src/kalshi/feed.rs`, and listing pages read by
|
||||||
|
`src/kalshi/listing.rs` (no network in tests).
|
||||||
|
|
||||||
- `gamma-markets.json`: Gamma's top 20 markets by 24 h volume on 2026-11-04,
|
- `gamma-markets.json`: Gamma's top 20 markets by 24 h volume on 2026-11-04,
|
||||||
trimmed to the fields the collector reads. `conditionId`, `active`, `closed`
|
trimmed to the fields the collector reads. `conditionId`, `active`, `closed`
|
||||||
@@ -21,3 +23,14 @@ Gamma pages read by `src/polymarket/gamma.rs` (no network in tests).
|
|||||||
- `record.py`: the recorder, if the fixtures need redoing, on CONNECTIONS
|
- `record.py`: the recorder, if the fixtures need redoing, on CONNECTIONS
|
||||||
connections at once (default 1):
|
connections at once (default 1):
|
||||||
`uvx --with websockets python -I record.py gamma-markets.json frames.jsonl 90 [CONNECTIONS]`.
|
`uvx --with websockets python -I record.py gamma-markets.json frames.jsonl 90 [CONNECTIONS]`.
|
||||||
|
- `kalshi-frames-two.jsonl`: 90 s (2026-11-11, from 18:43:27 UTC) of the
|
||||||
|
demo's `orderbook_delta` channel on two connections, the 26 markets with
|
||||||
|
the most 24 h volume, in the same line shape. Connection 0 asked for every
|
||||||
|
book again every 3 s (`get_snapshot`, 780 books); connection 1 left five
|
||||||
|
markets at 60 s and took them again at 65 s. Recorded with the throwaway
|
||||||
|
recorder of sprint 06 (`../kalshi-spike/`, outside the repo).
|
||||||
|
- `kalshi-open.json`, `kalshi-unopened.json`, `kalshi-closed.json`: one page
|
||||||
|
each (`limit=3`, `mve_filter=exclude`) of production's listing by
|
||||||
|
`status=open`, `status=unopened` and `status=closed&min_close_ts`,
|
||||||
|
2026-11-11 ~19:35 UTC, trimmed to `ticker`, `status`, `created_time`,
|
||||||
|
`open_time`, `close_time`.
|
||||||
Vendored
+26
@@ -0,0 +1,26 @@
|
|||||||
|
{
|
||||||
|
"markets": [
|
||||||
|
{
|
||||||
|
"ticker": "KXFIBAECUPSPREAD-26NOV111230SZOZIG-SZO23",
|
||||||
|
"status": "determined",
|
||||||
|
"created_time": "2026-11-11T13:04:40.111474Z",
|
||||||
|
"open_time": "2026-11-11T13:06:00Z",
|
||||||
|
"close_time": "2026-11-11T19:12:27Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ticker": "KXFIBAECUPSPREAD-26NOV111230SZOZIG-SZO19",
|
||||||
|
"status": "determined",
|
||||||
|
"created_time": "2026-11-11T13:04:40.111474Z",
|
||||||
|
"open_time": "2026-11-11T13:06:00Z",
|
||||||
|
"close_time": "2026-11-11T19:12:27Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ticker": "KXFIBAECUPSPREAD-26NOV111230SZOZIG-SZO15",
|
||||||
|
"status": "determined",
|
||||||
|
"created_time": "2026-11-11T13:04:40.111474Z",
|
||||||
|
"open_time": "2026-11-11T13:06:00Z",
|
||||||
|
"close_time": "2026-11-11T19:12:27Z"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"cursor": "CgYIm_zS1wYSKEtYRklCQUVDVVBTUFJFQUQtMjZOT1YxMTEyMzBTWk9aSUctU1pPMTU"
|
||||||
|
}
|
||||||
+2852
File diff suppressed because it is too large.
Load diff
Vendored
+26
@@ -0,0 +1,26 @@
|
|||||||
|
{
|
||||||
|
"markets": [
|
||||||
|
{
|
||||||
|
"ticker": "KXNCAAWBTOTAL-26NOV11QUINSJU-106",
|
||||||
|
"status": "active",
|
||||||
|
"created_time": "2026-11-11T19:12:59.634595Z",
|
||||||
|
"open_time": "2026-11-11T19:14:00Z",
|
||||||
|
"close_time": "2026-11-13T19:00:00Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ticker": "KXNCAAWBTOTAL-26NOV11USAFAU-160",
|
||||||
|
"status": "active",
|
||||||
|
"created_time": "2026-11-11T19:12:39.6174Z",
|
||||||
|
"open_time": "2026-11-11T19:14:00Z",
|
||||||
|
"close_time": "2026-11-13T18:00:00Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ticker": "KXESOCCERGAME-26NOV111710EDERBONY-TIE",
|
||||||
|
"status": "active",
|
||||||
|
"created_time": "2026-11-11T19:11:51.921073Z",
|
||||||
|
"open_time": "2026-11-11T19:12:00Z",
|
||||||
|
"close_time": "2026-11-25T22:10:00Z"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"cursor": "CgwI9_vS1wYQ6OqZtwMSJUtYRVNPQ0NFUkdBTUUtMjZOT1YxMTE3MTBFREVSQk9OWS1USUU"
|
||||||
|
}
|
||||||
Vendored
+26
@@ -0,0 +1,26 @@
|
|||||||
|
{
|
||||||
|
"markets": [
|
||||||
|
{
|
||||||
|
"ticker": "KXEUROCUPTOTAL-26NOV121400MAXSKY-179",
|
||||||
|
"status": "initialized",
|
||||||
|
"created_time": "2026-11-11T19:14:08.970811Z",
|
||||||
|
"open_time": "2026-11-11T19:19:08Z",
|
||||||
|
"close_time": "2026-11-14T19:00:00Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ticker": "KXEUROCUPTOTAL-26NOV121400MAXSKY-171",
|
||||||
|
"status": "initialized",
|
||||||
|
"created_time": "2026-11-11T19:14:08.970811Z",
|
||||||
|
"open_time": "2026-11-11T19:19:08Z",
|
||||||
|
"close_time": "2026-11-14T19:00:00Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ticker": "KXEUROCUPTOTAL-26NOV121400MAXSKY-163",
|
||||||
|
"status": "initialized",
|
||||||
|
"created_time": "2026-11-11T19:14:08.970811Z",
|
||||||
|
"open_time": "2026-11-11T19:19:08Z",
|
||||||
|
"close_time": "2026-11-14T19:00:00Z"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"cursor": "CgwIgP3S1wYQ-Mz1zgMSJEtYRVVST0NVUFRPVEFMLTI2Tk9WMTIxNDAwTUFYU0tZLTE2Mw"
|
||||||
|
}
|
||||||
Reference in new issue
Block a user