Archived
feat(polymarket): skeleton and market-channel connectivity spike
Replaces the template's demo main.rs with a spike that connects to Polymarket's public market channel, subscribes to one market, and logs every frame with its receive time. Adds scripts/ci.sh (fmt, clippy, tests), the project CLAUDE.md, ADR 0001 (OS threads, shared-nothing), and the market channel's message shapes in docs/notes.md: no sequence number, a book hash on every change, mirrored Yes/No books.
This commit is contained in:
11 files changed
+713
-138
No files matched your search
+21
-10
@@ -11,31 +11,39 @@ abstraction is *extracted* once a second exchange exists, not designed up front.
|
||||
|
||||
## Milestones
|
||||
|
||||
1. **Skeleton + connectivity** — *not started*
|
||||
1. **Skeleton + connectivity** — *done (2026-11-04)*
|
||||
Project scaffolding inside the template: module layout (replace `main.rs`),
|
||||
`scripts/ci.sh` (fmt + clippy `-D warnings` + tests + audit), project
|
||||
CLAUDE.md, chosen websocket/HTTP libraries. Settle the pivotal
|
||||
async-vs-threads call. Resolve the output destination (stdout vs file).
|
||||
`scripts/ci.sh` (fmt + clippy `-D warnings` + tests), project CLAUDE.md, the
|
||||
websocket library (`tungstenite` + `rustls`). Runtime settled: thread-only V1
|
||||
(async deferred to M4; see [adr/0001-concurrency-model.md](adr/0001-concurrency-model.md)).
|
||||
Output destination is a non-issue (an output path; `/dev/stdout` for stdout).
|
||||
Connect to Polymarket and log raw messages.
|
||||
*Ends:* CI green; raw Polymarket messages flowing.
|
||||
|
||||
2. **Single-source book pipeline (Polymarket, subset of symbols)** — *not started*
|
||||
The central bet in its first real form: normalize events → per-symbol book
|
||||
(fold absolute deltas) → gap-detect → re-snapshot → SBE output (deltas +
|
||||
(fold absolute deltas) → loss-detect → re-snapshot → SBE output (deltas +
|
||||
snapshot every minute, on subscribe, on re-snapshot) → receive-time stamping
|
||||
→ clean SIGINT. One/few sockets, a fixed subset of symbols. Deterministic
|
||||
→ clean SIGINT, with the `PING` keepalive every 10 s. Polymarket has no
|
||||
sequence number, so loss detection (its book `hash`, or best bid/ask) is the
|
||||
opening question. One/few sockets, a fixed subset of symbols. Deterministic
|
||||
replay tests + snapshot↔delta reconciliation.
|
||||
*Ends:* correct SBE output for a symbol subset; tests green.
|
||||
|
||||
3. **Full coverage: discovery + sharding (base challenge complete)** — *not started*
|
||||
REST discovery loop with diffing (subscribe new, unsubscribe gone);
|
||||
load-balance all live regular-market symbols across N sockets.
|
||||
Discovery with diffing (subscribe new, unsubscribe gone) — REST polling, or
|
||||
Polymarket's `new_market` / `market_resolved` events — using subscribe /
|
||||
unsubscribe on open connections; load-balance all live regular-market symbols
|
||||
across N sockets.
|
||||
*Ends:* every live Polymarket symbol tracked — base challenge satisfied for
|
||||
one exchange.
|
||||
|
||||
4. **Extension 1 — redundant websockets** — *not started*
|
||||
Multiple connections per symbol; choose a trusted source and switch on
|
||||
loss/quiet/delay; emit a fresh snapshot on every source switch.
|
||||
**Async re-evaluation checkpoint:** before building this, reassess whether the
|
||||
socket-management complexity (reconnect/timeout/select, redundancy) warrants
|
||||
swapping the thread-only runtime for tokio (see ADR 0001).
|
||||
*Ends:* redundancy demonstrably reduces data loss; source-switch re-snapshots.
|
||||
|
||||
5. **Extension 2 — subscription monitoring** — *not started*
|
||||
@@ -54,11 +62,14 @@ abstraction is *extracted* once a second exchange exists, not designed up front.
|
||||
7. **Extension 4 — latency experiments + writeup** — *not started*
|
||||
Experiment with faster connections (first-after-maintenance, connection
|
||||
racing, source-port tricks) on both exchanges; document findings in a
|
||||
markdown file.
|
||||
markdown file, with the local clock's offset measured alongside (the dev
|
||||
machine ran ~30 ms slow; see notes.md).
|
||||
*Ends:* experiments documented.
|
||||
|
||||
8. **Polish + submission** — *not started*
|
||||
Final review, README for the solution, packaging/handoff for the challenge.
|
||||
Final review, README for the solution (the template's README still says the
|
||||
helpers are "demonstrated in `main.rs`", whose demo M1 replaced),
|
||||
packaging/handoff for the challenge.
|
||||
*Ends:* ready to submit.
|
||||
|
||||
## Why this order
|
||||
|
||||
Reference in new issue
Block a user