Archived
feat(polymarket): skeleton and market-channel connectivity spike
Replaces the template's demo main.rs with a spike that connects to Polymarket's public market channel, subscribes to one market, and logs every frame with its receive time. Adds scripts/ci.sh (fmt, clippy, tests), the project CLAUDE.md, ADR 0001 (OS threads, shared-nothing), and the market channel's message shapes in docs/notes.md: no sequence number, a book hash on every change, mirrored Yes/No books.
This commit is contained in:
11 files changed
+713
-138
No files matched your search
@@ -0,0 +1,35 @@
|
||||
# 0001. Concurrency model: threads, shared-nothing
|
||||
|
||||
## Context
|
||||
|
||||
The collector moves a firehose of websocket messages from many sockets to a
|
||||
single output sink. This is the project's central bet (see
|
||||
[../project.md](../project.md)): it must be correct (no dropped messages, no
|
||||
races) and time-accurate (receive time stamped at the socket edge) before it is
|
||||
fast. The runtime is the one foundational, costly-to-change choice. The template
|
||||
already uses `std` threads + `crossbeam` channels and pulls in no tokio.
|
||||
|
||||
## Decision
|
||||
|
||||
V1 uses **OS threads**, not an async runtime — a sync websocket client
|
||||
(`tungstenite`), with one read loop per socket. State is **shared-nothing**: no
|
||||
mutexes. Each pipeline stage either owns its state on a single thread or is
|
||||
sharded so each shard owns its slice, and stages communicate over channels.
|
||||
|
||||
Async (tokio) is deferred as a possible optimization, re-evaluated at the start
|
||||
of extension 1 (M4), when socket-management complexity (reconnect, timeouts,
|
||||
redundancy) makes async ergonomics worth their cost.
|
||||
|
||||
## Why
|
||||
|
||||
Thread-per-socket is comfortable at this scale (symbols shard across tens of
|
||||
sockets, not thousands) and is the simplest, most readable option — correctness
|
||||
first, speed later. Shared-nothing + channels removes lock contention and data
|
||||
races by construction, and keeps the runtime pluggable: only the socket layer is
|
||||
runtime-specific, so a later async swap is localized to that boundary rather than
|
||||
a pipeline rewrite. Deferring async is therefore safe through the base challenge.
|
||||
|
||||
Trade-off: thread-per-socket does not scale to very large socket counts, and the
|
||||
extension-era socket-management logic (select / timeout / reconnect) is where
|
||||
threads and async diverge most — hence the explicit M4 re-evaluation point rather
|
||||
than "never."
|
||||
Reference in new issue
Block a user