This repository has been archived on 2026-11-16. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
orderbook-collector/docs/adr/0001-concurrency-model.md
T
pascal 0e687e7eba feat(polymarket): skeleton and market-channel connectivity spike
Replaces the template's demo main.rs with a spike that connects to
Polymarket's public market channel, subscribes to one market, and logs every
frame with its receive time. Adds scripts/ci.sh (fmt, clippy, tests), the
project CLAUDE.md, ADR 0001 (OS threads, shared-nothing), and the market
channel's message shapes in docs/notes.md: no sequence number, a book hash on
every change, mirrored Yes/No books.
2026-11-04 14:47:14 +01:00

1.7 KiB

0001. Concurrency model: threads, shared-nothing

Context

The collector moves a firehose of websocket messages from many sockets to a single output sink. This is the project's central bet (see ../project.md): it must be correct (no dropped messages, no races) and time-accurate (receive time stamped at the socket edge) before it is fast. The runtime is the one foundational, costly-to-change choice. The template already uses std threads + crossbeam channels and pulls in no tokio.

Decision

V1 uses OS threads, not an async runtime — a sync websocket client (tungstenite), with one read loop per socket. State is shared-nothing: no mutexes. Each pipeline stage either owns its state on a single thread or is sharded so each shard owns its slice, and stages communicate over channels.

Async (tokio) is deferred as a possible optimization, re-evaluated at the start of extension 1 (M4), when socket-management complexity (reconnect, timeouts, redundancy) makes async ergonomics worth their cost.

Why

Thread-per-socket is comfortable at this scale (symbols shard across tens of sockets, not thousands) and is the simplest, most readable option — correctness first, speed later. Shared-nothing + channels removes lock contention and data races by construction, and keeps the runtime pluggable: only the socket layer is runtime-specific, so a later async swap is localized to that boundary rather than a pipeline rewrite. Deferring async is therefore safe through the base challenge.

Trade-off: thread-per-socket does not scale to very large socket counts, and the extension-era socket-management logic (select / timeout / reconnect) is where threads and async diverge most — hence the explicit M4 re-evaluation point rather than "never."