Archived
feat(polymarket): skeleton and market-channel connectivity spike
Replaces the template's demo main.rs with a spike that connects to Polymarket's public market channel, subscribes to one market, and logs every frame with its receive time. Adds scripts/ci.sh (fmt, clippy, tests), the project CLAUDE.md, ADR 0001 (OS threads, shared-nothing), and the market channel's message shapes in docs/notes.md: no sequence number, a book hash on every change, mirrored Yes/No books.
This commit is contained in:
11 files changed
+713
-138
No files matched your search
@@ -0,0 +1,63 @@
|
||||
# Order-book collector (Voluntas engineering challenge)
|
||||
|
||||
A collector for prediction-market order books (Polymarket, then Kalshi) that
|
||||
emits SBE-encoded updates and per-minute snapshots, built inside the challenge
|
||||
template. The brief is `../website/Engineering Challenge _ Voluntas.html`,
|
||||
outside this repo.
|
||||
|
||||
## Where things are
|
||||
|
||||
- `docs/project.md`: definition, central bet (the concurrency pipeline), and the
|
||||
reasons behind the design principles. Read before any design discussion.
|
||||
- `docs/roadmap.md` and `roadmap/NN. name.md`: milestones and each sprint's
|
||||
agreed plan. Read the current sprint file before writing code.
|
||||
- `docs/adr/`: hard-to-reverse decisions (0001: threads, shared-nothing).
|
||||
- `docs/notes.md`: domain rules and exchange protocol facts (delta semantics,
|
||||
loss detection, message shapes). Read before touching exchange parsing.
|
||||
- `src/util/` and `sbe_schemas/`: the template's SBE machinery, used as given so
|
||||
output stays readable by the template's `AlignedSbeReader`. Our code is the
|
||||
rest of `src/`.
|
||||
|
||||
## Working together
|
||||
|
||||
The user drives the pipeline / concurrency architecture: propose and debate it,
|
||||
and let the user settle it. Claude owns the data model.
|
||||
|
||||
## Writing code
|
||||
|
||||
- Shared-nothing threads: each stage owns its state on one thread or is sharded,
|
||||
and stages talk over channels. Mutexes stay out of the pipeline.
|
||||
- Boring and readable: std collections (`BTreeMap` for a price-ordered side),
|
||||
plain threads and channels. Speed work waits for the profiler, after V1.
|
||||
- An abstraction arrives with its second implementation (the exchange trait in
|
||||
M6).
|
||||
- Stamp receive time immediately after the socket read, and rebuild a book from
|
||||
a fresh snapshot whenever a lost message is detected (how depends on the
|
||||
exchange: `docs/notes.md`).
|
||||
- A corner cut for speed carries a comment naming its ceiling and upgrade path.
|
||||
|
||||
## Environment traps
|
||||
|
||||
- **Polymarket and Kalshi are DNS-blocked on the dev network**: the local
|
||||
resolver answers `195.186.4.210` / `2001:4d98:4:b10c::1` (a Swisscom block
|
||||
page). The user pins the hosts with `sudo ../host.sh on` (workspace root,
|
||||
outside this repo; `off` removes them). A timeout or the block address again
|
||||
means the CDN addresses rotated: ask the user to rerun it.
|
||||
- **Write 64 MiB chunks**: `AlignedSbeReader::open` hardcodes 64 MiB
|
||||
(`open_with_chunk_size` is private), so any other chunk size is unreadable.
|
||||
- **The binary runs until the server closes the socket.** macOS has no
|
||||
`timeout`; bound a run with
|
||||
`perl -e 'alarm 10; exec @ARGV' ./target/debug/engineering-challenge`.
|
||||
|
||||
## Workflow differences
|
||||
|
||||
A job-challenge repo cloned from the template: no Gitea, Woodpecker or release
|
||||
pipeline.
|
||||
|
||||
- **Kept**: `docs/`, `roadmap/` sprint files, `feat/*` branches, and
|
||||
`scripts/ci.sh` (fmt, clippy `-D warnings`, tests).
|
||||
- **Dropped**: the dependency audit (supply-chain checks are for deployed
|
||||
products), Woodpecker, `release.sh`, versioning and changelog, rc images, and
|
||||
the `prod` / `dev` branches. CI targets application quality, not prod-grade.
|
||||
- **Branches**: `main` plays the role of `dev`; `feat/*` branches are squashed
|
||||
into `main`.
|
||||
Reference in new issue
Block a user