From d06cfe1305958046350fa59becee9561e5cd31a5 Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Wed, 24 Jun 2026 18:32:09 +0000 Subject: [PATCH 1/8] docs(architecture): record multi-process topology ADRs and glossary Capture the from-scratch architecture grilling: a single-host, multi-process, event-sourced design. ADRs 0001-0006 (topology, Bus, canonical model, risk control loop, single-writer Core, broker reconciliation) plus 0007 (binding time: no hot-path dyn), 0008 (single-owner Kernel + stateless Policies over a StateView, emitting Decisions into a sink), and 0009 (spine-inverted, process-aligned crate topology). CONTEXT.md holds the ubiquitous language; the grilling summary tracks parked questions and the remaining branches (B strategy runtime, C Frontend/CLI). The 0009 topology is the decided target; the crates are not yet restructured. Co-Authored-By: Claude Opus 4.8 --- CONTEXT.md | 143 ++++++++++++++++++ ...0001-single-host-multi-process-topology.md | 27 ++++ ...nd-agnostic-bus-canonical-message-model.md | 34 +++++ ...003-canonical-model-adapter-translation.md | 21 +++ .../0004-risk-as-continuous-control-loop.md | 17 +++ .../0005-single-writer-event-sourced-core.md | 53 +++++++ .../0006-broker-reconciliation-contract.md | 35 +++++ ...ime-runtime-pluggable-iff-cross-process.md | 37 +++++ ...-single-owner-kernel-stateless-policies.md | 52 +++++++ ...topology-spine-inverted-process-aligned.md | 93 ++++++++++++ ...6-23-from-scratch-architecture-grilling.md | 97 ++++++++++++ 11 files changed, 609 insertions(+) create mode 100644 CONTEXT.md create mode 100644 docs/adr/0001-single-host-multi-process-topology.md create mode 100644 docs/adr/0002-backend-agnostic-bus-canonical-message-model.md create mode 100644 docs/adr/0003-canonical-model-adapter-translation.md create mode 100644 docs/adr/0004-risk-as-continuous-control-loop.md create mode 100644 docs/adr/0005-single-writer-event-sourced-core.md create mode 100644 docs/adr/0006-broker-reconciliation-contract.md create mode 100644 docs/adr/0007-binding-time-runtime-pluggable-iff-cross-process.md create mode 100644 docs/adr/0008-single-owner-kernel-stateless-policies.md create mode 100644 docs/adr/0009-crate-topology-spine-inverted-process-aligned.md create mode 100644 docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000..83dd44e --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,143 @@ +# OATH — Open Automatic Trading Hub + +The ubiquitous language for a single-host, multi-process, backend-agnostic +trading engine. This file is a glossary only — no implementation details. + +## Domain primitives + +**Symbol**: +OATH's canonical identifier for a tradable instrument, independent of any one +venue's ticker or internal id, so the same instrument offered by different +brokers collapses to a single `Symbol` (e.g. via perm_id / OpenFIGI). +_Avoid_: ticker, instrument, contract (for the canonical form). + +**Price**: +The value per unit of an instrument, expressed in its quote currency. +_Avoid_: cost, rate, level. + +**Quantity**: +An amount of an instrument. +_Avoid_: size, amount, volume. + +**Side**: +The direction of an order or trade — buy or sell. +_Avoid_: direction, way, sign. + +**Timestamp**: +A point in time, always UTC, with no timezone or offset attached. +_Avoid_: datetime, date, clock time. + +## Processes & topology + +**Adapter**: +A process that connects OATH to exactly one external venue, translating +between that venue's representation and OATH's canonical model. The translation +is the adapter's responsibility and never leaks inward (anti-corruption layer). +_Avoid_: connector, plugin, integration. + +**Broker**: +An adapter to a trading venue that provides everything needed to trade an +instrument: market data, instrument/reference data, and the order path (place +orders, receive fills). +_Avoid_: exchange, venue (when meaning the integration). + +**Data Provider**: +An adapter to a source of enrichment data a broker does not supply — news, +social, macro/country data — used to improve trading and risk decisions. +_Avoid_: feed, vendor, source. + +**Core**: +The process and central decision authority. Holds all live state needed for +portfolio and risk decisions (positions, open orders, exposure) and +continuously surveils it. Acts autonomously — e.g. cancels or amends a resting +order when conditions change — not only in response to a strategy. Initially +also hosts strategies. +_Avoid_: server, hub, main, master, cache. + +**Kernel**: +The single-writer heart of Core: the one logical thread that owns all canonical +state and is the only thing permitted to mutate it. Core's state is a pure fold +the Kernel applies, and decision Policies run inside it over a read-only view of +that state. +_Avoid_: engine, main loop, scheduler, reactor. + +**Risk Engine**: +The component within Core that continuously evaluates live state and holds +veto/cancel/amend authority over every order. A control loop, not a one-shot +pre-trade gate. +_Avoid_: risk check, validator, guard. + +**Policy**: +A stateless, compile-time-selected decision rule the Kernel invokes over a +read-only view of state — for example a risk Policy (cancel/amend rules) or an +execution Policy (how to work an order). Carries configuration, not state; any +private state it needs is custodied by the Kernel. +_Avoid_: rule engine, handler, check, plugin, strategy (reserved for the +user-facing Strategy). + +**Strategy Node**: +A process hosting one or more user strategies, isolated so that a strategy +fault cannot crash the Core. +_Avoid_: worker, runner, bot. + +**Supervisor**: +The operational-plane process that boots and watches a host's topology — starts +the Bus, spawns Core, Adapters, and Strategy Nodes, runs health checks and +throughput monitoring, and restarts failed processes. Purely effectful; it never +participates in Core's deterministic decision path. +_Avoid_: orchestrator, launcher, manager, daemon, conductor. + +**Frontend**: +An external process that observes — and may control — the running hub from +outside Core's deterministic path: positions, orders, P&L, process health. The +CLI is the first Frontend; TUIs, web, and desktop UIs are later ones. Depends only +on the public message model, Bus, and query interfaces. +_Avoid_: UI, dashboard, console, client. + +## Messages & decisions + +**Signal**: +A strategy's proposal to trade (which `Symbol`, `Side`, size/urgency), +submitted to Core for a decision. Not itself an order — Core decides whether, +when, and how much to act. +_Avoid_: order, intent, trade (from a strategy). + +**Decision**: +What a Policy returns to the Kernel for one input: the set of intended actions — +admit/shape/reject a Signal, or cancel/amend/flatten resting orders. The Policy +decides; the Kernel performs the actions. Internal to Core, never sent on the Bus. +_Avoid_: verdict, judgment, command, ruling. + +**Order**: +An instruction Core sends to a broker to buy or sell, after the Risk Engine +approves. Carries the freshness/validity context it was decided under. +_Avoid_: trade, transaction. + +**Fill**: +A partial or complete execution of an order, reported by a broker. +_Avoid_: execution, trade, transaction. + +## Persistence & recovery + +**Event Log**: +The persisted, totally-ordered record of every input Core consumed, in the +order it consumed them. Core's state is a pure fold over it. +_Avoid_: journal, history, audit log, WAL. + +**Snapshot**: +A point-in-time capture of Core's full state, tagged with its position in the +Event Log, so recovery can resume without replaying from the beginning. +_Avoid_: checkpoint, dump. + +**Replay**: +Re-feeding the Event Log through the identical fold to reconstruct state, with +all external side effects suppressed. The basis of both recovery and +backtesting. +_Avoid_: rerun, simulation (for the deterministic re-feed specifically). + +## Transport + +**Bus**: +The transport over which processes exchange canonical messages. Backend-agnostic +(e.g. shared-memory zero-copy, Unix sockets, Kafka) behind a single trait. +_Avoid_: queue, channel, broker (reserved for the venue role above). diff --git a/docs/adr/0001-single-host-multi-process-topology.md b/docs/adr/0001-single-host-multi-process-topology.md new file mode 100644 index 0000000..e85d08d --- /dev/null +++ b/docs/adr/0001-single-host-multi-process-topology.md @@ -0,0 +1,27 @@ +# Single-host, multi-process topology with fault isolation + +OATH runs as several cooperating processes on a single host: one process per +**Adapter** (broker or data provider), one **Core** process (risk engine, order +execution, portfolio, and initially strategies), and one or more **Strategy +Node** processes. We chose this over a single-process monolith so that a flaky +external integration — the thing that actually hangs, leaks, or gets +rate-limited — cannot crash the decision-making centre, and over a +multi-machine distributed system because a retail, single-box deployment lets us +use shared-memory IPC for near-in-process latency. + +## Consequences + +- Fault isolation: an adapter crash takes down only that venue's connectivity. +- The **zero-copy fast path** (shared-memory IPC, e.g. iceoryx2) requires all + processes on one host. Network Bus backends (Kafka, RabbitMQ, …) relax this: + processes may live on different machines at a latency cost. Cross-machine + distribution is therefore supported via those backends, just not on the + optimized path. (See ADR-0002.) +- The dominant design concern becomes the inter-process contract (see ADR-0002), + not in-process trait dispatch. +- Strategies start co-located in Core and split into Strategy Nodes later; the + strategy↔Core seam is designed as a swappable call from day one. +- Adapters **and** Strategy Nodes are hot-pluggable: a new venue or strategy + process can join the Bus at runtime (subject to symbology resolution and a + registration handshake) without restarting Core — something a single-process + design cannot do. diff --git a/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md b/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md new file mode 100644 index 0000000..cd45d0c --- /dev/null +++ b/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md @@ -0,0 +1,34 @@ +# Backend-agnostic Bus with one canonical message model + +Processes communicate over a **Bus** abstracted behind a single trait, so the +transport backend is swappable (zero-copy shared memory such as iceoryx2 for the +fast path; Unix sockets or Kafka for other users). There is exactly **one +canonical message model** shared by every backend — backends differ only in how +they move bytes, never in the data they carry. + +## Considered options + +- *Per-backend data models* — rejected: it recreates, one layer up, the + representation swamp that ADR-0003 outlaws in the core. +- *One model, intersection of backend constraints* — chosen. + +## Consequences + +- Message payloads are designed to the **intersection** of backend constraints: + fixed-layout, `#[repr(C)]` plain-old-data (for zero-copy) **and** serializable + (for network backends) — roughly a `Pod + Serialize` bound. +- The trait must be designed to the **stricter** ownership contract: zero-copy + hands back a *loaned*, lifetime-bounded sample; network backends hand back an + *owned* value. Modelling the borrowed/lifecycle-bounded case keeps zero-copy. +- `#[repr(C)]` payloads are brittle under schema evolution; versioning is a + known open concern to be addressed separately. +- Delivery semantics are per-topic: market data is drop-to-latest (ring buffer); + orders and fills are reliably delivered (never dropped on the wire). Acting on + stale-but-delivered messages is a separate, consumer-side freshness concern. +- **Durability is a backend capability, not a Bus guarantee.** iceoryx2 is + ephemeral (zero-copy, fast, lost on crash); Kafka and Chronicle Queue are + durable replayable logs. A durable backend can double as Core's event log / + recovery journal (see ADR-0005). +- Pub/sub backends support **dynamic publishers/subscribers**, so adapters and + strategy nodes can be added or removed at runtime without restarting Core + (see ADR-0001). diff --git a/docs/adr/0003-canonical-model-adapter-translation.md b/docs/adr/0003-canonical-model-adapter-translation.md new file mode 100644 index 0000000..d2339cd --- /dev/null +++ b/docs/adr/0003-canonical-model-adapter-translation.md @@ -0,0 +1,21 @@ +# Canonical core model with adapter-side translation + +The Core and the Bus speak exactly one canonical vocabulary (`Symbol`, `Price`, +`Quantity`, …), chosen at compile time and identical for every adapter. Each +**Adapter** owns the translation between its venue's representation and the +canonical model at its own boundary (an anti-corruption layer); venue-specific +representations never leak inward. A central symbology (e.g. perm_id / OpenFIGI) +gives the canonical identity, so the same instrument offered by different +brokers collapses to a single `Symbol` and is not double-counted by portfolio or +risk. + +## Consequences + +- Every adapter carries a translation layer (precision conversion, symbol + resolution, mapping tables) — accepted cost, paid to keep the core clean. +- `Price`/`Quantity` are newtypes over a swappable inner numeric type + (`rust_decimal` for the MVP — 16-byte, `Copy`, heap-free, so zero-copy-safe); + the inner type can later become fixed-point `i64` or a bignum at compile time + without touching call sites. +- Translation lives in the adapter process, so a malformed-message bug in one + venue's translation cannot corrupt another's. diff --git a/docs/adr/0004-risk-as-continuous-control-loop.md b/docs/adr/0004-risk-as-continuous-control-loop.md new file mode 100644 index 0000000..ca37336 --- /dev/null +++ b/docs/adr/0004-risk-as-continuous-control-loop.md @@ -0,0 +1,17 @@ +# Risk as a continuous, autonomous control loop + +The Risk Engine is a continuous control loop with veto/cancel/amend authority +over every order, not a one-shot pre-trade gate. It surveils all live state and +may act autonomously — for example, cancelling a resting order already on a +venue's book because a correlated fill changed the exposure or capital is now +better deployed elsewhere. Strategies only *detect* and propose **Signals**; +Core *decides and acts*. + +## Consequences + +- The `risk-core` / `execution-core` traits are an event-in / command-out state + machine, not a `check(order) -> verdict` function. +- Risk is un-bypassable: a Strategy cannot reach a Broker directly; it emits a + Signal that Core adjudicates. +- Core must hold and continuously update the full state risk depends on, which + motivates the single-writer design under discussion in ADR-0005. diff --git a/docs/adr/0005-single-writer-event-sourced-core.md b/docs/adr/0005-single-writer-event-sourced-core.md new file mode 100644 index 0000000..8801d6a --- /dev/null +++ b/docs/adr/0005-single-writer-event-sourced-core.md @@ -0,0 +1,53 @@ +# Single-writer, event-sourced, deterministic Core + +Deterministic replay is a non-negotiable goal, and Core must enforce global +shared invariants (account buying power, global buying power, per-asset-class +risk) that are inherently cross-symbol. A global invariant plus deterministic +ordering forces a **single writer**: one logical thread owns all state — per +-symbol and global — and is the only thing that may mutate it. Inputs from all +processes are assigned a total order at a single ingress and appended to a +persisted, totally-ordered input log; Core's state is a pure fold over that log, +and **replay re-feeds the log through the identical fold with external side +effects suppressed**. + +For the MVP this is implemented as a **single-threaded deterministic kernel** +with I/O and persistence offloaded to async worker threads that feed results +back as events — the model proven in production by NautilusTrader. The single +*writer* (the decision stage) is fundamental and stays; the single *thread* is +an MVP implementation detail we can later replace with a staged pipeline +(LMAX-Disruptor-shaped: parallel parse/journal/publish stages around one +decision stage) **if and only if** profiling shows the kernel is the bottleneck. + +## Considered options + +- *Shard by Symbol* — rejected: global cross-symbol invariants would force a + distributed two-phase reservation on the hot path, fighting both latency and + determinism. +- *Single-writer, single-threaded kernel + offloaded async I/O (MVP)* — + accepted. Matches NautilusTrader's shipping architecture. +- *LMAX-Disruptor staged pipeline (single decision stage)* — the measured + optimization path, not built until needed. Caveats: stall propagation, + busy-spin CPU burn, cache-line/`unsafe` tuning, ring sizing, debuggability. + +## Consequences + +- The persisted ordered input log is the `oath-persistence-core` event log; Core + is an event-sourced state machine and persistence is its journal. +- **Recovery = snapshot + replay-tail**: periodically snapshot Core state; on + restart, load the last snapshot then replay only the log tail after it. A + durable Bus backend (Kafka, Chronicle Queue) can serve as this log (see + ADR-0002). +- A hard line separates the *deterministic decision* (replayable) from the + *effectful action* (live-only, suppressed during replay). Reconciling + in-flight orders with the broker after a crash is a known open problem. +- The MVP kernel is bounded by one core; parallel work (ML, analytics) runs + off-loop in workers and feeds results back as events. +- `unsafe` is permitted where it demonstrably buys performance, via a justified + per-crate override of the workspace `unsafe_code = "deny"` lint. + +## Future fault tolerance + +A **hot-standby Core** that replays the same log via consensus (Aeron-Cluster +style replicated deterministic state machine) is the documented path to Core +failover. Out of scope for the MVP; recorded so the single-writer model is not +mistaken for a single point of failure with no exit. diff --git a/docs/adr/0006-broker-reconciliation-contract.md b/docs/adr/0006-broker-reconciliation-contract.md new file mode 100644 index 0000000..510a9e6 --- /dev/null +++ b/docs/adr/0006-broker-reconciliation-contract.md @@ -0,0 +1,35 @@ +# Crash recovery: broker-authoritative reconciliation + +After a crash, Core recovers in two steps: **replay** the Event Log to rebuild +decision state (side effects suppressed), then **reconcile** against the broker, +which is the **single source of truth** for what actually happened to orders, +positions, and fills. Replay restores intent; reconciliation restores reality; +Core repairs any divergence into its recovered state. + +## The reconciliation join key + +Every order carries a **client order id** — the join key that lets Core ask the +broker a precise question ("what happened to `abc123`?") instead of matching by +ambiguous attributes (symbol/side/qty/price). Because the kernel is +deterministic, this id is regenerated identically on replay, so an order decided +before a crash is matched exactly to the broker's report. Without the id, +reconciliation degrades to fuzzy attribute matching and silently repairs +state incorrectly whenever two similar orders or partial fills exist. + +## The ordering invariant (write-ahead) + +The input(s) that trigger an order decision **must be durably appended to the +Event Log before the resulting order is transmitted to the broker.** Otherwise +replay cannot regenerate the order (or its client order id) and the broker is +left holding an unmatchable orphan order. A *separate* write-ahead "decision +record" is not needed — replaying the inputs regenerates the decision — but this +log-before-send ordering is mandatory. + +## Mandatory adapter capability + +Every **Broker** adapter MUST provide (a) **idempotent submit** keyed by a +client order id (deduping retransmissions), and (b) a **queryable** view of +order / position / fill state for reconciliation. Venues lacking native support +must emulate it inside the adapter (persisted id mapping, dedup table). Venues +where this cannot be made to hold are excluded — a deliberate scope boundary +favouring a clean, trustworthy core over maximal venue coverage. diff --git a/docs/adr/0007-binding-time-runtime-pluggable-iff-cross-process.md b/docs/adr/0007-binding-time-runtime-pluggable-iff-cross-process.md new file mode 100644 index 0000000..6d72b80 --- /dev/null +++ b/docs/adr/0007-binding-time-runtime-pluggable-iff-cross-process.md @@ -0,0 +1,37 @@ +# Binding time: runtime-pluggable iff cross-process + +Every OATH binary — Core, each Adapter, each Strategy Node — is fully +monomorphized internally: **no `dyn` trait object sits on any decision hot +path.** "Swappable" is achieved two different ways depending on *what* is +swapped — **compile-time generics** for anything in-process, and **process-level +pluggability over the Bus** (spawn a process, run a registration handshake) for +anything that joins at runtime. The rule: *runtime-pluggable ⟺ a separate +process across the Bus; in-process ⟹ compile-time static.* + +## Considered options + +- *Trait objects (`dyn`) for swappable subsystems* — rejected: dynamic dispatch + on the per-event decision path defeats the latency goal (no inlining across the + call, vtable indirection) and is unnecessary — the only thing that must change + at runtime is *which processes are connected*, not which code a hot loop calls. +- *Compile-time generics in-process + process/Bus pluggability across* — chosen. + +## Consequences + +- Risk/execution Policies and I/O backends (net, persistence, Bus impl) are bound + at build time via generics; a user *builds their Core* with the rules and + backends they need. There is no runtime in-process plugin loader. +- The polymorphism that lets Core talk to "any adapter" is carried by the + canonical message model + Bus trait (ADR-0002/0003), **not** by an in-process + trait. Core holds a Bus handle, never a `Box`, and never depends on + a `Broker` / `DataProvider` / `Strategy` trait — those traits live on the + adapter/strategy side and are called statically by their host harness. +- Adapters and Strategy Nodes are added at runtime by spawning a process that + connects to the Bus and completes a registration handshake (ADR-0001) — + pluggability without any in-process dynamic dispatch. +- Co-location is a *backend choice, not a code path*: an initially co-located + Strategy (ADR-0001) talks through an in-memory Bus backend; moving it to its own + process swaps the backend, not the strategy code. The strategy↔Core seam never + branches on local-vs-remote. +- Cost accepted: no pre-built generic binary with runtime-loadable in-process + plugins; each deployment is compiled for its chosen Policies/backends. diff --git a/docs/adr/0008-single-owner-kernel-stateless-policies.md b/docs/adr/0008-single-owner-kernel-stateless-policies.md new file mode 100644 index 0000000..c817db8 --- /dev/null +++ b/docs/adr/0008-single-owner-kernel-stateless-policies.md @@ -0,0 +1,52 @@ +# Single-owner Kernel with stateless Policies over a StateView + +Core's decision components — risk, execution, portfolio — hold **no state of +their own and do not depend on one another**. The single-writer **Kernel** +(ADR-0005) owns *all* canonical state; each component is a **stateless Policy** +the Kernel invokes as a decision function over a read-only `StateView` of that +state. This extends ADR-0005's single-owner argument one level down: the same +reason cross-symbol invariants forced a single writer *between* processes (rather +than sharding) forbids distributing state ownership *within* Core across +co-owning components. + +## Considered options + +- *Component co-owners* — portfolio owns positions, execution owns the open-order + table, risk owns limit state; the Kernel sequences calls but each mutates its + own slice. Rejected: a global invariant (e.g. buying power) spans all three + owners, so the check must either read across them (which re-creates the + `StateView` anyway) or be hoisted into the Kernel — and distributed mutation + widens the determinism-audit surface and complicates Snapshot. It re-litigates + ADR-0005 one level down. +- *Kernel sole owner; components are stateless Policies over a `StateView`* — + chosen. + +## Consequences + +- The old `risk → execution → portfolio` dependency edges vanish: a Policy + depends only on the `StateView` contract, never on a sibling. Portfolio is + modelled as *fold logic* (how a Fill updates positions) plus read accessors, not + as a state owner. +- "Swappable" here is a **Policy swap** (different risk *rules*, a different + execution *algo*), not a backend swap — bound at compile time per ADR-0007. A + swapped-in Policy holds a read-only view and therefore *cannot* corrupt + canonical state. +- A Policy that needs private state declares an associated `Private` type the + Kernel **custodies** and the Policy mutates **in place** (`&mut Self::Private`) + — keeping it inside the one Snapshot and one fold, and avoiding the per-event + allocation a functional `-> NewState` return would impose on the Disruptor + decision stage (ADR-0005). +- A Policy never *acts*; it emits a **Decision** — the set of intended actions for + one input (admit/shape/reject a Signal; cancel/amend/flatten resting orders). + The Kernel drains the Decision and performs every action — it is the sole actor, + with effects suppressed on replay. To avoid per-event allocation the Policy + pushes actions into a Kernel-provided, reused **sink** (`&mut ActionSink`) + rather than returning an owned collection — the same in-place discipline as + `Private`. +- `StateView` is a single read-only trait, **generic-dispatched** (monomorphized, + zero-cost, fixture-implementable for tests), exposing all canonical read-state. + It must expose **no mutation or interior mutability** and **no nondeterministic + iteration** (keyed/ordered accessors only), so replay cannot diverge. +- `StateView`, `Decision`, and the Policy traits are a *Core-subsystem* contract: + they live in a Core-internal hub crate, not in `oath-model` (reserved for + cross-subsystem / over-the-Bus types). diff --git a/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md b/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md new file mode 100644 index 0000000..0acb128 --- /dev/null +++ b/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md @@ -0,0 +1,93 @@ +# Crate topology: spine-inverted and process-aligned + +The original workspace is a single-process monolith — `oath-engine` composes every +layer (`engine → {everything}`), with inter-layer trait edges (`risk → execution → +portfolio`). ADRs 0001–0008 commit instead to a single-host, multi-process, +event-sourced system, which inverts the graph: there is no top composer, only a +**bottom contract** every process depends on. We restructure the crates to match — +the dependency arrows point *inward to `oath-model`* (the canonical model + Bus + +Event Log spine), and the directory layout encodes the process boundaries. + +Two naming rules make the structure self-describing: + +- **`/api` is the trait crate**; `/` are + implementations (backends or Policies). `api` replaces the old `*-core` suffix. +- **`core/` is reserved for the Core process.** Its single-writer loop is + `core/kernel` (the **Kernel**); its trait hub is `core/api`. Top-level + directories are processes/subsystems; crates nested under `core/` are Core + internals — so an illegal reach from, say, `adapter/` into `core/kernel` is + visible at a glance. + +## Target layout + +``` +crates/ + model/ oath-model primitives + message payloads + + bus/ + api/ oath-bus-api Bus trait + iceoryx2/ chronicle/… oath-bus-* backends (chronicle also impls event-log) + event-log/ + api/ oath-event-log-api Event Log + Snapshot traits + chronicle/ parquet/… oath-event-log-* backends (DataFusion/DuckDB/parquet — parked) + persistence/ ── reserved ── + api/ oath-persistence-api Repository trait + …/ oath-persistence-* backends + + core/ ── Core process ── + api/ oath-core-api StateView, Decision, ActionSink, + RiskPolicy/ExecutionPolicy/Portfolio + risk/ oath-core-risk RiskPolicy impl + execution/ oath-core-execution ExecutionPolicy impl + portfolio/ oath-core-portfolio Portfolio impl (generic, for now) + kernel/ oath-core-kernel Kernel⟨R,E,P⟩ single-writer loop (lib) + host/ oath-core bin: the Core process + + adapter/ + api/ oath-adapter-api harness + Broker/DataProvider traits + net/ + api/ oath-adapter-net-api HTTP/WS traits + reqwest/… oath-adapter-net-* backends + ibkr/… oath-adapter-ibkr bin: a venue + + strategy/ + api/ oath-strategy-api Strategy trait + host harness + host/ oath-strategy-host bin: Strategy Node + + cli/ oath-cli bin: Frontend (MVP) + supervisor/ oath-supervisor bin: operational plane +``` + +## Considered options + +- *Keep the monolith graph (`engine → {everything}`)* — rejected: it is the + single-process composition ADR-0001 explicitly chose against. +- *Per-process top composers* — unnecessary: with binding-time pluggability + (ADR-0007) each process binary composes itself from generic libraries; no shared + composer crate is needed. + +## Consequences + +- **`oath-engine` is deleted.** Its composition role is gone; the Core process is + the binary `oath-core` (at `core/host`), assembled from `core/kernel` + chosen + Policies + chosen backends. Topology orchestration moves to `oath-supervisor`, + kept out of Core for determinism (ADR-0005). +- **The Kernel is generic** (`Kernel`, in `core/kernel`) and depends only + on the trait hub `core/api`, never on a concrete Policy; the `oath-core` binary + binds the concrete risk/execution/portfolio crates (ADR-0007/0008). +- **Adapter/Strategy traits live on their own side.** `Broker`/`DataProvider` + (`adapter/api`) and `Strategy` (`strategy/api`) are called statically by their + host harness; Core depends on neither — it shares only the canonical model over + the Bus (ADR-0007). +- **`oath-ingest-core` is deleted.** Market data is canonical messages in + `oath-model`, published by adapters, carried on the Bus. +- **Event Log and repositories split.** `event-log/api` (append-only, ordered, + replayed — the recovery spine) is separated from the reserved `persistence/api` + (keyed, queryable repositories: read-models, symbology, adapter dedup tables). +- **`net` moves under `adapter/`** (`adapter/net/api`) — adapters are its only + user; Core and strategies speak only the Bus. +- **Two new process roles**: `oath-supervisor` (operational plane) and `oath-cli` + (the first Frontend). Both sit outside Core's deterministic path and depend only + on public `*-api` + `oath-model` + the Bus. +- The README dependency graph is updated when the restructure is implemented + (one issue, one PR); **this ADR is the authoritative target until then.** diff --git a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md new file mode 100644 index 0000000..c2c9430 --- /dev/null +++ b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md @@ -0,0 +1,97 @@ +# From-scratch architecture — grilling summary (2026-06-23) + +**Status:** Branch (A) — crate & dependency-graph revision — **closed 2026-06-24** +(ADRs 0007–0009). Branches (B) strategy runtime and (C) Frontend/CLI remain. + +## The question + +"If we built OATH's scaffolding from scratch, would we do it differently?" Goal: +a high-performance, production-grade, secure, maintainable trading engine with +everything behind traits so backends are swappable. + +**Short answer: yes, structurally.** The existing scaffolding assumes a +single-process, statically-composed monolith (`oath-engine` composes all layers +via `EngineBuilder`; dep graph `engine → {everything}`). The design we converged +on is a **single-host, multi-process, event-sourced system communicating over a +swappable message Bus**. The current crate graph is aimed at the monolith we +decided *not* to build — revising it is open branch (A) below. + +## Decisions made (see docs/adr/) + +- **[ADR-0001](../../adr/0001-single-host-multi-process-topology.md)** — + Single-host, multi-process topology: one process per Adapter, one Core (risk + + execution + portfolio, initially also strategies), one or more Strategy Nodes. + Fault isolation is the driver. Adapters and strategy nodes are hot-pluggable. +- **[ADR-0002](../../adr/0002-backend-agnostic-bus-canonical-message-model.md)** — + Backend-agnostic Bus behind a trait; **one** canonical message model for all + backends (`Pod + Serialize`). Per-topic delivery semantics; durability is a + backend capability (iceoryx2 ephemeral; Kafka / Chronicle Queue durable). +- **[ADR-0003](../../adr/0003-canonical-model-adapter-translation.md)** — + Canonical core model; each Adapter translates at its boundary (anti-corruption + layer); central symbology (perm_id / OpenFIGI). `Price`/`Quantity` are + newtypes over a swappable inner type (`rust_decimal` MVP). +- **[ADR-0004](../../adr/0004-risk-as-continuous-control-loop.md)** — Risk is a + continuous autonomous control loop with cancel/amend authority, not a pre-trade + gate. Strategies detect & propose Signals; Core decides & acts. +- **[ADR-0005](../../adr/0005-single-writer-event-sourced-core.md)** — + Single-writer, event-sourced, deterministic Core. MVP = single-threaded kernel + + offloaded async I/O (NautilusTrader-validated); disruptor pipeline is a + later, *measured* optimization. Replay = fold over the Event Log. +- **[ADR-0006](../../adr/0006-broker-reconciliation-contract.md)** — Broker is + the source of truth. Recovery = replay + reconcile, joined by a client order + id. Log-before-send ordering invariant. Idempotent submit + queryable order + state are **mandatory** Broker-adapter capabilities. +- **[ADR-0007](../../adr/0007-binding-time-runtime-pluggable-iff-cross-process.md)** — + Binding time: runtime-pluggable ⟺ a separate process across the Bus; everything + in-process is compile-time static. **No `dyn` on any hot path.** +- **[ADR-0008](../../adr/0008-single-owner-kernel-stateless-policies.md)** — + Single-owner Kernel; risk/execution/portfolio are stateless **Policies** over a + read-only `StateView`. A Policy emits a **Decision** (actions into a reused + sink); the Kernel is the sole actor. +- **[ADR-0009](../../adr/0009-crate-topology-spine-inverted-process-aligned.md)** — + Crate topology: spine-inverted (everything → `oath-model`), process-aligned; + `/api` = traits, `core/` = the Core process. `oath-engine` and + `oath-ingest-core` deleted; Event Log / repositories split; new `supervisor` and + `cli` (Frontend) binaries. + +Glossary: **[CONTEXT.md](../../../CONTEXT.md)** (primitives, processes, messages, +persistence/recovery, transport). + +## Validated against prior art + +NautilusTrader independently uses nearly the same model (single-threaded +deterministic kernel, offloaded I/O, centralized cache, no sharding, +strategy→risk→exec). Key *deliberate* divergence: Nautilus is single-process +with a non-swappable bus; OATH is multi-process with a swappable bus — paying +cross-process complexity to buy crash containment and hot-pluggability. + +## Open branches (next sessions) + +- **(A) Crate & dependency-graph revision** — ✅ **closed 2026-06-24** (ADR-0009). + Spine-inverted, process-aligned topology; full target tree recorded in the ADR. +- **(B) Strategy runtime & multi-topic-join framework** — subscribing to and + *fusing* multiple streams (market data + news → Signal); the strategy↔Core + seam; strategy sandboxing contract. +- **(C) Frontend / CLI design** — the MVP CLI Frontend: observability + operational + control (no trading control — operator orders route through Signal→risk later); + query channels (req/resp to Core & Supervisor) and telemetry topics. + +## Parked sub-questions (don't lose these) + +- Schema **evolution/versioning** of `#[repr(C)]` POD messages (ADR-0002). +- Bus trait's **loan-vs-own** contract (design to the borrowed/lifecycle-bounded + case to keep zero-copy). +- Per-topic **delivery semantics** detail + backpressure / ring sizing. +- **Req/resp pattern** (registration, reconciliation queries, Frontend queries): + the request is effectful/live-only, the response enters Core as an ordered input; + Bus-capability vs side-channel TBD. +- **Event Log / repository backend**: parquet + DataFusion / DuckDB candidate + (keep the log↔repository split from ADR-0009). +- **Snapshot** cadence and contents (recovery substrate). +- **Symbology** design: canonical identity (perm_id/OpenFIGI) + per-adapter + mapping. +- `Price`/`Quantity` numeric needs per asset class (crypto/wei) — when to move + the inner type off `rust_decimal`. +- **Deterministic client-order-id** generation scheme. +- **Core failover** (Aeron-Cluster-style hot standby) — future, documented in + ADR-0005. From 96ec5fb7f854ac5d270bc1eeec4c796236ac25b4 Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Thu, 25 Jun 2026 19:37:10 +0000 Subject: [PATCH 2/8] docs(architecture): strategy-runtime ADRs 0010-0013 + glossary Close grilling Branch B (strategy runtime & multi-topic-join). - ADR-0010: strategies are out-of-Core deterministic folds in Strategy Nodes (supersedes ADR-0001 co-location); capability-derived backtest fidelity (DetCtx/IoCtx), not a gate. - ADR-0011: trading mode = (data feed x execution backend); Simulated Broker as an adapter backend; each mode an isolated Environment (Backtest/Shadow/Paper/Live); Core + Strategy mode-agnostic. - ADR-0012: event-time-ordered fusion + latest-value view; per-Environment lateness bound L (L=0 live); ingestion-order logging for bit-exact session replay. - ADR-0013: one push framework; Signal = idempotent target + freshness + StrategyId; two-part registration; fault isolation. CONTEXT.md: sharpen Strategy/Strategy Node/Core/Signal; add Backtest and the Execution environments section. Spec: status, parked leaves, Extism. Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 68 +++++++++++++++++-- ...ategies-out-of-core-deterministic-folds.md | 52 ++++++++++++++ ...1-execution-environments-mode-isolation.md | 40 +++++++++++ ...strategy-input-fusion-event-time-parity.md | 39 +++++++++++ ...runtime-push-signal-target-registration.md | 53 +++++++++++++++ ...6-23-from-scratch-architecture-grilling.md | 55 ++++++++++++--- 6 files changed, 291 insertions(+), 16 deletions(-) create mode 100644 docs/adr/0010-strategies-out-of-core-deterministic-folds.md create mode 100644 docs/adr/0011-execution-environments-mode-isolation.md create mode 100644 docs/adr/0012-strategy-input-fusion-event-time-parity.md create mode 100644 docs/adr/0013-strategy-runtime-push-signal-target-registration.md diff --git a/CONTEXT.md b/CONTEXT.md index 83dd44e..70de151 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -50,8 +50,8 @@ _Avoid_: feed, vendor, source. The process and central decision authority. Holds all live state needed for portfolio and risk decisions (positions, open orders, exposure) and continuously surveils it. Acts autonomously — e.g. cancels or amends a resting -order when conditions change — not only in response to a strategy. Initially -also hosts strategies. +order when conditions change — not only in response to a strategy. Never hosts +strategies in-process; they always run in separate Strategy Nodes. _Avoid_: server, hub, main, master, cache. **Kernel**: @@ -75,9 +75,17 @@ private state it needs is custodied by the Kernel. _Avoid_: rule engine, handler, check, plugin, strategy (reserved for the user-facing Strategy). +**Strategy**: +A unit of user-authored logic, hosted in a Strategy Node, that consumes market +data and enrichment off the Bus and proposes Signals. It never trades directly — +Core decides and acts. A _deterministic_ Strategy folds purely over its Bus +inputs; an _effectful_ Strategy may also do ad-hoc I/O. +_Avoid_: algo, bot, model, signal generator. + **Strategy Node**: -A process hosting one or more user strategies, isolated so that a strategy -fault cannot crash the Core. +A separate process — never Core itself — hosting one or more user strategies, +isolated so that a strategy fault cannot crash the Core. Communicates with Core +only over the Bus. _Avoid_: worker, runner, bot. **Supervisor**: @@ -94,12 +102,48 @@ CLI is the first Frontend; TUIs, web, and desktop UIs are later ones. Depends on on the public message model, Bus, and query interfaces. _Avoid_: UI, dashboard, console, client. +## Execution environments + +**Environment**: +An isolated instance of the OATH topology — its own Core, Event Log, +portfolio/risk state, execution-adapter binding, data feeds, and Bus namespace — +so that several can run on one host without their orders, fills, positions, or +logs colliding. Its mode is its data feed × execution backend (e.g. live feed × +live account); all its feeds share one temporal profile (real-time, delayed-by-D, +or historical). +_Avoid_: instance, deployment, tenant, session. + +**Simulated Broker**: +A Broker-adapter backend that fills Orders internally against the Environment's +own market-data feed instead of routing to a real venue. The execution backend +for Backtest and Shadow. +_Avoid_: mock, fake, matching engine, paper (paper uses a real broker account). + +**Shadow**: +An Environment running live (or delayed) data through a Simulated Broker, +alongside Live or Paper, to test a Strategy on real-time data with no capital at +risk. It fills against the exact data the Strategy saw, exercising the model +end-to-end. +_Avoid_: dry-run, what-if, sim. + +**Paper Trading**: +An Environment routing real Orders to a broker's paper (demo) account — real +broker-side execution, no real money, often on the broker's delayed market data. +_Avoid_: demo, sandbox, simulation. + +**Live Trading**: +An Environment routing real Orders to a broker's live account, with real money at +risk. +_Avoid_: production, real-money mode. + ## Messages & decisions **Signal**: -A strategy's proposal to trade (which `Symbol`, `Side`, size/urgency), -submitted to Core for a decision. Not itself an order — Core decides whether, -when, and how much to act. +A Strategy's proposal of a _desired target_ — the position or exposure it wants +in a `Symbol` — submitted to Core for a decision, never an Order. Idempotent and +nettable: Core reconciles actual → target across strategies under risk, deciding +whether, when, and how much to act. Carries the as-of freshness it was decided +under and the proposing Strategy's identity. _Avoid_: order, intent, trade (from a strategy). **Decision**: @@ -135,6 +179,16 @@ all external side effects suppressed. The basis of both recovery and backtesting. _Avoid_: rerun, simulation (for the deterministic re-feed specifically). +**Backtest**: +Running live Strategy code against recorded historical market data and +enrichment, fed in timestamp order through the same Core wired to a Simulated +Broker, to evaluate what it would have done. A deterministic Strategy's Backtest reproducibly matches live; +an effectful Strategy can still be backtested, but parity and freedom from +lookahead are then the author's responsibility, not the framework's. Distinct +from Replay: Replay re-feeds Core's _logged_ inputs and never re-runs strategies, +whereas a Backtest _regenerates_ Signals from the Strategy. +_Avoid_: simulation, paper trading, replay (for this). + ## Transport **Bus**: diff --git a/docs/adr/0010-strategies-out-of-core-deterministic-folds.md b/docs/adr/0010-strategies-out-of-core-deterministic-folds.md new file mode 100644 index 0000000..b635387 --- /dev/null +++ b/docs/adr/0010-strategies-out-of-core-deterministic-folds.md @@ -0,0 +1,52 @@ +# Strategies as out-of-Core deterministic folds with capability-derived backtest fidelity + +Strategies run only in **Strategy Node** processes — never co-located in Core +(superseding ADR-0001's "initially also hosts strategies") — and sit entirely +**outside Core's deterministic boundary**: the Event Log records their Signals as +ordered inputs, and Replay never re-runs a strategy. A Strategy is a +**deterministic fold** over its Bus inputs plus a framework-injected clock and +seeded RNG; external or nondeterministic data must enter as **Data Provider** +messages on the Bus (ADR-0003) — ADR-0005's "deterministic core + offloaded I/O" +applied one level out. Backtest-safety is consequently a **fidelity label, not a +gate**. + +## Status + +Accepted. Supersedes the strategy co-location in ADR-0001 — strategies are never +in-process with Core. + +## Considered options + +- _Strategies inside Core's deterministic fold_ (re-run on Replay) — rejected: it + would forbid the async / ML / external-data strategies that Data Providers exist + to feed, purchasing only "free" strategy replay. +- _Co-location in Core for the simple case_ (ADR-0001) — rejected and superseded: a + co-located strategy shares Core's address space and scheduler (a panic or + hot-loop hits the kernel — the exact fault isolation ADR-0001 exists for), and + being compile-time-bound it cannot be hot-pluggable, contradicting ADR-0001's own + goal and ADR-0007 (runtime-pluggable ⟺ separate process). +- _A self-declared `backtest_safe` flag_ — rejected: a claim, not a guarantee; a + "safe" strategy can still read the OS clock, so the flag's backtests cannot be + trusted. +- _Capability-derived contexts_ (chosen): a deterministic strategy binds a + `DetCtx` (injected clock, seeded RNG, `emit`) and **structurally cannot** be + nondeterministic through the framework; an effectful strategy binds an `IoCtx` + that adds ad-hoc I/O. + +## Consequences + +- **Backtest is a fidelity label.** A `DetCtx` strategy's Backtest reproducibly + matches live; an `IoCtx` strategy still backtests, but parity and freedom from + **lookahead** are the author's responsibility (the framework cannot rewind a live + API). Effectful strategies are never forbidden from backtest — only labelled + advisory. +- **The seam is the Bus even in-process.** A strategy is decoupled from Core's fold + by construction; the in-memory Bus backend is now a test/backtest device, not a + production co-location mode. +- **Enforced by construction now, physically later.** `DetCtx`/`IoCtx` is enforced + because the framework supplies no other inputs — the same discipline as ADR-0008's + read-only `StateView`. A WASM sandbox that withholds nondeterministic + host-functions is later hardening that also serves fault isolation (candidate + Extism; see ADR-0013). +- **Time and timers are always framework-injected, never ambient** — a strategy + that calls `sleep` or `SystemTime::now()` breaks Replay and Backtest. diff --git a/docs/adr/0011-execution-environments-mode-isolation.md b/docs/adr/0011-execution-environments-mode-isolation.md new file mode 100644 index 0000000..5f1b955 --- /dev/null +++ b/docs/adr/0011-execution-environments-mode-isolation.md @@ -0,0 +1,40 @@ +# Execution Environments and trading-mode isolation + +A trading **mode** is not a concept Core knows; it is the pair _(data feed × +execution backend)_ the Supervisor wires into an isolated **Environment**. The +**Simulated Broker** is just a Broker-adapter backend (ADR-0003) that fills against +the Environment's own market data, so **Backtest, Shadow, Paper, and Live are cells +of one matrix** and Core + Strategy code are **byte-identical across all of them**. +Each Environment is a fully isolated instance — its own Core, Event Log, +portfolio/risk state, execution-adapter binding, data feeds, and Bus namespace — so +several run on one host without their orders, fills, positions, or logs colliding. + +## Considered options + +- _One Core, environment-tagged messages_ — rejected: a single bug in one tag check + spends real money; tagging makes the cardinal collision (a test order reaching the + live account, a paper fill perturbing live risk) merely unlikely rather than + structurally impossible. +- _A separate Environment per running mode_ (chosen): the multi-process topology + (ADR-0001) already gives physical isolation, so a mode is just another + fault-and-safety domain. + +## Consequences + +- **The order path and account binding are never shared across Environments.** A + live and a paper Broker adapter are different processes even for the same venue, + so a paper order — published only into the paper Bus namespace — physically cannot + reach the live account. +- **An Environment is temporally homogeneous:** all its feeds share one as-of / + delay profile (real-time, delayed-by-D, or historical), and event-time processing + then aligns them (a 15-min-delayed paper feed needs its enrichment delayed to + match, e.g. via a delay relay). Feeds are shareable read-only only across + **same-profile** Environments — Live + Shadow-on-live, yes; Live + delayed-Paper, + no. +- **Shadow** = live feed × Simulated Broker, run alongside Live or Paper to test a + strategy on real-time data at zero capital risk. Because the sim fills against the + _exact_ data the strategy saw, Shadow tests _your_ model end-to-end, whereas Paper + tests the _broker's_ real execution path. +- A **single-Environment** deployment is the trivial N=1 case; the isolation + machinery is invisible when unused. +- **Replay and Snapshot are per-Environment;** only Live's Event Log is audit-grade. diff --git a/docs/adr/0012-strategy-input-fusion-event-time-parity.md b/docs/adr/0012-strategy-input-fusion-event-time-parity.md new file mode 100644 index 0000000..75f9afe --- /dev/null +++ b/docs/adr/0012-strategy-input-fusion-event-time-parity.md @@ -0,0 +1,39 @@ +# Strategy input fusion, event-time ordering, and the parity/latency model + +The strategy framework delivers a strategy's subscribed Bus topics as a single +**event-time-ordered** merged stream — mandatory for backtest↔live parity — plus a +read-only **latest-value view** (the `StateView` analogue, ADR-0008); windowed / +correlation joins are strategy-owned state for MVP. Live ordering uses a +configurable per-Environment **lateness bound `L`** (default small; `L = 0` for +latency-critical Live), so the live decision path adds no buffering latency, and +each Environment records its consumed input stream in **ingestion order** (a compact +index over the durable Bus topics) so a recorded run replays bit-exactly regardless +of `L`. + +## Considered options + +- _Raw ordered merge only_ (strategy hand-rolls all fusion) — too little; every + strategy re-rolls latest-value boilerplate. +- _Full join / CEP framework_ (windowed, temporal-correlation operators) — + premature; building Flink inside OATH. Promote specific operators only when a real + need recurs. +- _Ordered merge + latest-value view_ (chosen): the 80% case, cheaply. +- _Always strict reorder_ (buffer to completeness) — rejected: a stalled adapter + would freeze the strategy waiting for an event that may never arrive, and it adds + latency Live cannot pay. + +## Consequences + +- **Two parity goals, two mechanisms.** Reproducing a _recorded_ run is exact via + the ingestion-order log — at zero added live latency, because reproduction reads + the logged order instead of re-deriving it. Predicting from _fresh_ history uses + event-time + `L` and holds only **above `L`'s granularity**; under-setting `L` + degrades parity, surfaced as a late-event metric. +- **Late events** (arriving past their watermark) are delivered + **marked-and-counted, never silently dropped** — dropping market data is + dangerous, and a deterministic fold is never fed lossy input. A rising late rate + signals that `L` is set too low. +- The input-stream recording is **async / off-hot-path** and is _not_ ADR-0006's + log-before-send (which remains a deliberate order-path cost). +- A Strategy is structurally a **Kernel one level out**: a push-driven event-time + fold over an ordered input stream, with a read-only view, emitting into a sink. diff --git a/docs/adr/0013-strategy-runtime-push-signal-target-registration.md b/docs/adr/0013-strategy-runtime-push-signal-target-registration.md new file mode 100644 index 0000000..d7c8199 --- /dev/null +++ b/docs/adr/0013-strategy-runtime-push-signal-target-registration.md @@ -0,0 +1,53 @@ +# Strategy runtime: push framework, Signal-as-target, registration, fault isolation + +One **push-based host framework** drives every strategy — it owns the loop, the +event-time merge, the latest-value view, the ingestion log, and lifecycle; the only +per-flavour difference is which capability context is handed in (`DetCtx`, sync / +`IoCtx`, async). A **Signal** is an **idempotent** proposal of a _desired target_ +(position / exposure) for a `Symbol` — never an order — carrying the as-of freshness +it was decided under and the proposing `StrategyId`; Core nets targets across +strategies and reconciles actual → target under risk (ADR-0004). A Strategy Node +**registers** via a two-part handshake: the Supervisor performs the effectful join +(authenticate, assign a restart-stable `StrategyId`, resolve symbology, attach to +the Environment's Bus namespace) and then emits an ordered **"Strategy admitted"** +record into Core's Event Log, so the active-strategy set and per-strategy +authorization / limits are deterministic, replayable state. + +## Considered options + +- _Pull / async loop for all strategies_ — rejected: it lets a strategy interleave + I/O and own its timing, breaking determinism, and a `sleep`-based timer breaks + Replay. Push with **framework-injected timers** (a timer fires as an event in the + merge stream) keeps even time-driven strategies deterministic and backtestable. +- _Signal as an order-proposal_ ("buy 100 now") — rejected as the primitive: not + idempotent (a dropped or duplicated proposal corrupts the position), cannot be + netted across strategies, and fights the risk control loop (ADR-0004). A strategy + that thinks in orders expresses target = current + delta; raw-alpha with + Core-side portfolio construction is a possible additive Signal _kind_ later. +- _Registration as Supervisor-only control-plane state_ — rejected: attribution and + per-strategy limits are decision-relevant and must survive Replay; Supervisor-only + state cannot be replayed, and Replay could not reconstruct which strategies were + active when. + +## Consequences + +- **Two trait variants over one framework:** `fn on_event(&mut self, input, ctx: + &mut DetCtx)` (deterministic, sync — no async runtime on the hot path) and `async + fn on_event(&mut self, input, ctx: &mut IoCtx)` (effectful). Both receive the + merged event-time input + latest-value view and emit Signals into a reused sink + (ADR-0008 discipline). +- **Hot-plug is trivial:** admitting or evicting a strategy is just another folded + Core input — no Core restart (ADR-0001). This is the parked req/resp pattern's + clean instance: the join _request_ is effectful and Supervisor-only; the _fact_ + ("admitted as of event-time T") enters Core as an ordered input. +- **Fault isolation.** A deterministic strategy is never fed lossy input — a slow + one lags and its stale Signals are freshness-rejected, with the Supervisor + evicting a chronic laggard (a logged deregistration). Panics and leaks are + contained by process isolation (ADR-0001). Signal floods are largely defanged by + idempotent targets (Core coalesces to the latest target per symbol) plus a + per-strategy host rate-limit. Authorization breaches are rejected by Core's + per-strategy limits. +- **Isolation granularity is operator-tunable:** one strategy per node by default + (full isolation, independently restartable); native co-hosting as a trusted + opt-in for density (shared fate); WASM sandboxing (Extism) later for density + _with_ per-strategy isolation at 1000s-of-strategies scale. diff --git a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md index c2c9430..d48b1fd 100644 --- a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md +++ b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md @@ -1,7 +1,8 @@ # From-scratch architecture — grilling summary (2026-06-23) **Status:** Branch (A) — crate & dependency-graph revision — **closed 2026-06-24** -(ADRs 0007–0009). Branches (B) strategy runtime and (C) Frontend/CLI remain. +(ADRs 0007–0009). Branch (B) — strategy runtime & multi-topic-join — **closed +2026-06-25** (ADRs 0010–0013). Branch (C) Frontend/CLI remains. ## The question @@ -34,8 +35,8 @@ decided *not* to build — revising it is open branch (A) below. continuous autonomous control loop with cancel/amend authority, not a pre-trade gate. Strategies detect & propose Signals; Core decides & acts. - **[ADR-0005](../../adr/0005-single-writer-event-sourced-core.md)** — - Single-writer, event-sourced, deterministic Core. MVP = single-threaded kernel - + offloaded async I/O (NautilusTrader-validated); disruptor pipeline is a + Single-writer, event-sourced, deterministic Core. MVP = single-threaded kernel + + offloaded async I/O (NautilusTrader-validated); disruptor pipeline is a later, *measured* optimization. Replay = fold over the Event Log. - **[ADR-0006](../../adr/0006-broker-reconciliation-contract.md)** — Broker is the source of truth. Recovery = replay + reconcile, joined by a client order @@ -53,6 +54,29 @@ decided *not* to build — revising it is open branch (A) below. `/api` = traits, `core/` = the Core process. `oath-engine` and `oath-ingest-core` deleted; Event Log / repositories split; new `supervisor` and `cli` (Frontend) binaries. +- **[ADR-0010](../../adr/0010-strategies-out-of-core-deterministic-folds.md)** — + Strategies run only in Strategy Nodes (never in Core; supersedes ADR-0001 + co-location), wholly outside Core's deterministic boundary: a Strategy is a + deterministic fold over Bus inputs + injected clock/seed, external data via Data + Providers. Backtest-safety is a capability-derived (`DetCtx`/`IoCtx`) _fidelity + label_, not a gate. +- **[ADR-0011](../../adr/0011-execution-environments-mode-isolation.md)** — Trading + mode = _(data feed × execution backend)_; the Simulated Broker is a Broker-adapter + backend, so Backtest/Shadow/Paper/Live are one matrix and Core + Strategy are + mode-agnostic. Each mode is an isolated **Environment** (own Core, Event Log, + state, execution adapter, Bus namespace), temporally homogeneous; the order path + is never shared. +- **[ADR-0012](../../adr/0012-strategy-input-fusion-event-time-parity.md)** — + Framework delivers an event-time-ordered merge + latest-value view; per-Environment + lateness bound `L` (`L=0` live, zero added latency). Ingestion-order logging gives + bit-exact session replay; fresh backtest uses event-time + `L` (parity above `L`). + Late events marked-and-counted, never dropped. +- **[ADR-0013](../../adr/0013-strategy-runtime-push-signal-target-registration.md)** — + One push framework (`DetCtx` sync / `IoCtx` async; timers as injected events). A + Signal is an idempotent _desired target_ + freshness + `StrategyId`. Registration + is a two-part handshake: Supervisor joins effectfully, then a logged "Strategy + admitted" Core input makes the active-strategy set + limits deterministic. Fault + isolation: freshness-reject + evict laggards; one-strategy-per-node default. Glossary: **[CONTEXT.md](../../../CONTEXT.md)** (primitives, processes, messages, persistence/recovery, transport). @@ -69,9 +93,10 @@ cross-process complexity to buy crash containment and hot-pluggability. - **(A) Crate & dependency-graph revision** — ✅ **closed 2026-06-24** (ADR-0009). Spine-inverted, process-aligned topology; full target tree recorded in the ADR. -- **(B) Strategy runtime & multi-topic-join framework** — subscribing to and - *fusing* multiple streams (market data + news → Signal); the strategy↔Core - seam; strategy sandboxing contract. +- **(B) Strategy runtime & multi-topic-join framework** — ✅ **closed 2026-06-25** + (ADRs 0010–0013). Out-of-Core deterministic-fold strategies; capability-derived + backtest fidelity; Environments & mode isolation; event-time fusion + parity/`L`; + push framework, Signal-as-target, registration, fault isolation. - **(C) Frontend / CLI design** — the MVP CLI Frontend: observability + operational control (no trading control — operator orders route through Signal→risk later); query channels (req/resp to Core & Supervisor) and telemetry topics. @@ -82,9 +107,9 @@ cross-process complexity to buy crash containment and hot-pluggability. - Bus trait's **loan-vs-own** contract (design to the borrowed/lifecycle-bounded case to keep zero-copy). - Per-topic **delivery semantics** detail + backpressure / ring sizing. -- **Req/resp pattern** (registration, reconciliation queries, Frontend queries): - the request is effectful/live-only, the response enters Core as an ordered input; - Bus-capability vs side-channel TBD. +- **Req/resp pattern** (reconciliation queries, Frontend queries): the request is + effectful/live-only, the response enters Core as an ordered input; Bus-capability + vs side-channel TBD. _Registration is now a settled instance (ADR-0013)._ - **Event Log / repository backend**: parquet + DataFusion / DuckDB candidate (keep the log↔repository split from ADR-0009). - **Snapshot** cadence and contents (recovery substrate). @@ -95,3 +120,15 @@ cross-process complexity to buy crash containment and hot-pluggability. - **Deterministic client-order-id** generation scheme. - **Core failover** (Aeron-Cluster-style hot standby) — future, documented in ADR-0005. +- **Strategy sandbox** (Branch B): MVP relies on process isolation (Strategy Node) + and the capability-derived determinism contexts (`DetCtx`/`IoCtx`). Physical + WASM isolation for untrusted/effectful strategies is later hardening — candidate + runtime **Extism** (), whose host-function + capability model maps onto the `DetCtx`/`IoCtx` boundary. WASM is also the + density answer at scale: many sandboxed strategies in one process with + per-strategy fault isolation (process-per-strategy doesn't reach 1000s of + strategies on one host). +- **Simulated Broker fill model** (ADR-0011): fill-at-touch vs queue-position vs a + slippage/latency model — governs Backtest/Shadow fidelity. +- **Delay-alignment mechanism** (ADR-0011): delay-relay vs adapter delayed-mode for + aligning enrichment to a delayed market feed. From 03627bfb6f6820ed21bfd32b0f99f6ddcf572fca Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Fri, 26 Jun 2026 18:09:56 +0000 Subject: [PATCH 3/8] docs(architecture): frontend/CLI ADRs 0014-0019 + glossary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes Branch (C) of the from-scratch architecture grilling — three-plane observability (Business State / Domain Events / Telemetry) on a push-spine + narrow query; off-thread split egress; req/reply over the Bus (resolves the parked req/resp leaf); operational-only control with halt-via-risk; frontend-core library + two scopes + persistent session; host-OS trust boundary for MVP. Updates CONTEXT.md (Business State, Domain Event, Telemetry, Emergency Halt) and the grilling spec. All architecture branches now closed. Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 36 +++++++++++ ...ity-three-planes-deterministic-boundary.md | 54 ++++++++++++++++ ...-thread-split-egress-observable-outputs.md | 53 ++++++++++++++++ ...16-request-reply-over-bus-query-tiering.md | 51 ++++++++++++++++ ...frontend-control-plane-operational-only.md | 45 ++++++++++++++ ...end-architecture-scopes-library-session.md | 61 +++++++++++++++++++ ...019-frontend-trust-boundary-host-os-mvp.md | 40 ++++++++++++ ...6-23-from-scratch-architecture-grilling.md | 48 ++++++++++++--- 8 files changed, 381 insertions(+), 7 deletions(-) create mode 100644 docs/adr/0014-observability-three-planes-deterministic-boundary.md create mode 100644 docs/adr/0015-off-thread-split-egress-observable-outputs.md create mode 100644 docs/adr/0016-request-reply-over-bus-query-tiering.md create mode 100644 docs/adr/0017-frontend-control-plane-operational-only.md create mode 100644 docs/adr/0018-frontend-architecture-scopes-library-session.md create mode 100644 docs/adr/0019-frontend-trust-boundary-host-os-mvp.md diff --git a/CONTEXT.md b/CONTEXT.md index 70de151..2d243b9 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -161,6 +161,42 @@ _Avoid_: trade, transaction. A partial or complete execution of an order, reported by a broker. _Avoid_: execution, trade, transaction. +**Emergency Halt**: +An operator-tripped switch that puts Core's Risk Engine into cancel-all / flatten +mode — operational safety, not operator trading: it picks no Symbol, Side, or +Quantity, only invoking risk's existing authority (a control of the risk loop, not +an Order). The Supervisor performs the effectful trip and emits a logged Core +input, so it is deterministic and replayable. +_Avoid_: kill-switch, panic button, stop (for the process-lifecycle sense). + +## Observability + +**Business State**: +The continuous, observable state of the trading business — positions, P&L, +exposure — projected from Core's canonical fold and pushed as one coalesced, +latest-value snapshot, stamped with the Event Log sequence it reflects so +observers detect a stalled producer rather than a steady value. Rendered +directly, never re-folded. The observable subset of Core state — distinct from +the recovery Snapshot (full internal state) and from Telemetry (machinery, not +business). +_Avoid_: portfolio view, book, blotter, dashboard state. + +**Domain Event**: +A discrete, must-deliver fact Core's fold produced — order placed, fill applied, +signal admitted/rejected, breach fired/cleared, cancelled-by-risk, alert — +carried on one durable, ordered narrative stream for observers and audit. It +surfaces the outcome of a Decision as a derived fact; the Decision itself stays +internal and never reaches the Bus. Ordered, never coalesced. +_Avoid_: notification, log entry, message, decision. + +**Telemetry**: +Operational metrics of the machinery, not the business — per-topic throughput +(messages/sec), signal- and order-generation rates, latencies, queue depths, +process health. Sampled on wall-clock outside Core's deterministic fold, so it +is not canonical state and is not Event-Log sequenced. Coalescing latest-value, +like Business State, but instrumentation rather than business fact. +_Avoid_: metrics, stats, monitoring data. + ## Persistence & recovery **Event Log**: diff --git a/docs/adr/0014-observability-three-planes-deterministic-boundary.md b/docs/adr/0014-observability-three-planes-deterministic-boundary.md new file mode 100644 index 0000000..c8b5913 --- /dev/null +++ b/docs/adr/0014-observability-three-planes-deterministic-boundary.md @@ -0,0 +1,54 @@ +# Observability: three planes split across Core's deterministic boundary + +Every observer (the Frontend first) sees the running hub through **three distinct +planes**, drawn along Core's deterministic boundary. **Business State** — +positions, P&L, exposure — is the continuous observable subset of Core's +canonical fold, pushed as a single coalesced, Event-Log-seq-stamped snapshot. +**Domain Events** — order placed/filled, signal admitted/rejected, breach +fired/cleared, cancelled-by-risk, alert — are discrete facts the fold produces, +emitted on one durable, must-deliver, ordered narrative stream; each surfaces the +outcome of an internal **Decision** (ADR-0008) as a _derived_ fact, so the +Decision itself stays internal (never on the Bus) while the observable fact still +gets out. **Telemetry** — per-topic throughput, signal-/order-generation rates, +latencies, queue depths, process health — is wall-clock instrumentation _outside_ +the fold, self-reported per process (Core, each Adapter, each Strategy Node; +Supervisor adds health), never seq-stamped. The read path is **push-spine** +(observers subscribe) with a **narrow query** escape hatch; observers render all +three planes directly and **never re-fold**. + +## Considered options + +- _Pull / query as the primary read path_ — rejected: it puts a query-serving + obligation on Core's single-writer hot path for the steady-state dashboard. + Push keeps observers off the hot path (Core already emits its outputs); query + stays a rare, deliberately-narrow escape hatch. +- _Reuse raw operational topics for discrete events_ (the observer stitches its + own timeline from Core→broker orders, inbound fills on the Event Log, and the + trapped reject/breach Decisions) — rejected: the sources differ in direction + and semantics, and Decisions never hit the Bus at all, so every Frontend would + reverse-engineer an internal timeline. That is not a public contract; it is a + standing invitation to drift, and it breaks the "depends only on the public + message model" boundary (CONTEXT: Frontend) the moment a second observer exists. +- _One read-model projector from day one_ — deferred: re-folding canonical state + in a second process is the two-sources-of-truth trap, unjustified at MVP. The + durable Domain-Event stream is precisely the substrate such a projector tails + later, at no new emission cost. +- _One "telemetry" notion for all of it_ — rejected: it crosses the deterministic + boundary. Business State and Domain Events are products of the fold (replayable, + seq-stamped / ordered); Telemetry is wall-clock instrumentation that must never + contaminate canonical state. + +## Consequences + +- **Core gains an explicit emission obligation:** project Business State and emit + Domain Events as derived facts of its fold — a public contract, versioned with + the canonical message model. +- **Two authoritative records, two halves.** The Event Log is _what Core saw_ + (inputs); the Domain-Event stream is _what Core did_ (narrative). Together they + are the forensic / audit substrate. +- **Telemetry is per-process and out-of-fold**, so Core is never made omniscient + about processes it cannot see; throughput / health come from the processes + themselves (Supervisor aggregates health). Telemetry loss is acceptable; + Business State coalesces (latest wins); Domain Events must not be lost. +- The mechanism that delivers Business State, Domain Events, and query responses + **without touching the writer thread** is ADR-0015. diff --git a/docs/adr/0015-off-thread-split-egress-observable-outputs.md b/docs/adr/0015-off-thread-split-egress-observable-outputs.md new file mode 100644 index 0000000..1e38fc0 --- /dev/null +++ b/docs/adr/0015-off-thread-split-egress-observable-outputs.md @@ -0,0 +1,53 @@ +# Off-thread, split egress for Core's observable outputs + +Core's observable outputs — ADR-0014's **Business State** and **Domain Events**, +plus **query responses** — leave the Kernel with **no serialization or publish on +the single-writer thread**: the Kernel writes into in-process egress structures +and a small **non-blocking forwarder** thread (compile-time static, in the Core +process per ADR-0007) drains, serializes, and publishes to the Bus. Egress is +**split by delivery need**, which makes memory-boundedness _structural_ rather +than hoped-for: + +- **Business State → one coalescing latest-value slot.** A single atomic + `StateView` snapshot the Kernel overwrites (O(1), never blocks). One slot, not + per-field — so positions / P&L / exposure are never a torn cross-section of what + is canonically one fold. **Seq-stamped** with the input it reflects, so a frozen + sequence reads as a _stalled producer_, not a steady value. +- **Domain Events → a must-deliver, ordered, durable channel.** Never coalesced — + a heartbeat must never clobber a breach. +- **Query responses → a must-deliver, admission-bounded channel** keyed by + request-id. Depth-tiny (≈1 query outstanding at CLI rates); overflow **refuses + admission** of a new query, never drops a response. + +The enqueue is **unconditional** and the writer-thread code path is +**byte-identical live vs Replay** — under Replay the forwarder is wired to a null +sink — so the fold stays pure and no `if replay` branch sits on the hot path. + +## Considered options + +- _Kernel serializes + publishes inline_ — rejected: serialization and Bus I/O on + the single-writer thread is exactly the hot-path contention ADR-0005 offloads. + Egress is just another offloaded I/O. +- _One unified egress queue, one policy_ — rejected: any single policy is wrong + for one payload. Coalescing drops Domain Events; a must-deliver FIFO lets a + state/telemetry burst grow unbounded or evict a pending query answer. The split + gives each payload its correct, independently-bounded policy. +- _Per-field Business State slots_ (separate slots for positions, P&L, exposure) — + rejected: independent slots can publish a torn cross-section (new position, + stale P&L). One atomic snapshot is simpler and consistent. +- _Drop query responses on overflow_ — rejected: a human asked; the answer cannot + be silently coalesced away. Bound _admission_ instead, so "must-deliver" stays + honest. + +## Consequences + +- The writer thread's only egress cost is a cheap **enqueue / overwrite of an + owned compact value**; serialization happens on the forwarder. +- **Business State is lossy-by-design** (latest wins) and that is _correct_ — + observers want latest; the seq stamp distinguishes "steady" from "stalled." +- **Telemetry does not use this path** (ADR-0014): it is out-of-fold and + self-reported per process, with its own publish path. This egress is for + fold-products only. +- The coalescing slot and the durable Domain-Event channel are the natural inputs + for a future read-model projector (ADR-0014) — state side and event side + respectively — with no new Core emission. diff --git a/docs/adr/0016-request-reply-over-bus-query-tiering.md b/docs/adr/0016-request-reply-over-bus-query-tiering.md new file mode 100644 index 0000000..75b64bf --- /dev/null +++ b/docs/adr/0016-request-reply-over-bus-query-tiering.md @@ -0,0 +1,51 @@ +# Request/reply over the Bus; query tiering and read-isolation + +The Frontend's narrow query escape hatch (ADR-0014) and reconciliation queries +(ADR-0006) are served by **request/reply implemented as a thin correlation layer +over the Bus's pub/sub** — a request topic per responder (`core.query.req`, +`supervisor.control.req`, `repo.query.req`) and a reply topic keyed by a +request-id — **not** by a separate side-channel transport. The Bus trait gains no +req/reply method; correlation, reply-to, timeout, and retry live in one +backend-agnostic helper _above_ the trait, so every Bus backend (iceoryx2 local, +RabbitMQ / Kafka networked) carries req/reply for free over a **single backend +matrix**. Queries are **tiered by target**: the push-spine (Business State +snapshot + Domain-Event stream) answers the steady state with no round-trip; live +detail beyond the snapshot is a **non-logged read** the Kernel serves between +inputs (seq-stamped to the input it read at); historical / voluminous data (full +order / fill history) is served by the **repository / Event-Log store, entirely +off Core** (ADR-0009 log↔repository split). A **Frontend read query never enters +the Event Log** — only a reconciliation _response_, which Core folds into state, +does (ADR-0006). + +## Considered options + +- _A separate side-channel transport for req/reply_ — rejected: a second + transport trait with its own backend matrix, kept in lockstep with the Bus, + plus a bespoke network implementation the moment the Bus is networked. It + doubles transport maintenance merely to keep the Bus trait tidy, and a + local-socket channel would pin the Frontend to Core's host. +- _A native req/reply method on the Bus trait_ — rejected: request-id correlation + is supported very unevenly across backends; a trait method bloats the trait and + constrains backend choice. Built over pub/sub, backends only ever implement + pub/sub. +- _Logging Frontend read queries as Core inputs_ — rejected: a human's CLI reads + would alter the deterministic input stream and change Replay. Reads are pure and + non-logged; the parked req/resp note conflated this with reconciliation, whose + response genuinely is a folded input. + +## Consequences + +- **Must-deliver without durable reply topics:** requester timeout + retry over a + (possibly lossy) reply topic gives effective must-deliver, pairing with + ADR-0015's responder-side admission-bounding — so a query reply needs no durable + Bus topic. +- **Uniform backend-agnosticism:** swap to a networked Bus and the same Frontend + observes a remote host with no extra work; the query interface inherits the + Bus's reach. +- **The control plane reuses this verbatim** (`supervisor.control.req`): + operational commands are req/reply too, so they ride the same correlation layer + — no new mechanism. +- **Logical / physical split:** "query interface" stays a distinct logical API + (`request → response`) the Frontend programs against, decoupled from pub/sub but + physically just Bus messages — honoring the glossary's "Bus, and query + interfaces." diff --git a/docs/adr/0017-frontend-control-plane-operational-only.md b/docs/adr/0017-frontend-control-plane-operational-only.md new file mode 100644 index 0000000..ef4db69 --- /dev/null +++ b/docs/adr/0017-frontend-control-plane-operational-only.md @@ -0,0 +1,45 @@ +# Frontend control plane: operational-only, kill-switch via risk authority + +The Frontend's control half is **operational, not trading**. Lifecycle commands — +start / stop / restart an Adapter or Strategy Node, start / stop an Environment, +admit / evict a Strategy — go to the **Supervisor** over `supervisor.control.req` +(ADR-0016) and touch only topology, never the order path. The operator **cannot** +place, cancel, or amend individual orders or set targets; discretionary operator +trading is deferred to a future **Signal→risk seam** (the operator's order becomes +a Signal, decided by Core under risk like any other — ADR-0004 / ADR-0013), so no +path ever bypasses risk. The one order-affecting control in MVP is an **emergency +halt**, modeled not as operator trading but as a **trip of the Risk Engine's +existing cancel-all / flatten authority** (ADR-0004): the operator picks no +instrument and no size, only trips the switch. Mechanically it follows ADR-0013's +registration template — the Supervisor performs the effectful trip, then emits a +logged "halt as-of seq N" fact into Core's Event Log — so the halt is +deterministic, replayable, and attributable. + +## Considered options + +- _Kill-switch as direct order cancellation by the operator_ — rejected: that is + operator trading control (it issues order actions), bypasses the risk loop that + owns cancel / amend / flatten authority (ADR-0004), and adds a second, unlogged + actor on the order path against ADR-0008's single-actor Kernel. +- _Kill-switch out of MVP (Supervisor-only control)_ — rejected: a process halt + leaves resting orders live at the broker and positions unmanaged — more + dangerous, not less. A system that trades real money must be able to flatten + before it can be called done; observability without a safety stop is the wrong + order of priorities. +- _Operator discretionary orders in MVP_ — deferred (not rejected): routing + operator orders through Signal→risk is the correct eventual design, but it + depends on the Signal admission path and is outside the Frontend MVP's scope. + +## Consequences + +- **No path bypasses risk.** Every order-affecting action — strategy Signals and + the operator's emergency halt alike — is decided and performed by Core's Kernel + under risk; the Frontend never emits Orders. +- **One control mechanism.** Lifecycle and halt are both req/reply to the + Supervisor (ADR-0016); halt additionally produces a logged Core input via the + ADR-0013 template. No new transport or pattern. +- **Halt is auditable and replayable** like admission: "what was halted, as of + which seq" is reconstructable from the Event Log. +- **The Signal→risk seam is the single extension point** for all future trading + control (operator orders, manual target overrides), keeping the Frontend's + control surface from ever growing its own order path. diff --git a/docs/adr/0018-frontend-architecture-scopes-library-session.md b/docs/adr/0018-frontend-architecture-scopes-library-session.md new file mode 100644 index 0000000..2a11404 --- /dev/null +++ b/docs/adr/0018-frontend-architecture-scopes-library-session.md @@ -0,0 +1,61 @@ +# Frontend architecture: two observation scopes, library/presentation split, session model + +The Frontend is factored as a reusable **`frontend-core` library** — Environment +discovery, namespace attach/**switch**, the three-plane consumers (Business State, +Domain Events, Telemetry), and the query/control clients — with **thin +presentations** on top (the `cli` binary first; `tui` / `web` later reuse the same +library, differing only in how they present and switch). It observes through **two +scopes**: + +- **Host scope** — an always-on connection to the **Supervisor**: Environment + discovery, process health, topology, and lifecycle + halt control (ADR-0017). +- **Environment scope** — one Core's **Bus namespace**: that Environment's + Business State, Domain Events, Environment Telemetry, and queries. + +There is **one host Bus** with a namespace per Environment (ADR-0011), so the +Frontend makes one Bus connection and filters by namespace; **switching +Environment is a first-class interaction** (detach namespace A, attach namespace +B — the Supervisor connection persists). The MVP `cli` is a **persistent +interactive session** (live three-plane view; in-session commands including +`env ` to switch and `halt`) and *also* exposes **one-shot subcommands** for +scripting — both over the one library. Discovery is **Supervisor-driven** (it +spawned the Environments). MVP attaches to **one Environment at a time**; +multi-Environment side-by-side (e.g. Shadow-vs-Live) is a later additive view, the +capability (subscribe to N namespaces) already present. + +## Considered options + +- _One-shot CLI only for MVP_ — rejected: a push-spine observability tool that + cannot hold a live view underuses the telemetry/state design, and the live + session would be net-new work for the TUI instead of an evolution of the CLI. + One-shot is kept for scripting, not as the only mode. +- _CLI as a monolith (no shared library)_ — rejected: every later Frontend (TUI, + web) would re-implement discovery, switching, and the plane consumers. The + library/presentation split is what the glossary's "CLI is the first Frontend; + TUIs/web later" implies. +- _Per-Environment Bus instances_ — rejected (already by ADR-0011): one host Bus, + namespaced, lets the Frontend connect once; a faster-than-real-time Backtest + namespace cannot drown Live, and the coalescing snapshot (ADR-0015) absorbs the + flood. +- _Config-file Environment discovery_ — rejected: it duplicates the Supervisor's + authoritative topology and drifts. The Supervisor spawned them, so it answers + "what is running." +- _Multi-Environment aggregation in MVP_ — deferred: real value for Shadow-vs-Live, + but additive; the multi-namespace subscription is available when the combined + view is built. + +## Consequences + +- **Switching is cheap and central:** a switch is namespace re-subscription inside + `frontend-core`; presentations expose it however suits them (typed command, + dropdown). The host-scope Supervisor connection is stable across switches. +- **Two scopes, two cadences:** host scope (health / topology / discovery) is + low-rate and always present; Environment scope is the high-rate three-plane + stream for the attached Environment only — no cross-Environment contamination, + honoring ADR-0011 isolation. +- **The library is the contract surface:** future Frontends depend on + `frontend-core` (and through it the public message model + Bus + query + interface), never on Core internals (CONTEXT: Frontend). +- **`frontend-core` is a small addition to ADR-0009's tree** (a lib crate beside + `cli`), preserving the dependency direction — it depends only on the public + message model, Bus, and query interface. diff --git a/docs/adr/0019-frontend-trust-boundary-host-os-mvp.md b/docs/adr/0019-frontend-trust-boundary-host-os-mvp.md new file mode 100644 index 0000000..8540227 --- /dev/null +++ b/docs/adr/0019-frontend-trust-boundary-host-os-mvp.md @@ -0,0 +1,40 @@ +# Frontend trust boundary: host-OS for MVP, auth deferred to the networked Bus + +For MVP the Frontend's security boundary is the **host OS**: the Bus transport's +local access controls (shared-memory segment / socket file permissions) are the +trust boundary — any process the operator can run on the single host is trusted, +and **no application-level authentication or authorization is built**. On a +single-operator host this is the *real* boundary: anyone who can reach the +shared-memory segment can read process memory directly, so app-level auth over +localhost would be theater. The seams are **reserved, not built** — an +identity/principal field is kept on the control req/reply (ADR-0016) so +authorization can be added without a wire-format break — and control is **already +auditable**: an emergency halt is a logged Event Log input (ADR-0017) and +lifecycle commands go through the Supervisor. Authentication (tokens / mTLS) and +authorization tiering (observe-only vs control vs admin) land **together with the +networked-Bus capability** — the point at which the threat model changes from "my +own host" to "the network," and at which a networked Bus backend brings its own +transport security anyway. + +## Considered options + +- _App-level auth from day one_ (shared secret / token on control) — rejected for + MVP: theater over localhost shared-memory, where the OS already gates access and + an attacker with segment access has memory access. Its one real benefit — + exercising the auth seam early — is bought instead by reserving the principal + field, without paying for a mechanism that guards nothing yet. +- _No reserved seam at all_ — rejected: authorization must be addable without + breaking the control wire format, so the principal field is reserved now even + though unused. + +## Consequences + +- **The single-trusted-operator-host assumption is explicit** and must be + revisited the instant the Bus is networked or the host becomes multi-user. +- **Auditability is independent of authentication:** halt and lifecycle are + recorded (Event Log / Supervisor) whether or not a principal is yet + authenticated — so "what happened" is always answerable; "who" arrives with + authn. +- **Adjacent, but not security:** a confirmation guard on destructive ops + (`halt Live` → typed confirm) is a CLI-UX safety feature; encryption of + sensitive Business State is moot on localhost and arrives with the networked Bus. diff --git a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md index d48b1fd..52d01e8 100644 --- a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md +++ b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md @@ -2,7 +2,8 @@ **Status:** Branch (A) — crate & dependency-graph revision — **closed 2026-06-24** (ADRs 0007–0009). Branch (B) — strategy runtime & multi-topic-join — **closed -2026-06-25** (ADRs 0010–0013). Branch (C) Frontend/CLI remains. +2026-06-25** (ADRs 0010–0013). Branch (C) — Frontend/CLI — **closed 2026-06-26** +(ADRs 0014–0019). **All architecture branches closed.** ## The question @@ -77,6 +78,35 @@ decided *not* to build — revising it is open branch (A) below. is a two-part handshake: Supervisor joins effectfully, then a logged "Strategy admitted" Core input makes the active-strategy set + limits deterministic. Fault isolation: freshness-reject + evict laggards; one-strategy-per-node default. +- **[ADR-0014](../../adr/0014-observability-three-planes-deterministic-boundary.md)** — + Observability is three planes split on Core's deterministic boundary: **Business + State** (continuous, in-fold, seq-stamped coalescing snapshot) + **Domain Events** + (discrete, in-fold, durable curated narrative; Decisions stay internal, surfaced + as derived facts) + **Telemetry** (technical, out-of-fold, per-process). Read + path = push-spine + narrow query; observers never re-fold. +- **[ADR-0015](../../adr/0015-off-thread-split-egress-observable-outputs.md)** — + Off-thread, split egress: the Kernel enqueues, a non-blocking forwarder + publishes. Coalescing latest-value snapshot (one atomic slot, seq-stamped) + + must-deliver durable Domain-Event channel + admission-bounded query channel. + Memory-boundedness is structural; replay path byte-identical (null sink). +- **[ADR-0016](../../adr/0016-request-reply-over-bus-query-tiering.md)** — + Req/reply is a thin correlation layer over the Bus's pub/sub (no side-channel, no + native Bus method); one backend matrix. Queries tiered: push-spine → Kernel + non-logged read → repository / Event-Log store. Frontend reads never hit the + Event Log (only reconciliation responses do). +- **[ADR-0017](../../adr/0017-frontend-control-plane-operational-only.md)** — + Frontend control is operational-only (lifecycle to Supervisor); no operator + trading (deferred to a Signal→risk seam). The one order-affecting control is an + emergency halt, modeled as a logged trip of risk's existing flatten authority. +- **[ADR-0018](../../adr/0018-frontend-architecture-scopes-library-session.md)** — + Frontend = reusable `frontend-core` library + thin presentations (CLI first, + TUI/web later). Two scopes: host (always-on Supervisor) + Environment (one Core + namespace). Persistent interactive session + one-shot subcommands; one + Environment at a time; switch is first-class; discovery is Supervisor-driven. +- **[ADR-0019](../../adr/0019-frontend-trust-boundary-host-os-mvp.md)** — + MVP trust boundary = host OS (Bus segment/socket perms); no app-level auth. + Identity/principal seam reserved; control already audited. Authn + authz tiering + arrive with the networked Bus, where the threat model actually changes. Glossary: **[CONTEXT.md](../../../CONTEXT.md)** (primitives, processes, messages, persistence/recovery, transport). @@ -97,9 +127,11 @@ cross-process complexity to buy crash containment and hot-pluggability. (ADRs 0010–0013). Out-of-Core deterministic-fold strategies; capability-derived backtest fidelity; Environments & mode isolation; event-time fusion + parity/`L`; push framework, Signal-as-target, registration, fault isolation. -- **(C) Frontend / CLI design** — the MVP CLI Frontend: observability + operational - control (no trading control — operator orders route through Signal→risk later); - query channels (req/resp to Core & Supervisor) and telemetry topics. +- **(C) Frontend / CLI design** — ✅ **closed 2026-06-26** (ADRs 0014–0019). + Three-plane observability (Business State / Domain Events / Telemetry) on a + push-spine + narrow query; off-thread split egress; req/reply over the Bus; + operational-only control with halt-via-risk; `frontend-core` library + two + scopes + persistent session; host-OS trust boundary for MVP. ## Parked sub-questions (don't lose these) @@ -107,9 +139,11 @@ cross-process complexity to buy crash containment and hot-pluggability. - Bus trait's **loan-vs-own** contract (design to the borrowed/lifecycle-bounded case to keep zero-copy). - Per-topic **delivery semantics** detail + backpressure / ring sizing. -- **Req/resp pattern** (reconciliation queries, Frontend queries): the request is - effectful/live-only, the response enters Core as an ordered input; Bus-capability - vs side-channel TBD. _Registration is now a settled instance (ADR-0013)._ +- ~~**Req/resp pattern**~~ — **resolved (ADR-0016)**: req/reply is a thin + correlation layer over the Bus (not a side-channel); Frontend read-queries are + non-logged and tiered (push-spine → Kernel read → repository), while only a + reconciliation _response_ enters Core as an ordered input. _Registration was a + settled instance already (ADR-0013)._ - **Event Log / repository backend**: parquet + DataFusion / DuckDB candidate (keep the log↔repository split from ADR-0009). - **Snapshot** cadence and contents (recovery substrate). From 438cbe0957764d5dedf3e4af7a39673ea1f9fe9f Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Sat, 27 Jun 2026 13:40:46 +0000 Subject: [PATCH 4/8] docs(architecture): close Bus-contract cluster (ADRs 0020-0022) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Close the Bus-contract parked-question cluster from the from-scratch architecture grilling. - ADR-0020: Bus trait — two delivery classes (LatestValue keyed store / Reliable stream) x two access patterns; Serialize universal bound (zero-copy/POD a backend capability); loan-guard receive + three-boundary copy-out; publish(&M) + opt-in loan-to-write; (namespace, type, role, instance-key) identity with typed Key structs. Refines ADR-0002. - ADR-0021: two-level routing (coarse backend shard + in-process demux). - ADR-0022: Reliable order-path graduated failure; drain-fast adapter capability; risk-trip vs channel-death halt split. Amends ADR-0006/0017. - docs/design/bus-backend-realization.md: per-backend mechanism reference. - CONTEXT.md: add Source, LatestValue, Reliable; clarify Symbol. - ADR-0002: refinement pointer; spec: mark cluster closed. Co-Authored-By: Claude Opus 4.8 --- CONTEXT.md | 27 ++++++- ...nd-agnostic-bus-canonical-message-model.md | 5 ++ ...-trait-delivery-classes-access-patterns.md | 80 +++++++++++++++++++ ...0021-two-level-routing-in-process-demux.md | 44 ++++++++++ ...2-reliable-order-path-graduated-failure.md | 52 ++++++++++++ docs/design/bus-backend-realization.md | 47 +++++++++++ ...6-23-from-scratch-architecture-grilling.md | 15 +++- 7 files changed, 265 insertions(+), 5 deletions(-) create mode 100644 docs/adr/0020-bus-trait-delivery-classes-access-patterns.md create mode 100644 docs/adr/0021-two-level-routing-in-process-demux.md create mode 100644 docs/adr/0022-reliable-order-path-graduated-failure.md create mode 100644 docs/design/bus-backend-realization.md diff --git a/CONTEXT.md b/CONTEXT.md index 2d243b9..bcf747a 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -8,9 +8,20 @@ trading engine. This file is a glossary only — no implementation details. **Symbol**: OATH's canonical identifier for a tradable instrument, independent of any one venue's ticker or internal id, so the same instrument offered by different -brokers collapses to a single `Symbol` (e.g. via perm_id / OpenFIGI). +brokers collapses to a single `Symbol` (e.g. via perm_id / OpenFIGI). `Symbol` +is instrument _identity_ — used for positions, Signals, and risk — and is **not** +a data-stream routing key: market-data streams are additionally keyed by their +[Source], because the same `Symbol` priced by two Sources is two distinct streams. _Avoid_: ticker, instrument, contract (for the canonical form). +**Source**: +The Broker or Data-Provider Adapter that produced a given data stream — part of a +market-data topic's routing key, never part of instrument identity. The same +`Symbol` carried by two Sources is two distinct streams (different prices, +timestamps, and gaps); a consolidated/NBBO view is a _derived_ stream, never raw +per-Source topics conflated. +_Avoid_: venue, feed, provider (for this routing-key role). + **Price**: The value per unit of an instrument, expressed in its quote currency. _Avoid_: cost, rate, level. @@ -231,3 +242,17 @@ _Avoid_: simulation, paper trading, replay (for this). The transport over which processes exchange canonical messages. Backend-agnostic (e.g. shared-memory zero-copy, Unix sockets, Kafka) behind a single trait. _Avoid_: queue, channel, broker (reserved for the venue role above). + +**LatestValue**: +A Bus delivery class: a **keyed store** of the latest value(s) per instance-key +(depth ≥ 1), lossy and overwrite-allowed and **per-key isolated**, so a busy key +never starves a quiet one. Read **by key** (with change-notification), never as a +filtered firehose. Models current price/quote, order-book snapshot, position/P&L. +_Avoid_: cache, snapshot (reserved for recovery), drop-to-latest (as a noun). + +**Reliable**: +A Bus delivery class: an **ordered stream** in which no message is silently +dropped — a full queue yields an explicit error, never a block and never a silent +loss. Models every tick/fill/order/Domain Event. +_Avoid_: durable (durability is the [Event Log]'s concern, not the Bus's), +guaranteed. diff --git a/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md b/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md index cd45d0c..7fe9064 100644 --- a/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md +++ b/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md @@ -6,6 +6,11 @@ fast path; Unix sockets or Kafka for other users). There is exactly **one canonical message model** shared by every backend — backends differ only in how they move bytes, never in the data they carry. +> _Refined by **ADR-0020**: the universal type bound is `Serialize` (POD is a +> backend-specific zero-copy discipline, not a trait requirement), and the +> per-topic delivery semantics below are realized as two classes (`LatestValue` / +> `Reliable`) × two access patterns (keyed store / stream)._ + ## Considered options - *Per-backend data models* — rejected: it recreates, one layer up, the diff --git a/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md b/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md new file mode 100644 index 0000000..6b377c2 --- /dev/null +++ b/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md @@ -0,0 +1,80 @@ +# Bus trait: two delivery classes × two access patterns over one canonical model + +The Bus (ADR-0002) is one trait carrying a family of typed canonical messages, and +it offers exactly **two delivery classes**, each with its own **access pattern**: + +- **`LatestValue` — a keyed store.** Lossy / overwrite-allowed, **per-instance-key + isolated**: each key owns a slot of depth `N ≥ 1` (default 1; `N > 1` is a lossy + recent window). Accessed **read-by-key (+ change-notification)**, never as a + filtered firehose — so a high-volume key can never evict ("starve") a low-volume + key's latest, and a slow reader misses intermediate values but always reads the + current one. Publishing never blocks. Models current price/quote, order-book + snapshot, position/P&L. +- **`Reliable` — an ordered stream.** No message is silently dropped; on a full + bounded queue `publish` returns an explicit error (never blocks). Accessed + **subscribe/receive**. Models every tick/fill/order/Domain Event. + +**Bounds & ownership.** The **universal type bound is `Serialize`** plus `Message` +metadata (its `Key`, role, delivery class). **Zero-copyability is a backend +capability, not a trait requirement** — iceoryx2 concretizes it as `#[repr(C)]` POD +(`ZeroCopySend`); other zero-copy transports impose their own layout discipline. +OATH's *own* canonical messages additionally satisfy POD + `Serialize` by +convention so they ride every backend including the fast path; user-defined +messages need only `Serialize` (and opt into a backend's discipline for its fast +path). **Receive is a loan guard** (`Sample: Deref`, drop returns +the slot) — a real shared-memory loan where the backend supports it, an owned value +otherwise. A loan **may not cross three boundaries** — retention, thread hand-off, +`.await` — at each of which the consumer copies out to **owned `M: Copy` POD** and +drops the loan. (Core's drain — read field, append bytes to the Event Log, fold, +drop — is the inspect-and-discard hot path that earns the guard its public place.) +**Send is `publish(&M)`**; **loan-to-write (`SampleMut`) is an opt-in** +zero-copy-construct path for large/bulk payloads (order-book depth, history pages). + +**Identity & routing.** A topic is `(Environment namespace, payload type, role, +instance-key)`. `role` is the QoS-bearing channel (e.g. `bars.live` vs +`bars.history`); the **shard is *not* an identity coordinate** — it is derived from +the instance-key by the backend. The **instance-key is a typed, per-type `Key` +struct** (`QuoteKey { source, symbol }`, `BarKey { source, symbol, interval }`, +`FillKey { source }`, `OrderKey { account }`, …) with a canonical, escape-safe wire +encoding produced once inside the mapping layer — never hand-concatenated at call +sites. **Key granularity = consumer-interest granularity**: `LatestValue` stores +are read by fine `Symbol`-level keys; whole-consumed `Reliable` streams (fills, +orders, events) collapse to coarse `(Source/role)` keys; a per-`Symbol` `Reliable` +stream keeps the `Symbol`. A **per-key sequence stamp lives in the payload**, so +gap-detection is identical whether a key has its own channel or shares one. + +**Excluded — bootstrapping is not a Bus concern.** Acquiring state established +before a participant was listening (late-join) is the same question as crash +recovery, answered by the recovery/query substrate (ADR-0005 / 0006 / 0016) — never +by a "history" delivery knob. + +## Considered options + +- _One canonical envelope enum on shared topics_ — rejected: forces max-variant + slot size across a C ABI and prevents per-topic delivery semantics. Chosen: a + family of typed POD topics + an open, user-extensible `Message` trait (a category + enum is permitted as just another `M`). +- _`Pod + Serialize` as the universal bound_ (ADR-0002's phrasing) — refined: + `Serialize` is universal; POD is one backend's zero-copy discipline. +- _One delivery class delivered as a filtered stream_ — rejected: a shared lossy + ring lets a fast key starve a slow one, so "subscribe to AAPL, never see AAPL" + becomes possible. Chosen: `LatestValue` is a per-key-isolated keyed store, read + by key. +- _Owned copy-out as the public receive API_ — rejected: forecloses zero-copy on + Core's hot path. Chosen: the loan guard (degrading to owned) + mandated copy-out + at the three boundaries. + +## Consequences + +- **Refines ADR-0002**: `Serialize` is the universal bound (POD backend-specific); + per-topic delivery becomes two classes × two access patterns; the "stricter + ownership contract" is the loan guard, the layout discipline is not. +- **Delta order books fold in the *adapter*** (ADR-0003 anti-corruption): the + adapter applies venue deltas to a canonical book and publishes **snapshots** as + `LatestValue`; consumers read the current book and never see the venue's delta + protocol. A raw-delta `Reliable` stream is an opt-in. +- **No backend leaks into the contract** except the deliberate loan-guard exposure: + sharding, the keyed-store implementation (blackboard / compacted topic / storage + / cache), ring sizing, and delta math live below the trait — see + [the backend-realization note](../design/bus-backend-realization.md). +- **`Reliable` routing is ADR-0021; `Reliable` order-path failure is ADR-0022.** diff --git a/docs/adr/0021-two-level-routing-in-process-demux.md b/docs/adr/0021-two-level-routing-in-process-demux.md new file mode 100644 index 0000000..ab1c70d --- /dev/null +++ b/docs/adr/0021-two-level-routing-in-process-demux.md @@ -0,0 +1,44 @@ +# Two-level message routing: coarse backend shard + in-process demux + +Per-key routing happens at **two levels**. The **backend** delivers at a coarse, +cost-effective granularity (shards on iceoryx2 / Aeron / Kafka, native +key-expressions on Zenoh); a uniform **in-process demux** in the consumer runtime +then fans those messages out to per-key subscribers at memory speed +(`hashmap: key → [subscribers]`). The Bus trait and `subscribe(key)` API are +**unchanged** — consumers never see the demux. This applies to **`Reliable` +streams**; `LatestValue`'s keyed store (ADR-0020) is already read-by-key and needs +no demux. + +Cross-process selectivity stays with the backend (do **not** firehose every +process — it is expensive on Aeron / Kafka / networked Zenoh and wakeup-heavy even +on iceoryx2, and it discards the selectivity the network backends need). Fine +fan-out to multiple *same-process* consumers becomes a hashmap lookup instead of K +backend subscriptions. The demux is a **no-op at one-consumer-per-process** (the +MVP one-strategy-per-node default, where it degenerates to the existing sub-side +filter) and **load-bearing for the many-consumers-per-process future** (WASM-density +strategies, ADR-0013's parked sandbox question). + +## Considered options + +- _Native backend per-key routing everywhere_ — rejected as the default: routing + config becomes backend-specific and tier-dependent, whereas in-process routing is + one testable module with identical code on every backend. (A networked + deployment may still use finer native routing where bandwidth matters — the API + is unchanged.) +- _Firehose to every process + pure in-process routing_ — rejected: loses + cross-process selectivity; every process pays for the whole market. Chosen: + coarse shard (level 1) + in-process demux (level 2). The LMAX/exchange + "firehose → in-process dispatch" pattern is just this with level 1 set to one + coarse shard — a tunable special case for a single-host deployment, not the + general rule. + +## Consequences + +- **Zero-copy preserved where free, paid where it buys isolation.** Same-thread + fan-out and K = 1 filtering are zero-copy; cross-thread fan-out to *isolated* + consumers copies out per consumer — the thread-hand-off boundary of ADR-0020, + and the price of stopping one slow strategy from stalling its neighbours. +- **The demux is the home of per-key state for same-process consumers**: it holds + `LatestValue` per-key slots and per-subscriber `Reliable` queues. +- **A truly point-to-point transport** would put fan-out in its adapter + (relay / MDC), still below the trait. diff --git a/docs/adr/0022-reliable-order-path-graduated-failure.md b/docs/adr/0022-reliable-order-path-graduated-failure.md new file mode 100644 index 0000000..1f33767 --- /dev/null +++ b/docs/adr/0022-reliable-order-path-graduated-failure.md @@ -0,0 +1,52 @@ +# Reliable order-path failure: graduated ladder, two halt modes, drain-fast adapter + +When Core cannot publish an Order because the `Reliable` order topic is full +(ADR-0020), the response is a **graduated ladder**, not an immediate global halt: +**normal → telemetry-visible pressure → degraded/probing → scoped execution-path +safe-hold**. The first `Err(Full)` puts Core into a *degraded* state — a fold +state-transition, **never a blocked Kernel thread**: it withholds new Orders, fires +an **async health-probe** to the adapter (on a control topic, so it sends even +while the order ring is full), and waits in-state — bounded by an injected-clock +timer — for the ring to drain or the probe to reply. Only on confirmed timeout does +it declare the channel dead and enter a **scoped safe-hold** (stop emitting for +that one execution path, freeze, alert; the rest of the host runs on). On recovery, +held Orders are **freshness-re-checked** against their validity context before +sending — a stale Order is **dropped by Core's decision**, which is not a transport +drop and does not violate `Reliable`. + +This rests on a **mandatory broker-adapter capability** (amending ADR-0006): the +order adapter **drains the Bus fast into its own internal, rate-limited pending +buffer** and paces venue submission from there — so a full Bus ring reflects +*adapter liveness*, not venue rate-limiting, making "full ⇒ wedged" honest. + +It also **splits ADR-0017's halt into two modes**, because an overflow halt would +flatten through the very channel that is wedged: + +- **Risk-trip halt** (broker reachable): the ADR-0017 emergency-halt-via-risk works + — cancel/flatten through the live order path. +- **Channel-death halt** (the cancel path is itself the fault): cannot flatten — it + can only stop-emitting, freeze, alert, and **defer** flatten/reconcile to + reconnect-reconcile (ADR-0006). The halt protocol must distinguish the two; their + available actions differ. + +## Considered options + +- _Immediate global halt on overflow_ — rejected: a full queue is local to one + (Environment × broker), not a host-wide panic, and an unconfirmed `Err(Full)` may + be transient — hence the probe. +- _Bounded retry / spin on the Kernel thread_ — rejected: blocks the single-writer + and keeps generating un-sendable Orders. +- _A dedicated order fault channel_ — rejected: relocates the bound and adds a + second order-delivery path that complicates reconciliation (ADR-0006). + +## Consequences + +- **The Kernel never blocks**; "wait" is a fold state, the probe is async (its + reply/timeout are logged Core inputs), so the whole ladder is deterministic and + replayable. +- **Telemetry is the early-warning plane** (ADR-0014): per-topic queue depth and + fill-vs-drain rate, per adapter, surface pressure before any ring fills. +- **Ring depth is a false-positive-fault threshold**, not just a memory knob: bias + the order ring shallow (fast wedge detection, low in-flight exposure) and put + burst tolerance in the adapter's internal buffer. Depth is deployment config; + only the delivery class is topic-static. diff --git a/docs/design/bus-backend-realization.md b/docs/design/bus-backend-realization.md new file mode 100644 index 0000000..e2e294b --- /dev/null +++ b/docs/design/bus-backend-realization.md @@ -0,0 +1,47 @@ +# Bus backend realization (reference, not a decision) + +How each transport realizes the backend-agnostic Bus **contract** (ADR-0020). The +trait says *what*; this note records *how*. Nothing here is a guarantee the trait +makes — it is the map from one universal contract to per-backend mechanisms, and is +updated as adapters are actually built (none exist yet). + +The boundary test for everything below: *does surfacing it buy the consumer a +capability (→ trait, like the loan guard) or is it just how one backend copes +(→ adapter, here)?* All of this is the "copes" side. + +## Contract → mechanism + +| Contract (ADR-0020) | iceoryx2 | Kafka | Zenoh | Aeron | +|---|---|---|---|---| +| keyed routing | services + **shard-fn + within-shard filter** (service-count limit) | topic **partitions**, key→partition + consumer assignment | **key-expressions + wildcard subs** (native; *no manual shard*) | streams/stream-ids + sub-side filter | +| `LatestValue` keyed store (per-key, depth N) | **blackboard** (per-key slot) | **log-compacted topic / KTable** | **storage + queryable** | per-key **latest-cache** over a stream | +| `Reliable`, never-block, error-on-overflow | reliable service, publish → `Full` | producer non-block mode (`max.block.ms = 0` → error) | reliability QoS + congestion = drop/block → pick error | **`offer()` returns `BACK_PRESSURED`** — natively error-on-overflow | +| zero-copy loan *(capability-gated)* | real shared-mem loan | guard backs owned value | guard backs owned value | guard backs owned value | + +## Notes + +- **Sharding (ADR-0021 level 1)** is the iceoryx2 / Aeron / Kafka answer to a + bounded number of cheap channels: `shard = shard_fn(instance-key)` into + `[0, shard_count)`, both publisher and subscriber computing the same shard (it is + topic-class config, like QoS); the subscriber filters co-tenant keys on read + using the **per-key payload seq-stamp**, so gap-detection is shard-transparent. + `shard_fn = identity` is the 1:1 degenerate case for a small stable universe. + Default to a simple shard-fn; interest-correlated shards + hot-symbol carve-outs + are tuning applied when cardinality (`N symbols × M sources × K sub-keys`) + demands it. **Zenoh skips this layer** (native key-space routing). +- **Aeron sharding runs coarser** than iceoryx2: each publication is its own + shared-memory log buffer, so the cost pressure pushes toward fewer stream-ids + (smaller `shard_count`) — same model, different tuning. Aeron IPC is + 1-publisher : N-subscriber (multiple subscriptions per `(channel, stream-id)`), + which fits OATH's one-Source-per-adapter pattern. +- **`LatestValue` keyed-store depth `N`** is a per-topic knob (default 1 = pure + latest). `N > 1` ("hashmap-of-rings") is a lossy recent window; "need every one" + is `Reliable`, not a deeper store. +- **Delta order books** are folded in the **adapter** (ADR-0003 anti-corruption): + apply venue deltas to a canonical book, publish **snapshots** as `LatestValue` + (depth 1). Consumers read the current book; the venue's delta protocol never + leaks inward. A raw-delta `Reliable` stream is an opt-in for microstructure + consumers. +- **Order-path failure mechanics** (the drain-fast internal buffer, the degraded/ + probe ladder, the two halt modes) are specified in ADR-0022 and realized in the + broker adapter. diff --git a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md index 52d01e8..46875f5 100644 --- a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md +++ b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md @@ -3,7 +3,10 @@ **Status:** Branch (A) — crate & dependency-graph revision — **closed 2026-06-24** (ADRs 0007–0009). Branch (B) — strategy runtime & multi-topic-join — **closed 2026-06-25** (ADRs 0010–0013). Branch (C) — Frontend/CLI — **closed 2026-06-26** -(ADRs 0014–0019). **All architecture branches closed.** +(ADRs 0014–0019). **All architecture branches closed.** The **Bus-contract** +parked-question cluster — **closed 2026-06-27** (ADRs 0020–0022): trait delivery +classes × access patterns, two-level routing, and the `Reliable` order-path +failure model. ## The question @@ -136,9 +139,13 @@ cross-process complexity to buy crash containment and hot-pluggability. ## Parked sub-questions (don't lose these) - Schema **evolution/versioning** of `#[repr(C)]` POD messages (ADR-0002). -- Bus trait's **loan-vs-own** contract (design to the borrowed/lifecycle-bounded - case to keep zero-copy). -- Per-topic **delivery semantics** detail + backpressure / ring sizing. +- ~~Bus trait's **loan-vs-own** contract~~ — **resolved (ADR-0020)**: public RAII + loan guard (degrading to owned), with mandated copy-out to owned `M: Copy` POD at + the retention / thread-hand-off / `.await` boundaries. +- ~~Per-topic **delivery semantics** + backpressure / ring sizing~~ — **resolved + (ADR-0020 / 0022)**: two classes (`LatestValue` keyed store / `Reliable` stream); + `Reliable` overflow errors (never blocks, never silently drops) under a graduated + order-path ladder; routing is two-level (ADR-0021). - ~~**Req/resp pattern**~~ — **resolved (ADR-0016)**: req/reply is a thin correlation layer over the Bus (not a side-channel); Frontend read-queries are non-logged and tiered (push-spine → Kernel read → repository), while only a From e2e82184b83de3fc7b7c0beb97905f849683b32d Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Sat, 27 Jun 2026 14:19:55 +0000 Subject: [PATCH 5/8] docs(plans): add ADR-0009 crate-topology implementation plan --- .../2026-06-27-crate-topology-adr-0009.md | 1540 +++++++++++++++++ 1 file changed, 1540 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md diff --git a/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md b/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md new file mode 100644 index 0000000..08440fb --- /dev/null +++ b/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md @@ -0,0 +1,1540 @@ +# Crate Topology Restructure (ADR-0009) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Restructure the Cargo workspace so its crate layout matches the process-aligned, spine-inverted topology defined in [ADR-0009](../../adr/0009-crate-topology-spine-inverted-process-aligned.md). + +**Architecture:** Today's workspace is the monolith graph `oath-engine → {everything}` with inter-layer trait edges (`risk → execution → portfolio`). ADRs 0001–0008 commit to a single-host, multi-process, event-sourced system, which inverts the graph: there is no top composer, only a bottom contract (`oath-model`) every process depends on. This plan renames/moves/splits/deletes crates so the dependency arrows point inward to `oath-model` and the directory layout encodes process boundaries (`/api` = traits; `core/` = the Core process). + +**Tech Stack:** Rust (edition 2024), Cargo workspaces, `just` task runner, `cargo-machete`, `cargo-deny`, `taplo`, `nextest`. + +## Scope + +**In scope — structural only.** Every existing crate except `oath-net-core` is an empty 2-line skeleton (a `//!` doc comment + `#![forbid(unsafe_code)]`). This plan moves that skeleton to the new topology and creates compiling skeletons for the new process crates. The deliverable is: **the workspace matches ADR-0009's target layout and `just ci` is green.** + +**Out of scope — follow-on issues, one per ADR.** Designing trait bodies (`Bus`, `StateView`, `Decision`, `RiskPolicy`/`ExecutionPolicy`/`Portfolio`, `Broker`/`DataProvider`, `Strategy`), the `Kernel` loop, the Event-Log/Snapshot traits, and any backend or adapter implementation. The new `*-api`, `*-kernel`, host, `cli`, and `supervisor` crates are created as **empty compiling skeletons** with a doc comment describing their role; their contents are later work. Do **not** invent trait signatures in this plan. + +**Why skeleton-only is the right scope:** ADR-0009 is a *topology* decision; the trait designs live in other ADRs (0005, 0008, 0020–0022) and are separate `one issue, one PR` units. The project status is "core trait crates defined, backends not yet built." Getting the topology in place first gives every follow-on ADR a home to be implemented into. + +## Global Constraints + +Copied verbatim from the workspace lint config and `CLAUDE.md`. Every task implicitly includes these. + +- **Edition 2024, MSRV 1.90.** Crates inherit `edition.workspace = true` / `rust-version.workspace = true`. Validate MSRV with `just msrv`. +- **No `unsafe`** — `unsafe_code = "deny"` workspace-wide; every crate root also carries `#![forbid(unsafe_code)]`. +- **No `unwrap` / `expect` / indexing** in non-test code (warned). Not relevant to skeletons (no logic), but keep `main()` bodies empty rather than panicking. +- **Document public items** — `missing_docs` is warned and `just lint` runs `clippy … -- -D warnings`, so any **warning is a hard error**. Skeleton libs therefore expose **no `pub` items** (only a crate-level `//!` doc), and bins are a `//!` doc + `fn main() {}`. +- **`clippy::all` is deny-level.** Run `just lint` before every commit. +- **Every dependency declared ahead of first use must be in `[package.metadata.cargo-machete] ignored`** — `just ci` runs `cargo machete`. This is the established repo pattern; follow it exactly. +- **Respect dependency direction** — `oath-model` is the root; never introduce a cycle or an edge that contradicts ADR-0009. +- **Conventional Commits**, enforced by the `commit-msg` hook (subject ≤ 72 chars). Each task gives an exact message. +- **Definition of done is `just ci` green** — identical to the GitHub Actions gate. Do not bypass the git hooks. + +--- + +## Before you begin + +This is **one issue, one PR** (`CLAUDE.md` workflow). The plan document itself may already be committed on the current docs branch; the *restructure* happens on its own branch off `main`. + +1. **Open the issue** (use the feature-request template): "Implement ADR-0009 crate topology (spine-inverted, process-aligned)". Note it in the PR with `Closes #N`. +2. **Branch off `main`:** + ```bash + git switch main && git pull + git switch -c refactor/crate-topology-adr-0009 + ``` + (Optionally isolate via `superpowers:using-git-worktrees`.) +3. **Confirm a green baseline** before touching anything: + ```bash + just ci + ``` + Expected: all gates pass (`fmt`, `fmt-toml`, `typos`, `lint`, `check`, `test`, `deny`, `doc`, `machete`, `gitleaks`, `actionlint`, `shellcheck`). If it is not green on a fresh `main`, stop and fix that first — the plan assumes a green start. + +### Conventions every task follows + +- **Atomic & green.** Each task leaves the workspace compiling and `just ci`-green, then commits. Never split a rename across commits — the manifest, the directory, the members list, the `[workspace.dependencies]` entry, and every dependent must change together. +- **Preserve history.** Use `git mv` for moves, `git rm` for deletes. +- **Refresh the lockfile.** A member-list or package-name change makes `Cargo.lock` stale, and `just check` passes `--locked` (it will *fail* on a stale lock). So after editing manifests run, in order: + ```bash + cargo check --workspace --all-targets --all-features # recompiles + rewrites Cargo.lock + cargo fmt --all # normalize Rust + taplo fmt # normalize TOML (just ci runs `taplo fmt --check`) + ``` + then `just ci`. Commit the updated `Cargo.lock` alongside the change. +- **Verification gate.** This is a structural refactor with no behaviour to unit-test, so each task's "test" is **`just ci` green** plus a structural assertion (the package set / a `cargo tree` edge). That is the honest gate here. +- **The skeleton templates** (used verbatim by the "create" tasks): + + *Library crate `Cargo.toml`* (`oath-model`-only dependants): + ```toml + [package] + name = "oath-CRATE" + version.workspace = true + edition.workspace = true + rust-version.workspace = true + license.workspace = true + + [lints] + workspace = true + + [dependencies] + oath-model = { workspace = true } + thiserror = { workspace = true } + + # cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. + [package.metadata.cargo-machete] + ignored = ["oath-model", "thiserror"] + ``` + + *Library crate `src/lib.rs`:* + ```rust + //! ONE-LINE ROLE FROM ADR-0009. + #![forbid(unsafe_code)] + ``` + + *Binary crate `src/main.rs`:* + ```rust + //! ONE-LINE ROLE FROM ADR-0009. + #![forbid(unsafe_code)] + + fn main() {} + ``` + +--- + +## File Structure + +### Current → target map + +| Current crate (dir) | Action | Target crate (dir) | +|---|---|---| +| `oath-model` (`model/`) | keep | `oath-model` (`model/`) | +| `oath-engine` (`engine/`) | **delete** | — | +| `oath-ingest-core` (`ingest/core/`) | **delete** | — (market data becomes canonical messages in `oath-model`) | +| `oath-strategy-core` (`strategy/core/`) | rename + slim deps to `model` | `oath-strategy-api` (`strategy/api/`) | +| `oath-execution-core` (`execution/core/`) | move + repoint to `core/api` | `oath-core-execution` (`core/execution/`) | +| `oath-portfolio-core` (`portfolio/core/`) | move + repoint to `core/api` | `oath-core-portfolio` (`core/portfolio/`) | +| `oath-risk-core` (`risk/core/`) | move + repoint to `core/api` | `oath-core-risk` (`core/risk/`) | +| `oath-messaging-core` (`messaging/core/`) | **split** | `oath-bus-api` (`bus/api/`) + `oath-event-log-api` (`event-log/api/`) | +| `oath-persistence-core` (`persistence/core/`) | rename | `oath-persistence-api` (`persistence/api/`) | +| `oath-net-core` (`net/core/`) | move (real code + doctests) | `oath-adapter-net-api` (`adapter/net/api/`) | +| — | **create** | `oath-core-api` (`core/api/`) | +| — | **create** | `oath-core-kernel` (`core/kernel/`) | +| — | **create (bin)** | `oath-core` (`core/host/`) | +| — | **create** | `oath-adapter-api` (`adapter/api/`) | +| — | **create (bin)** | `oath-strategy-host` (`strategy/host/`) | +| — | **create (bin)** | `oath-cli` (`cli/`) | +| — | **create (bin)** | `oath-supervisor` (`supervisor/`) | + +Result: **16 crates** (12 lib + 4 bin), up from 10. + +### Target dependency edges (skeleton — minimal & defensible) + +Each declared dependency is `{ workspace = true }` and listed in that crate's `cargo-machete` `ignored` (no code uses it yet). + +| Crate | Internal deps | +|---|---| +| `oath-model` | — (root) | +| `oath-bus-api` | `oath-model` | +| `oath-event-log-api` | `oath-model` | +| `oath-persistence-api` | `oath-model` | +| `oath-core-api` | `oath-model` | +| `oath-core-risk` | `oath-core-api`, `oath-model` | +| `oath-core-execution` | `oath-core-api`, `oath-model` | +| `oath-core-portfolio` | `oath-core-api`, `oath-model` | +| `oath-core-kernel` | `oath-core-api`, `oath-model` | +| `oath-core` (bin) | `oath-core-kernel`, `oath-core-api`, `oath-core-risk`, `oath-core-execution`, `oath-core-portfolio`, `oath-model` | +| `oath-adapter-api` | `oath-model` | +| `oath-adapter-net-api` | — (external only: `bytes`, `http`, `http-body`, `futures-core`, `futures-sink`) | +| `oath-strategy-api` | `oath-model` | +| `oath-strategy-host` (bin) | `oath-strategy-api`, `oath-model` | +| `oath-cli` (bin) | `oath-model` | +| `oath-supervisor` (bin) | `oath-model` | + +**Dependency rules used to pick the above:** +- New crates depend on the minimum that expresses their ADR-0009 role; when in doubt, `oath-model` only. +- The three relocated Policies (`core-risk`/`-execution`/`-portfolio`) drop their old inter-layer edges and depend on the trait hub `oath-core-api` — this *is* the spine inversion. +- The `oath-core` binary is the one place the composition is visible: it binds the Kernel + concrete Policies (ADR-0007/0008). Bus / Event-Log / persistence *backends* are wired here when they exist (follow-on); the binary depends on those `*-api` traits only once it uses them. +- Only **library** crates go in `[workspace.dependencies]`; the four binaries (`oath-core`, `oath-strategy-host`, `oath-cli`, `oath-supervisor`) do not (nothing depends on them). + +### Target final state of root `Cargo.toml` (for reference — built up task by task) + +`members`: +```toml +members = [ + "crates/model", + "crates/bus/api", + "crates/event-log/api", + "crates/persistence/api", + "crates/core/api", + "crates/core/risk", + "crates/core/execution", + "crates/core/portfolio", + "crates/core/kernel", + "crates/core/host", + "crates/adapter/api", + "crates/adapter/net/api", + "crates/strategy/api", + "crates/strategy/host", + "crates/cli", + "crates/supervisor", +] +``` + +Internal `[workspace.dependencies]` (only the OATH path-deps shown; external deps unchanged): +```toml +oath-model = { path = "crates/model", version = "0.1.0" } +oath-bus-api = { path = "crates/bus/api", version = "0.1.0" } +oath-event-log-api = { path = "crates/event-log/api", version = "0.1.0" } +oath-persistence-api = { path = "crates/persistence/api", version = "0.1.0" } +oath-core-api = { path = "crates/core/api", version = "0.1.0" } +oath-core-risk = { path = "crates/core/risk", version = "0.1.0" } +oath-core-execution = { path = "crates/core/execution", version = "0.1.0" } +oath-core-portfolio = { path = "crates/core/portfolio", version = "0.1.0" } +oath-core-kernel = { path = "crates/core/kernel", version = "0.1.0" } +oath-adapter-api = { path = "crates/adapter/api", version = "0.1.0" } +oath-adapter-net-api = { path = "crates/adapter/net/api", version = "0.1.0" } +oath-strategy-api = { path = "crates/strategy/api", version = "0.1.0" } +``` + +--- + +## Task 1: Delete `oath-engine` + +The monolith composer. Nothing depends on it (it is the top of the old graph), so it deletes cleanly and shrinks the graph first. + +**Files:** +- Delete: `crates/engine/` (whole directory) +- Modify: `Cargo.toml` (root) — `members` + +**Interfaces:** +- Consumes: nothing. +- Produces: a workspace with no `oath-engine` member. Later tasks rely on `oath-engine` being absent (it is not in `[workspace.dependencies]`, so only `members` changes). + +- [ ] **Step 1: Remove the crate** + +```bash +git rm -r crates/engine +``` + +- [ ] **Step 2: Remove it from the workspace members** + +In `Cargo.toml` (root), delete this line from `members`: +```toml + "crates/engine", +``` + +- [ ] **Step 3: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: `cargo check` succeeds; `Cargo.lock` no longer contains `name = "oath-engine"`. + +- [ ] **Step 4: Verify the full gate** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "refactor(engine): delete oath-engine composer (ADR-0009)" +``` + +--- + +## Task 2: Rename `strategy/core` → `strategy/api`, slim deps to `oath-model` + +In the new topology `strategy/api` is the user-facing `Strategy` trait. It speaks only the canonical model (it consumes market data and proposes Signals over the Bus), so its old edges to `ingest`/`execution`/`portfolio` are dropped. Doing this *before* deleting `ingest` and moving the Policies removes `strategy` as a consumer of all three, decoupling the later tasks. + +**Files:** +- Move: `crates/strategy/core/` → `crates/strategy/api/` +- Modify: `crates/strategy/api/Cargo.toml` (name + deps) +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: a graph with no `oath-engine`. +- Produces: `oath-strategy-api` at `crates/strategy/api`, depending only on `oath-model`. `oath-ingest-core`, `oath-execution-core`, `oath-portfolio-core` now have one fewer consumer each. + +- [ ] **Step 1: Move the directory** + +```bash +mkdir -p crates/strategy +git mv crates/strategy/core crates/strategy/api +``` + +- [ ] **Step 2: Rewrite `crates/strategy/api/Cargo.toml`** + +Replace the whole file with: +```toml +[package] +name = "oath-strategy-api" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model", "thiserror"] +``` + +- [ ] **Step 3: Update the crate doc comment** + +In `crates/strategy/api/src/lib.rs`, the first line currently reads +`//! User-facing `Strategy` trait and signal types.` — keep it; it is still accurate. No change needed. + +- [ ] **Step 4: Update the root `Cargo.toml`** + +In `members`, change: +```toml + "crates/strategy/core", +``` +to: +```toml + "crates/strategy/api", +``` + +In `[workspace.dependencies]`, change: +```toml +oath-strategy-core = { path = "crates/strategy/core", version = "0.1.0" } +``` +to: +```toml +oath-strategy-api = { path = "crates/strategy/api", version = "0.1.0" } +``` + +- [ ] **Step 5: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; `Cargo.lock` shows `oath-strategy-api` and no `oath-strategy-core`. + +- [ ] **Step 6: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 7: Commit** + +```bash +git add -A +git commit -m "refactor(strategy): rename core->api, depend only on model (ADR-0009)" +``` + +--- + +## Task 3: Delete `oath-ingest-core` + +Per ADR-0009, market data is canonical messages in `oath-model`, published by adapters and carried on the Bus — there is no ingest trait crate. After Task 2 nothing depends on it. + +**Files:** +- Delete: `crates/ingest/` +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: a graph where `strategy/api` no longer depends on ingest. +- Produces: no `oath-ingest-core` anywhere. + +- [ ] **Step 1: Confirm there are no remaining consumers** + +```bash +grep -rn "oath-ingest-core\|oath_ingest_core" crates/ Cargo.toml +``` +Expected: only matches inside `crates/ingest/` itself and the root `Cargo.toml` (its member + workspace-dep lines). If anything else matches, stop — a consumer was missed. + +- [ ] **Step 2: Remove the crate** + +```bash +git rm -r crates/ingest +``` + +- [ ] **Step 3: Update the root `Cargo.toml`** + +In `members`, delete: +```toml + "crates/ingest/core", +``` +In `[workspace.dependencies]`, delete: +```toml +oath-ingest-core = { path = "crates/ingest/core", version = "0.1.0" } +``` + +- [ ] **Step 4: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; `Cargo.lock` no longer contains `oath-ingest-core`. + +- [ ] **Step 5: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "refactor(ingest): delete oath-ingest-core; data is canonical model (ADR-0009)" +``` + +--- + +## Task 4: Create `oath-core-api` (the Core trait hub) + +The trait hub the Kernel and Policies depend on (`StateView`, `Decision`, `ActionSink`, `RiskPolicy`/`ExecutionPolicy`/`Portfolio` — defined later). Must exist before Task 5 repoints the Policies onto it. + +**Files:** +- Create: `crates/core/api/Cargo.toml` +- Create: `crates/core/api/src/lib.rs` +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: `oath-model`. +- Produces: `oath-core-api` at `crates/core/api` (workspace-dep key `oath-core-api`). Tasks 5 and 6 depend on it. + +- [ ] **Step 1: Create `crates/core/api/Cargo.toml`** + +```toml +[package] +name = "oath-core-api" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model", "thiserror"] +``` + +- [ ] **Step 2: Create `crates/core/api/src/lib.rs`** + +```rust +//! Core trait hub: `StateView`, `Decision`, `ActionSink`, and the +//! `RiskPolicy` / `ExecutionPolicy` / `Portfolio` Policy contracts. +#![forbid(unsafe_code)] +``` + +- [ ] **Step 3: Register in the root `Cargo.toml`** + +Add to `members` (after `"crates/persistence/core",` is fine — order is cosmetic): +```toml + "crates/core/api", +``` +Add to `[workspace.dependencies]`: +```toml +oath-core-api = { path = "crates/core/api", version = "0.1.0" } +``` + +- [ ] **Step 4: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; `cargo check` reports `oath-core-api` compiled. + +- [ ] **Step 5: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "feat(core): add oath-core-api trait hub skeleton (ADR-0009)" +``` + +--- + +## Task 5: Move the Policies under `core/` and invert their edges onto `oath-core-api` + +`execution/core` → `core/execution`, `portfolio/core` → `core/portfolio`, `risk/core` → `core/risk`. These three are entangled in the old graph (`risk → execution, portfolio`; `portfolio → execution`), and ADR-0009 inverts all of those edges to point at the trait hub. They therefore move in **one atomic task** — a reviewer evaluates the relocation as a unit, and no intermediate single-move compiles. + +**Files:** +- Move: `crates/execution/core/` → `crates/core/execution/` +- Move: `crates/portfolio/core/` → `crates/core/portfolio/` +- Move: `crates/risk/core/` → `crates/core/risk/` +- Modify: each moved crate's `Cargo.toml` (name + deps) +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: `oath-core-api` (Task 4), `oath-model`. +- Produces: `oath-core-execution`, `oath-core-portfolio`, `oath-core-risk` at `crates/core/{execution,portfolio,risk}`, each depending on `oath-core-api` + `oath-model` only. Task 6 (`oath-core` host) binds all three. + +- [ ] **Step 1: Move the three directories** + +```bash +mkdir -p crates/core +git mv crates/execution/core crates/core/execution +git mv crates/portfolio/core crates/core/portfolio +git mv crates/risk/core crates/core/risk +rmdir crates/execution crates/portfolio crates/risk 2>/dev/null || true +``` + +- [ ] **Step 2: Rewrite `crates/core/execution/Cargo.toml`** + +```toml +[package] +name = "oath-core-execution" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-core-api = { workspace = true } +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-core-api", "oath-model", "thiserror"] +``` + +- [ ] **Step 3: Rewrite `crates/core/portfolio/Cargo.toml`** + +```toml +[package] +name = "oath-core-portfolio" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-core-api = { workspace = true } +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-core-api", "oath-model", "thiserror"] +``` + +- [ ] **Step 4: Rewrite `crates/core/risk/Cargo.toml`** + +```toml +[package] +name = "oath-core-risk" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-core-api = { workspace = true } +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-core-api", "oath-model", "thiserror"] +``` + +- [ ] **Step 5: Update the crate doc comments to the new (Policy-impl) role** + +In `crates/core/risk/src/lib.rs`, replace line 1: +```rust +//! Risk check, risk engine, and verdict traits. +``` +with: +```rust +//! `RiskPolicy` implementation: cancel/amend rules run by the Kernel. +``` + +In `crates/core/execution/src/lib.rs`, replace line 1: +```rust +//! Order lifecycle, fills, and execution report traits. +``` +with: +```rust +//! `ExecutionPolicy` implementation: how the Kernel works an order. +``` + +In `crates/core/portfolio/src/lib.rs`, replace line 1: +```rust +//! Positions, P&L, and account management traits. +``` +with: +```rust +//! `Portfolio` implementation: positions and P&L folded by the Kernel. +``` + +- [ ] **Step 6: Update the root `Cargo.toml`** + +In `members`, replace the three old lines: +```toml + "crates/execution/core", + "crates/portfolio/core", + "crates/risk/core", +``` +with: +```toml + "crates/core/execution", + "crates/core/portfolio", + "crates/core/risk", +``` + +In `[workspace.dependencies]`, replace: +```toml +oath-execution-core = { path = "crates/execution/core", version = "0.1.0" } +oath-portfolio-core = { path = "crates/portfolio/core", version = "0.1.0" } +oath-risk-core = { path = "crates/risk/core", version = "0.1.0" } +``` +with: +```toml +oath-core-execution = { path = "crates/core/execution", version = "0.1.0" } +oath-core-portfolio = { path = "crates/core/portfolio", version = "0.1.0" } +oath-core-risk = { path = "crates/core/risk", version = "0.1.0" } +``` + +- [ ] **Step 7: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; no `oath-execution-core`/`oath-portfolio-core`/`oath-risk-core` remain in `Cargo.lock`. + +- [ ] **Step 8: Assert the inversion held** + +```bash +cargo tree -p oath-core-risk --edges normal --depth 1 +``` +Expected: lists `oath-core-api` and `oath-model` — and **not** `oath-core-execution` or `oath-core-portfolio` (the old inter-layer edges are gone). + +- [ ] **Step 9: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 10: Commit** + +```bash +git add -A +git commit -m "refactor(core): move policies under core/, invert onto core-api (ADR-0009)" +``` + +--- + +## Task 6: Create `oath-core-kernel` and the `oath-core` host binary + +The Kernel library (`Kernel` single-writer loop — body is follow-on) and the Core process binary that binds it to the concrete Policies (ADR-0007/0008). + +**Files:** +- Create: `crates/core/kernel/Cargo.toml`, `crates/core/kernel/src/lib.rs` +- Create: `crates/core/host/Cargo.toml`, `crates/core/host/src/main.rs` +- Modify: `Cargo.toml` (root) — `members` (both) + `[workspace.dependencies]` (kernel only) + +**Interfaces:** +- Consumes: `oath-core-api`, `oath-core-risk`, `oath-core-execution`, `oath-core-portfolio`, `oath-model`. +- Produces: `oath-core-kernel` (lib) and `oath-core` (bin `oath-core`). No later task depends on the binary. + +- [ ] **Step 1: Create `crates/core/kernel/Cargo.toml`** + +```toml +[package] +name = "oath-core-kernel" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-core-api = { workspace = true } +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-core-api", "oath-model", "thiserror"] +``` + +- [ ] **Step 2: Create `crates/core/kernel/src/lib.rs`** + +```rust +//! The Kernel: the single-writer loop that owns canonical state and runs +//! Policies over a read-only view of it. Generic over ``. +#![forbid(unsafe_code)] +``` + +- [ ] **Step 3: Create `crates/core/host/Cargo.toml`** + +```toml +[package] +name = "oath-core" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-core-kernel = { workspace = true } +oath-core-api = { workspace = true } +oath-core-risk = { workspace = true } +oath-core-execution = { workspace = true } +oath-core-portfolio = { workspace = true } +oath-model = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = [ + "oath-core-api", + "oath-core-execution", + "oath-core-kernel", + "oath-core-portfolio", + "oath-core-risk", + "oath-model", +] +``` + +- [ ] **Step 4: Create `crates/core/host/src/main.rs`** + +```rust +//! The Core process binary: binds the Kernel to the chosen Policies and backends. +#![forbid(unsafe_code)] + +fn main() {} +``` + +- [ ] **Step 5: Update the root `Cargo.toml`** + +Add to `members`: +```toml + "crates/core/kernel", + "crates/core/host", +``` +Add to `[workspace.dependencies]` (kernel only — `oath-core` is a binary): +```toml +oath-core-kernel = { path = "crates/core/kernel", version = "0.1.0" } +``` + +- [ ] **Step 6: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; both new crates compile. + +- [ ] **Step 7: Assert the binary exists** + +```bash +cargo build -p oath-core +ls target/debug/oath-core +``` +Expected: the `oath-core` binary is produced. + +- [ ] **Step 8: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 9: Commit** + +```bash +git add -A +git commit -m "feat(core): add oath-core-kernel lib and oath-core host bin (ADR-0009)" +``` + +--- + +## Task 7: Split `messaging/core` into `bus/api` + `event-log/api` + +ADR-0009 separates the Bus (transport) from the Event Log (the persisted, totally-ordered recovery spine). The old `oath-messaging-core` (an empty skeleton) is deleted and replaced by two skeletons. Nothing depends on `oath-messaging-core` after Task 1. + +**Files:** +- Delete: `crates/messaging/` +- Create: `crates/bus/api/Cargo.toml`, `crates/bus/api/src/lib.rs` +- Create: `crates/event-log/api/Cargo.toml`, `crates/event-log/api/src/lib.rs` +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: `oath-model`. +- Produces: `oath-bus-api` (`crates/bus/api`) and `oath-event-log-api` (`crates/event-log/api`). + +- [ ] **Step 1: Confirm no remaining consumers, then remove** + +```bash +grep -rn "oath-messaging-core\|oath_messaging_core" crates/ Cargo.toml +git rm -r crates/messaging +``` +Expected (before removal): matches only inside `crates/messaging/` and the root `Cargo.toml`. + +- [ ] **Step 2: Create `crates/bus/api/Cargo.toml`** + +```toml +[package] +name = "oath-bus-api" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model", "thiserror"] +``` + +- [ ] **Step 3: Create `crates/bus/api/src/lib.rs`** + +```rust +//! The Bus trait: backend-agnostic transport for canonical messages, with the +//! LatestValue and Reliable delivery classes. +#![forbid(unsafe_code)] +``` + +- [ ] **Step 4: Create `crates/event-log/api/Cargo.toml`** + +```toml +[package] +name = "oath-event-log-api" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model", "thiserror"] +``` + +- [ ] **Step 5: Create `crates/event-log/api/src/lib.rs`** + +```rust +//! The Event Log and Snapshot traits: the append-only, totally-ordered record +//! Core's state is a pure fold over, plus point-in-time recovery captures. +#![forbid(unsafe_code)] +``` + +- [ ] **Step 6: Update the root `Cargo.toml`** + +In `members`, replace: +```toml + "crates/messaging/core", +``` +with: +```toml + "crates/bus/api", + "crates/event-log/api", +``` + +In `[workspace.dependencies]`, replace: +```toml +oath-messaging-core = { path = "crates/messaging/core", version = "0.1.0" } +``` +with: +```toml +oath-bus-api = { path = "crates/bus/api", version = "0.1.0" } +oath-event-log-api = { path = "crates/event-log/api", version = "0.1.0" } +``` + +- [ ] **Step 7: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; `Cargo.lock` shows `oath-bus-api` + `oath-event-log-api`, no `oath-messaging-core`. + +- [ ] **Step 8: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 9: Commit** + +```bash +git add -A +git commit -m "refactor(bus): split messaging into bus/api and event-log/api (ADR-0009)" +``` + +--- + +## Task 8: Rename `persistence/core` → `persistence/api` + +The reserved keyed-repository trait crate (read-models, symbology, adapter dedup tables) — distinct from the Event Log. Pure rename; nothing depends on it after Task 1. + +**Files:** +- Move: `crates/persistence/core/` → `crates/persistence/api/` +- Modify: `crates/persistence/api/Cargo.toml` (name) +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: `oath-model`. +- Produces: `oath-persistence-api` at `crates/persistence/api`. + +- [ ] **Step 1: Move the directory** + +```bash +git mv crates/persistence/core crates/persistence/api +``` + +- [ ] **Step 2: Rename the package** + +In `crates/persistence/api/Cargo.toml`, change: +```toml +name = "oath-persistence-core" +``` +to: +```toml +name = "oath-persistence-api" +``` + +- [ ] **Step 3: Update the root `Cargo.toml`** + +In `members`, change `"crates/persistence/core",` → `"crates/persistence/api",`. + +In `[workspace.dependencies]`, change: +```toml +oath-persistence-core = { path = "crates/persistence/core", version = "0.1.0" } +``` +to: +```toml +oath-persistence-api = { path = "crates/persistence/api", version = "0.1.0" } +``` + +- [ ] **Step 4: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; `Cargo.lock` shows `oath-persistence-api`, no `oath-persistence-core`. + +- [ ] **Step 5: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "refactor(persistence): rename core->api per spine layout (ADR-0009)" +``` + +--- + +## Task 9: Move `net/core` → `adapter/net/api` (real code + doctests) + +The only crate with real content (the `Service`/`Layer` composition primitives). `net` moves under `adapter/` because adapters are its only user. The crate name changes from `oath-net-core` to `oath-adapter-net-api`, so the **Rust path** changes from `oath_net_core` to `oath_adapter_net_api` — the doctests and doc comment that name the crate must be rewritten or `cargo test --doc` fails. + +**Files:** +- Move: `crates/net/core/` → `crates/adapter/net/api/` +- Modify: `crates/adapter/net/api/Cargo.toml` (name) +- Modify: `crates/adapter/net/api/src/lib.rs` (doc text), `crates/adapter/net/api/src/service.rs` (two doctest `use` lines) +- Modify: `Cargo.toml` (root) — `members` + `[workspace.dependencies]` + +**Interfaces:** +- Consumes: external crates only (`bytes`, `http`, `http-body`, `futures-core`, `futures-sink`); no internal deps. +- Produces: `oath-adapter-net-api` at `crates/adapter/net/api`, re-exporting `Service`, `Layer`, `ServiceBuilder`, `Identity`, `Stack`, `ErrorKind`, `HasErrorKind` (unchanged API). + +- [ ] **Step 1: Move the directory** + +```bash +mkdir -p crates/adapter/net +git mv crates/net/core crates/adapter/net/api +rmdir crates/net 2>/dev/null || true +``` + +- [ ] **Step 2: Rename the package** + +In `crates/adapter/net/api/Cargo.toml`, change: +```toml +name = "oath-net-core" +``` +to: +```toml +name = "oath-adapter-net-api" +``` + +- [ ] **Step 3: Fix the crate-name references in code** + +In `crates/adapter/net/api/src/lib.rs`, replace line 1: +```rust +//! `oath-net-core` — composition primitives and capability trait contracts. +``` +with: +```rust +//! `oath-adapter-net-api` — composition primitives and capability trait contracts. +``` + +In `crates/adapter/net/api/src/service.rs`, replace line 14: +```rust +//! # use oath_net_core::service::{Layer, Service, ServiceBuilder}; +``` +with: +```rust +//! # use oath_adapter_net_api::service::{Layer, Service, ServiceBuilder}; +``` +and replace line 76: +```rust +/// # use oath_net_core::service::{Identity, ServiceBuilder}; +``` +with: +```rust +/// # use oath_adapter_net_api::service::{Identity, ServiceBuilder}; +``` + +- [ ] **Step 4: Confirm no stale references remain** + +```bash +grep -rn "oath_net_core\|oath-net-core" crates/ +``` +Expected: **no output**. + +- [ ] **Step 5: Update the root `Cargo.toml`** + +In `members`, change `"crates/net/core",` → `"crates/adapter/net/api",`. + +In `[workspace.dependencies]`, change: +```toml +oath-net-core = { path = "crates/net/core", version = "0.1.0" } +``` +to: +```toml +oath-adapter-net-api = { path = "crates/adapter/net/api", version = "0.1.0" } +``` + +- [ ] **Step 6: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success. + +- [ ] **Step 7: Run the doctests explicitly (the risk area for this task)** + +```bash +cargo test -p oath-adapter-net-api --doc +``` +Expected: the `service` doctests compile and pass under the new crate path. + +- [ ] **Step 8: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 9: Commit** + +```bash +git add -A +git commit -m "refactor(adapter): move net/core to adapter/net/api (ADR-0009)" +``` + +--- + +## Task 10: Create the remaining process skeletons (`adapter/api`, `strategy/host`, `cli`, `supervisor`) + +The last new crates: the adapter trait hub (`Broker`/`DataProvider`), the Strategy Node binary, the first Frontend (CLI), and the operational-plane Supervisor. + +**Files:** +- Create: `crates/adapter/api/Cargo.toml`, `crates/adapter/api/src/lib.rs` +- Create: `crates/strategy/host/Cargo.toml`, `crates/strategy/host/src/main.rs` +- Create: `crates/cli/Cargo.toml`, `crates/cli/src/main.rs` +- Create: `crates/supervisor/Cargo.toml`, `crates/supervisor/src/main.rs` +- Modify: `Cargo.toml` (root) — `members` (all four) + `[workspace.dependencies]` (`adapter/api` only) + +**Interfaces:** +- Consumes: `oath-model`; `strategy/host` also consumes `oath-strategy-api` (Task 2). +- Produces: `oath-adapter-api` (lib) + binaries `oath-strategy-host`, `oath-cli`, `oath-supervisor`. This completes the 16-crate target. + +- [ ] **Step 1: Create `crates/adapter/api/Cargo.toml`** + +```toml +[package] +name = "oath-adapter-api" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } +thiserror = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model", "thiserror"] +``` + +- [ ] **Step 2: Create `crates/adapter/api/src/lib.rs`** + +```rust +//! Adapter trait hub: the host harness plus the `Broker` and `DataProvider` +//! contracts every venue adapter implements. +#![forbid(unsafe_code)] +``` + +- [ ] **Step 3: Create `crates/strategy/host/Cargo.toml`** + +```toml +[package] +name = "oath-strategy-host" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-strategy-api = { workspace = true } +oath-model = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model", "oath-strategy-api"] +``` + +- [ ] **Step 4: Create `crates/strategy/host/src/main.rs`** + +```rust +//! The Strategy Node binary: hosts user strategies, isolated from Core. +#![forbid(unsafe_code)] + +fn main() {} +``` + +- [ ] **Step 5: Create `crates/cli/Cargo.toml`** + +```toml +[package] +name = "oath-cli" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model"] +``` + +- [ ] **Step 6: Create `crates/cli/src/main.rs`** + +```rust +//! The CLI: the first Frontend — observes and controls the hub from outside Core. +#![forbid(unsafe_code)] + +fn main() {} +``` + +- [ ] **Step 7: Create `crates/supervisor/Cargo.toml`** + +```toml +[package] +name = "oath-supervisor" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true + +[lints] +workspace = true + +[dependencies] +oath-model = { workspace = true } + +# cargo-machete: deps declared ahead of first use; prune from `ignored` as they are adopted. +[package.metadata.cargo-machete] +ignored = ["oath-model"] +``` + +- [ ] **Step 8: Create `crates/supervisor/src/main.rs`** + +```rust +//! The Supervisor: the operational-plane process that boots and watches the +//! host's topology. Never participates in Core's deterministic path. +#![forbid(unsafe_code)] + +fn main() {} +``` + +- [ ] **Step 9: Update the root `Cargo.toml`** + +Add to `members`: +```toml + "crates/adapter/api", + "crates/strategy/host", + "crates/cli", + "crates/supervisor", +``` +Add to `[workspace.dependencies]` (lib only): +```toml +oath-adapter-api = { path = "crates/adapter/api", version = "0.1.0" } +``` + +- [ ] **Step 10: Refresh lock + formatting** + +```bash +cargo check --workspace --all-targets --all-features +cargo fmt --all +taplo fmt +``` +Expected: success; all four new crates compile. + +- [ ] **Step 11: Verify** + +```bash +just ci +``` +Expected: all gates pass. + +- [ ] **Step 12: Commit** + +```bash +git add -A +git commit -m "feat: add adapter-api, strategy-host, cli, supervisor skeletons (ADR-0009)" +``` + +--- + +## Task 11: Sync docs & repo config to the new topology, then final verification + +The code now matches ADR-0009; the human-facing docs and GitHub config still describe the old monolith. ADR-0009's own consequences require the README graph be updated "when the restructure is implemented." This task brings `README.md`, `CLAUDE.md`, `CHANGELOG.md`, and the two issue templates into line, then runs the full gate and the topology assertion. + +**Files:** +- Modify: `README.md` (Domain Layout table + Mermaid graph + "coming soon" line) +- Modify: `CLAUDE.md` (Status section) +- Modify: `CHANGELOG.md` (Unreleased → Changed) +- Modify: `.github/ISSUE_TEMPLATE/bug_report.yml` (area dropdown) +- Modify: `.github/ISSUE_TEMPLATE/feature_request.yml` (area dropdown) + +**Interfaces:** +- Consumes: the completed 16-crate topology. +- Produces: docs/config consistent with ADR-0009. Terminal task. + +- [ ] **Step 1: Rewrite the README "Domain Layout" table** + +In `README.md`, replace the entire table under `## Domain Layout` (the `| Crate | Purpose |` block, rows `oath-model` … `oath-engine`) with: +```markdown +| Crate | Purpose | +|---|---| +| `oath-model` | Canonical domain primitives and message payloads — the root contract | +| `oath-bus-api` | Bus trait: backend-agnostic transport (LatestValue / Reliable classes) | +| `oath-event-log-api` | Event Log + Snapshot traits: the ordered recovery spine | +| `oath-persistence-api` | Repository trait: keyed, queryable read-models and dedup tables | +| `oath-core-api` | Core trait hub: `StateView`, `Decision`, `ActionSink`, the Policy contracts | +| `oath-core-risk` / `-execution` / `-portfolio` | Policy implementations bound by the Core binary | +| `oath-core-kernel` | The `Kernel` single-writer loop | +| `oath-core` | The Core process binary | +| `oath-adapter-api` | Harness + `Broker` / `DataProvider` traits for venue adapters | +| `oath-adapter-net-api` | HTTP/WS composition primitives (`Service`, `Layer`) for adapters | +| `oath-strategy-api` | User-facing `Strategy` trait and Signal types | +| `oath-strategy-host` | Strategy Node binary: hosts user strategies, isolated from Core | +| `oath-cli` | The first Frontend (MVP) | +| `oath-supervisor` | Operational-plane process: boots and watches the topology | +``` + +- [ ] **Step 2: Replace the README Mermaid dependency graph** + +In `README.md`, replace the whole ```` ```mermaid ```` … ```` ``` ```` block under `## Dependency Graph` with: +```markdown +```mermaid +graph TD + model[oath-model] + + busapi[oath-bus-api] --> model + evlog[oath-event-log-api] --> model + perapi[oath-persistence-api] --> model + coreapi[oath-core-api] --> model + adapterapi[oath-adapter-api] --> model + stratapi[oath-strategy-api] --> model + netapi[oath-adapter-net-api] + + risk[oath-core-risk] --> coreapi + exe[oath-core-execution] --> coreapi + por[oath-core-portfolio] --> coreapi + kernel[oath-core-kernel] --> coreapi + + core[oath-core] --> kernel + core --> risk + core --> exe + core --> por + + strathost[oath-strategy-host] --> stratapi + cli[oath-cli] --> model + sup[oath-supervisor] --> model +``` +``` +(Arrows point inward to `oath-model`; `oath-core` composes the Kernel and Policies. `oath-adapter-net-api` has no internal edges. This reflects the skeleton dependency table above; extend it as backends/traits are added.) + +- [ ] **Step 3: Update the README "coming soon" line** + +In `README.md`, replace: +```markdown +Backend crates (e.g. `oath-net-reqwest`, `oath-messaging-memory`, `oath-persistence-sqlite`) and adapter crates (e.g. `oath-adapter-ibkr`) are coming soon. +``` +with: +```markdown +The crates above are compiling skeletons. Bus/Event-Log/persistence backends (e.g. `oath-bus-iceoryx2`, `oath-event-log-chronicle`, `oath-persistence-sqlite`) and venue adapters (e.g. `oath-adapter-ibkr`) are coming soon. +``` + +- [ ] **Step 4: Update the `CLAUDE.md` Status section** + +In `CLAUDE.md`, replace the paragraph under `## Status: …` that begins "The Cargo workspace and the ten trait-defining `*-core` crates" and ends "see the \"do not use\" notice in the README." with: +```markdown +The Cargo workspace follows the process-aligned, spine-inverted crate topology of +[ADR-0009](docs/adr/0009-crate-topology-spine-inverted-process-aligned.md): +`oath-model` is the root contract; `/api` crates define traits; `core/` +holds the Core process (`core/kernel` + Policies + the `oath-core` binary); and +`adapter/`, `strategy/`, `cli/`, and `supervisor/` are the other process roles. The +crates are compiling skeletons — trait bodies, the Kernel loop, Policies, and all +backends/adapters are **not yet built**. The project is pre-release — see the "do +not use" notice in the README. +``` +Also update the heading line `## Status: core trait crates defined, backends not yet built` to: +```markdown +## Status: spine-inverted crate skeletons in place, contents not yet built +``` + +- [ ] **Step 5: Update `CHANGELOG.md`** + +In `CHANGELOG.md`, under `## [Unreleased]`, add a `### Changed` subsection above `### Added`: +```markdown +### Changed + +- Restructured the workspace to the process-aligned, spine-inverted crate topology + of ADR-0009: deleted `oath-engine` and `oath-ingest-core`; split + `oath-messaging-core` into `oath-bus-api` + `oath-event-log-api`; renamed the + `*-core` trait crates to `/api`; moved the risk/execution/portfolio + Policies under `core/`; relocated `oath-net-core` to `oath-adapter-net-api`; and + added `oath-core-api`, `oath-core-kernel`, and the `oath-core`, `oath-strategy-host`, + `oath-cli`, and `oath-supervisor` process crates. +``` +Then fix the now-stale "10 domain crates" bullet in `### Added` — replace: +```markdown +- Cargo workspace scaffold with 10 domain crates: + `oath-model`, `oath-net-core`, `oath-messaging-core`, `oath-persistence-core`, + `oath-ingest-core`, `oath-execution-core`, `oath-portfolio-core`, `oath-risk-core`, + `oath-strategy-core`, `oath-engine`. +``` +with: +```markdown +- Cargo workspace scaffold (initial 10 domain crates; later restructured — see Changed). +``` + +- [ ] **Step 6: Update both issue-template area dropdowns** + +In **both** `.github/ISSUE_TEMPLATE/bug_report.yml` and `.github/ISSUE_TEMPLATE/feature_request.yml`, replace the crate options block: +```yaml + - "oath-model" + - "oath-net" + - "oath-messaging" + - "oath-persistence" + - "oath-ingest" + - "oath-execution" + - "oath-portfolio" + - "oath-risk" + - "oath-strategy" + - "oath-engine" +``` +with: +```yaml + - "oath-model" + - "oath-bus" + - "oath-event-log" + - "oath-persistence" + - "oath-core" + - "oath-adapter" + - "oath-strategy" + - "oath-cli" + - "oath-supervisor" +``` +(Leave the surrounding `"Unsure / other"`, `"tooling / CI"`, and `"docs"` options untouched.) + +- [ ] **Step 7: Normalize formatting** + +```bash +taplo fmt +cargo fmt --all +``` +Expected: no changes needed in Rust (docs-only task), TOML already clean. + +- [ ] **Step 8: Final full verification** + +```bash +just ci +``` +Expected: all gates pass — crucially `typos` (new prose), `doc` (no broken intra-doc links), and `actionlint` (the edited issue templates are YAML, validated by the YAML tooling; `actionlint` covers workflows — run it to be safe). + +- [ ] **Step 9: Assert the complete target topology** + +```bash +cargo metadata --format-version 1 --no-deps | jq -r '.packages[].name' | sort +``` +Expected — exactly these 16 names: +``` +oath-adapter-api +oath-adapter-net-api +oath-bus-api +oath-cli +oath-core +oath-core-api +oath-core-execution +oath-core-kernel +oath-core-portfolio +oath-core-risk +oath-event-log-api +oath-model +oath-persistence-api +oath-strategy-api +oath-strategy-host +oath-supervisor +``` +And the directory layout: +```bash +find crates -name Cargo.toml | sort +``` +Expected: +``` +crates/adapter/api/Cargo.toml +crates/adapter/net/api/Cargo.toml +crates/bus/api/Cargo.toml +crates/cli/Cargo.toml +crates/core/api/Cargo.toml +crates/core/execution/Cargo.toml +crates/core/host/Cargo.toml +crates/core/kernel/Cargo.toml +crates/core/portfolio/Cargo.toml +crates/core/risk/Cargo.toml +crates/event-log/api/Cargo.toml +crates/model/Cargo.toml +crates/persistence/api/Cargo.toml +crates/strategy/api/Cargo.toml +crates/strategy/host/Cargo.toml +crates/supervisor/Cargo.toml +``` + +- [ ] **Step 10: Commit** + +```bash +git add -A +git commit -m "docs: sync README, CLAUDE, CHANGELOG, issue templates to ADR-0009" +``` + +- [ ] **Step 11: Open the PR** + +```bash +git push -u origin refactor/crate-topology-adr-0009 +gh pr create --fill --base main \ + --title "refactor: implement ADR-0009 crate topology" \ + --body "Closes #N. Restructures the workspace to the spine-inverted, process-aligned topology of ADR-0009. Skeleton-only: trait bodies, the Kernel loop, Policies, and backends are follow-on issues." +``` +The pre-push hook runs the full `just ci`; cloud CI re-runs it plus the MSRV job. Both must be green to merge. Squash-merge into `main`; the issue closes. + +--- + +## Self-Review + +**1. Spec coverage** — every ADR-0009 element maps to a task: + +| ADR-0009 element | Task | +|---|---| +| `oath-engine` deleted | 1 | +| `oath-ingest-core` deleted (data is canonical model) | 3 | +| `/api` naming (strategy, persistence) | 2, 8 | +| Policies under `core/`, edges inverted onto `core/api` | 4, 5 | +| `core/kernel` (Kernel lib) + `core/host` (`oath-core` bin) | 6 | +| Bus / Event-Log split (`bus/api` + `event-log/api`) | 7 | +| `net` moved under `adapter/` (`adapter/net/api`) | 9 | +| `core/api`, `adapter/api` trait hubs | 4, 10 | +| New process roles `supervisor` + `cli` (Frontend) | 10 | +| README graph updated on implementation | 11 | + +**2. Placeholder scan** — no `TBD`/`TODO`/"implement later"/"add error handling". Every new file's full contents and every manifest edit are shown verbatim. The only intentionally empty bodies are `fn main() {}` skeletons and doc-comment-only libs, which is the *defined deliverable* (scope section), not a placeholder. + +**3. Name/type consistency** — crate names are identical everywhere they appear (table, dep edges, per-task manifests, README, final assertion): `oath-bus-api`, `oath-event-log-api`, `oath-persistence-api`, `oath-core-api`, `oath-core-risk`, `oath-core-execution`, `oath-core-portfolio`, `oath-core-kernel`, `oath-core`, `oath-adapter-api`, `oath-adapter-net-api`, `oath-strategy-api`, `oath-strategy-host`, `oath-cli`, `oath-supervisor`, `oath-model`. Workspace-dep keys match package names. Directory paths match `members`. The Task 5 `cargo tree` check and Task 9 `grep` guard catch the two highest-risk edits (the edge inversion and the crate-path rename). + +**4. Ordering safety** — the sequence keeps every commit compiling: delete the top composer (1) → decouple `strategy` (2) so `ingest` (3) and the Policies (5) have no outside consumers → create `core-api` (4) before the Policies repoint onto it (5) → independent renames/splits (6–9) → leaf skeletons (10) → docs (11). Each task refreshes `Cargo.lock` before the `--locked` gate. From 619122dfa2a5c24ddbd7f74dc066c95931cc3037 Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Sat, 27 Jun 2026 15:57:27 +0000 Subject: [PATCH 6/8] docs(architecture): numeric types - fixed-point i128 (ADR-0023) Close the numeric-types parked question. Drop rust_decimal; Price/Quantity are fixed-point always-i128 on the wire (Price signed, Quantity unsigned magnitude + Side). Two-domain split (exact i128 / analytical f64, convert at the strategy boundary); precision is instrument metadata, raw-only wire; money ops are checked with widen-to-256 notional; layered float-determinism scope (refines ADR-0012). Couples to the symbology thread (precision/tick/lot are instrument metadata). Co-Authored-By: Claude Opus 4.8 --- CONTEXT.md | 11 +- ...ypes-fixed-point-exact-analytical-split.md | 140 ++++++++++++++++++ ...6-23-from-scratch-architecture-grilling.md | 13 +- 3 files changed, 158 insertions(+), 6 deletions(-) create mode 100644 docs/adr/0023-numeric-types-fixed-point-exact-analytical-split.md diff --git a/CONTEXT.md b/CONTEXT.md index bcf747a..1f8f54b 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -23,12 +23,17 @@ per-Source topics conflated. _Avoid_: venue, feed, provider (for this routing-key role). **Price**: -The value per unit of an instrument, expressed in its quote currency. +The value per unit of an instrument, expressed in its quote currency. Can be +**negative** (e.g. spreads, or a commodity in backwardation gone sub-zero). Its +decimal granularity is a property of the **instrument** (its tick size), not of the +price value itself. _Avoid_: cost, rate, level. **Quantity**: -An amount of an instrument. -_Avoid_: size, amount, volume. +A **magnitude** — a non-negative amount of an instrument. Direction is never +carried by a Quantity; it lives in [Side], and net signed exposure is a property of +a Position derived from Side + Quantity. +_Avoid_: size, amount, volume; a "signed quantity". **Side**: The direction of an order or trade — buy or sell. diff --git a/docs/adr/0023-numeric-types-fixed-point-exact-analytical-split.md b/docs/adr/0023-numeric-types-fixed-point-exact-analytical-split.md new file mode 100644 index 0000000..7091327 --- /dev/null +++ b/docs/adr/0023-numeric-types-fixed-point-exact-analytical-split.md @@ -0,0 +1,140 @@ +# Numeric types: fixed-point `i128` on the wire, two-domain exact/analytical split + +`Price` and `Quantity` (ADR-0002's canonical model) are **fixed-point scaled +integers**, not a decimal type. `rust_decimal` is dropped from `oath-model`: it +carries a validity invariant (the `flags` word bounds the scale and reserves bits), +so not every 16-byte pattern is a valid `Decimal` — which makes `bytemuck::Pod` +unsound and breaks "read slot bytes → deterministic value" (ADR-0002 / 0020). A +scaled integer is trivially POD (every bit pattern valid), deterministic, and +branch-free. This mirrors NautilusTrader, which carries no decimal type on the wire. + +## The numbers — `Price`, `Quantity`, money ops + +- **Width: always `i128`/`u128`** (not a `cfg`-switched i64/i128 like Nautilus's + `high-precision` flag). The cost of always-128 is concentrated almost entirely in + **full-depth order books** (≈2× the bytes on the one hottest message family); + trades/quotes barely move (≤1 cache line), and i128 multiply/divide is slower but + lives on the low-rate order/fill/PnL path, not the data plane. We take the + simplification — one format forever, no 64→128 migration, crypto/wei handled from + day one — and treat **depth-message layout as a separately-optimizable concern** + (packed/delta form later if it ever bottlenecks; the delta-fold note in + [bus-backend-realization](../design/bus-backend-realization.md) already anticipates + it). The depth tail does not wag the whole-model dog. +- **Signedness: `Price` is signed `i128`, `Quantity` is unsigned `u128`.** Negative + prices are real (WTI −$37, calendar/inter-commodity spreads, basis instruments). + `Quantity` is a **magnitude**; **direction lives in `Side`** (the single source of + truth, per the glossary) and **signed exposure is derived in `Position`**, never + stored as a signed quantity — this keeps "a Quantity is a magnitude" a type + invariant, matches how venues report (size + side), and buys a bit of range where + magnitudes get huge (wei). +- **Precision lives with the instrument, not on the wire.** The wire/Event-Log + `Price`/`Quantity` are **precision-free raw `i128`/`u128` newtypes** (clean 16 + bytes, no padding — self-describing `{raw, precision}` is rejected for the wire + because `i128`'s 16-byte alignment pads `{i128, u8}` to 32 bytes *and* reintroduces + the padding-determinism hazard). Precision/tick/lot are **instrument metadata** + (sourced from symbology / the contract — IBKR's `minTick` etc.), resolved once at + the boundary. This is sound because a price **always travels under an + instrument-keyed topic** (`instance-key` carries the symbol, ADR-0020), so the + scale is never orphaned. A **self-describing in-process working type is optional + ergonomics**, not part of the canonical model contract — adopt it lazily where + threading the instrument gets painful; the MVP ships with raw newtypes + + instrument-taking conversions. +- **Money-op contract — no bare arithmetic.** `Price`/`Quantity` do not expose + `+ - * /`. The exact-domain ops are a small set of **explicit, checked** functions. + Add/sub use `checked_*` and **error rather than wrap** (a silent wrap in accounting + is the unrecoverable money bug; Rust release builds wrap on bare `*` by default). + `notional = price × qty` overflows `i128` at wei scale, so products **widen to a + 256-bit intermediate, rescale (÷10ᵏ), then checked-narrow to `i128`** — using a + **vetted pure-Rust, no-`unsafe` bigint** (e.g. `bnum`), not hand-rolled money math + (same principle as buying redb's crash-testing). It is confined to a few cold-path + functions, so the software-256 cost is irrelevant. +- **Rounding — two contexts.** Internal accounting (avg fill price, P&L allocation, + commission) uses **round-half-to-even (banker's)** as the one documented default + (no accumulating bias). The **order-emission boundary** (price→tick, qty→lot) + rounds **explicitly and direction-aware per call** (a buy limit rounds *down*; qty + rounds *down* to not exceed lot/budget) — never an implicit global mode. + +## Two numeric domains + +- **Exact domain — fixed-point `i128`.** Prices, quantities, money, P&L, order + fields, the bus, the Event-Log. Exact, deterministic, POD, zero-copy. +- **Analytical domain — `f64`.** Strategy indicators, signals, statistical features. + Indicators (EMA, stddev, log-returns) involve `exp`/`log`/`sqrt` and + non-terminating ratios that **have no exact fixed-point representation** — `f64` is + the correct type for that work, not a compromise. Strategies **convert at the + boundary**: fixed→float on input (one convert + multiply, negligible vs the + indicator math), float→fixed on order emission — and that reverse conversion **is** + the tick/lot rounding point, so it is not wasted work. Exact position/P&L + *accounting* stays in the exact domain (the Core kernel, ADR-0008); strategies read + exact business state for decisions and do their *analytics* in float. + +## Determinism scope (refines ADR-0012) + +ADR-0012's "a recorded run replays bit-exactly" carries an implicit qualifier that +`f64` makes explicit: bit-exact replay re-runs the **same machine code over the same +logged inputs** → identical floats only on the **same binary + same target**. The +guarantee is therefore **layered**: + +- **Exact domain** (ordering, matching, accounting, P&L) — **bit-exact, + cross-platform, audit-grade** (integers + integer time + single-writer, ADR-0005 / + 0006 / 0008). +- **Strategy float domain** — **bit-exact replay on same binary + same target**; + **cross-target = parity within bounded tolerance** (drift surfaced, not bit-exact). + This is achievable because Rust is **strict IEEE-754 by default** (no implicit FMA + contraction / fast-math), `+ − × ÷` and `sqrt` are correctly-rounded across IEEE + platforms, the **only** cross-target divergence is transcendental `libm` + implementations, and the strategy fold is **single-threaded by construction** + (ADR-0012 / 0013) so no nondeterministic parallel reduction exists. +- **NaN/inf policy:** indicators must be NaN/inf-safe (or the framework guards) — + `NaN != NaN` silently breaks replay-equality. +- **Reserved lever** for future cross-platform float bit-exactness: replace the + platform `libm` with a single portable correctly-rounded math library. Deferred, + not MVP. + +## Real-money correctness + +Layout does not give correctness — no layout makes "wrong precision applied" +impossible (even self-describing `{raw, precision}` is wrong if constructed wrong). +Fixed-point already eliminates the worst class (silent float-rounding drift); the +residual risk is a discrete 10ᵏ scale error, which is wildly out-of-band and trips +risk/sanity bounds immediately. The correctness stack: + +1. Precision is applied **only** through typed functions that *require* the + instrument — misapplication won't compile. +2. **Round-trip property tests / fuzz:** `from_decimal(to_decimal(p, i), i) == p`. +3. **No bare arithmetic; errors never wrap** (above). +4. **ADR-0006 reconciliation is the money-moving backstop:** the order adapter + re-derives human price/qty from raw + contract tick/lot, validates, the broker + echoes the accepted order, and reconciliation compares broker-truth to intent — a + scale bug cannot reach a fill silently. + +## Considered options + +- _`rust_decimal` inner type_ — rejected: not byte-castable (validity invariant → + `Pod` unsound), 16 bytes, software arithmetic on the hot path. Kept only at the + adapter boundary (parsing) and frontend (display), never on the bus or log. +- _`i64` / 9-dp default with an `i128` `cfg` feature flag_ (NautilusTrader) — + rejected: `i64` cannot hold wei (10 ETH = 1e19 > `i64::MAX`), and `cfg`-switched + primitive widths split the on-disk format and add complexity. Chosen: always-128, + depth optimized separately. +- _Self-describing `{raw, precision}` on the wire_ — rejected: 32-byte padded + footprint + padding-determinism hazard. Chosen: raw-only wire, precision from + instrument metadata; self-describing is an optional in-process ergonomic. +- _`{whole, frac}` split / rational `{num, den}`_ — rejected: the split still needs a + precision to interpret `frac` (it just moves the problem) and wrecks arithmetic; + rational denominators grow unbounded and need gcd-normalization. Neither buys + correctness over single fixed-point. +- _Signed `Quantity`_ — rejected: duplicates `Side` as a second source of truth for + direction. Chosen: unsigned magnitude + `Side` + derived `Position` exposure. + +## Consequences + +- **Refines ADR-0002** (numeric inner type) and **ADR-0012** (float-determinism + scope); `rust_decimal` leaves `oath-model`'s dependencies. +- **Couples to symbology** (the next parked thread): precision, tick size, and lot + size are exactly the instrument metadata the symbology layer must supply. +- **Adds a `bnum`-class pure-Rust bigint dependency** for the 256-bit money + intermediate (cold path only). +- The exact/analytical boundary is the same "convert at the edge" pattern already + adopted for event ingestion — float→fixed on order emission is where tick/lot + rounding lives. diff --git a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md index 46875f5..1ceaf68 100644 --- a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md +++ b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md @@ -6,7 +6,9 @@ (ADRs 0014–0019). **All architecture branches closed.** The **Bus-contract** parked-question cluster — **closed 2026-06-27** (ADRs 0020–0022): trait delivery classes × access patterns, two-level routing, and the `Reliable` order-path -failure model. +failure model. **Numeric types** parked question — **closed 2026-06-27** +(ADR-0023): fixed-point always-`i128`, exact/analytical two-domain split, +instrument-sourced precision, checked money ops. ## The question @@ -156,8 +158,13 @@ cross-process complexity to buy crash containment and hot-pluggability. - **Snapshot** cadence and contents (recovery substrate). - **Symbology** design: canonical identity (perm_id/OpenFIGI) + per-adapter mapping. -- `Price`/`Quantity` numeric needs per asset class (crypto/wei) — when to move - the inner type off `rust_decimal`. +- ~~`Price`/`Quantity` numeric needs per asset class (crypto/wei) — when to move + the inner type off `rust_decimal`.~~ — **resolved (ADR-0023)**: drop + `rust_decimal`; fixed-point **always-`i128`** on the wire (`Price` signed, + `Quantity` unsigned magnitude + `Side`); **two-domain split** (exact `i128` / + analytical `f64`, convert at the strategy boundary); precision is **instrument + metadata**, raw-only wire; money ops are checked / no-bare-arithmetic / widen-to-256 + for notional; layered float-determinism scope (refines ADR-0012). - **Deterministic client-order-id** generation scheme. - **Core failover** (Aeron-Cluster-style hot standby) — future, documented in ADR-0005. From a03833440dea995bd6406981e59469202d94d567 Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Sat, 27 Jun 2026 18:03:28 +0000 Subject: [PATCH 7/8] docs(adr): close symbology (0025) + multi-broker environments (0024) Symbology grilling thread (ADR-0025): InstrumentId is the self-identifying canonical identity (externally anchored ISIN/FIGI/OCC where it exists, venue-qualified fallback, never guess a collapse); Symbol demoted to the venue ticker; off-wire Instrument record keyed (InstrumentId, Source) is the single home for ADR-0023 precision; fixed-size self-identifying name on the wire; deterministic rule + curated mapping with a cross-Source price-plausibility monitor; logged resolution + lifecycle (immutable id + logged succession, corporate actions as Core inputs). It reopened ADR-0011 -> ADR-0024: an Environment binds one-or-more same-safety-class execution backends, so cross-broker risk is in-fold; shadow/ live/off is per-strategy targeting, not an in-Core tag. Glossary gains InstrumentId, Instrument, Account, Position; Symbol redefined. Co-Authored-By: Claude Opus 4.8 (1M context) --- CONTEXT.md | 87 ++++++++-- ...t-multi-broker-binding-shadow-targeting.md | 69 ++++++++ ...-symbology-instrument-id-reference-data.md | 162 ++++++++++++++++++ ...6-23-from-scratch-architecture-grilling.md | 29 +++- 4 files changed, 329 insertions(+), 18 deletions(-) create mode 100644 docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md create mode 100644 docs/adr/0025-symbology-instrument-id-reference-data.md diff --git a/CONTEXT.md b/CONTEXT.md index 1f8f54b..bfc6ac3 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -5,23 +5,55 @@ trading engine. This file is a glossary only — no implementation details. ## Domain primitives +**InstrumentId**: +OATH's canonical, **venue-independent identity** for a tradable instrument, +independent of any one venue's ticker or internal id, so the same instrument +offered by different brokers collapses to a single `InstrumentId` — anchored, where +one exists, on an external standard (FIGI/ISIN-level), and **venue-qualified** where +no venue-independent identity exists, but always _normalized_, never a raw broker +string. It is the key for [Position]s (together with [Account]), [Signal]s, and +risk, and is **not** a data-stream routing key: market-data streams are +additionally keyed by their [Source], because the same `InstrumentId` priced by two +Sources is two distinct streams. **Self-identifying** — a stable, standards-based +(e.g. ISIN/FIGI/OCC) or deterministically-derived denotation that needs **no +OATH-private id registry** to say _which_ instrument it is, and never drifts, so the +Event Log stays interpretable forever. This is distinct from _self-explaining_: the +attributes (tick, currency, underlying, strike) still live in the [Instrument] +record, which is reference data _about_ an already-identified instrument, **not** a +fragile id↔meaning table. +_Avoid_: symbol, ticker, instrument, contract (for the identity). + **Symbol**: -OATH's canonical identifier for a tradable instrument, independent of any one -venue's ticker or internal id, so the same instrument offered by different -brokers collapses to a single `Symbol` (e.g. via perm_id / OpenFIGI). `Symbol` -is instrument _identity_ — used for positions, Signals, and risk — and is **not** -a data-stream routing key: market-data streams are additionally keyed by their -[Source], because the same `Symbol` priced by two Sources is two distinct streams. -_Avoid_: ticker, instrument, contract (for the canonical form). +The human-facing **ticker/label** for an instrument as a venue names it ("AAPL", +"ESM4") — a display attribute carried on the [Instrument] record, **never** the +identity (that is [InstrumentId]). Two venues may use different Symbols for the same +`InstrumentId`, and the same Symbol string can mean different instruments at +different venues — which is exactly why it cannot be the identity. +_Avoid_: identifier, id, key (for this label sense). **Source**: The Broker or Data-Provider Adapter that produced a given data stream — part of a market-data topic's routing key, never part of instrument identity. The same -`Symbol` carried by two Sources is two distinct streams (different prices, +[InstrumentId] carried by two Sources is two distinct streams (different prices, timestamps, and gaps); a consolidated/NBBO view is a _derived_ stream, never raw per-Source topics conflated. _Avoid_: venue, feed, provider (for this routing-key role). +**Instrument**: +The resolved _reference-data_ record for an [InstrumentId] — its [Symbol] (the +venue ticker), tick size, lot/min size, multiplier (contract size), quote currency, +asset class, and later expiry/strike/right/underlying. It is **not** identity (that +is [InstrumentId]) and never travels on the wire or the Event Log: ADR-0023 keeps +[Price]/[Quantity] precision-free raw `i128`/`u128`, and the `Instrument` is the +single home for the precision used to interpret them. The adapter **resolves it once +at the boundary** from a [Source]'s symbology / contract details, caches it, and any +consumer needing precision (order emission, display) looks it up. Resolution is keyed +by `InstrumentId` **per [Source]** — contract facts like tick can differ across +venues, so the same `InstrumentId` from two Sources may resolve to two `Instrument`s, +exactly as it is two market-data streams. +_Avoid_: contract, security, product (for this record); do not conflate with +[InstrumentId] (identity) or [Symbol] (ticker). + **Price**: The value per unit of an instrument, expressed in its quote currency. Can be **negative** (e.g. spreads, or a commodity in backwardation gone sub-zero). Its @@ -39,6 +71,24 @@ _Avoid_: size, amount, volume; a "signed quantity". The direction of an order or trade — buy or sell. _Avoid_: direction, way, sign. +**Account**: +A specific trading account at a [Broker] that owns [Position]s and receives +[Fill]s — the unit OATH settles, margins, and flattens against. Its identity is a +**normalized composite that includes the [Source]**, because account ids are only +unique within a broker (account `U123` at broker A is unrelated to `U123` at +broker B), and one Broker may expose **several** Accounts. There is no cross-broker +account consistency. +_Avoid_: portfolio, wallet, login, subaccount (for the canonical term). + +**Position**: +The held exposure in one [InstrumentId] at one [Account] — a [Quantity] magnitude +plus the [Side] that signs it, with signed exposure and average price derived. Keyed +by **`(Account, InstrumentId)`** and **never netted across Accounts**: a long at one broker +and a short at another are two Positions you must flatten separately, not a flat +zero. Net exposure across Accounts, brokers, or asset classes is a **derived +roll-up**, never a stored Position. +_Avoid_: holding, balance, inventory (for the canonical term). + **Timestamp**: A point in time, always UTC, with no timezone or offset attached. _Avoid_: datetime, date, clock time. @@ -122,11 +172,18 @@ _Avoid_: UI, dashboard, console, client. **Environment**: An isolated instance of the OATH topology — its own Core, Event Log, -portfolio/risk state, execution-adapter binding, data feeds, and Bus namespace — -so that several can run on one host without their orders, fills, positions, or -logs colliding. Its mode is its data feed × execution backend (e.g. live feed × -live account); all its feeds share one temporal profile (real-time, delayed-by-D, -or historical). +portfolio/risk state, data feeds, and Bus namespace — so that several run on one +host without their orders, fills, positions, or logs colliding. An Environment +occupies **one cell of the mode matrix**: a single **temporal profile** (real-time, +delayed-by-D, or historical) × a single execution **safety-class** (Simulated, +Paper, or Live). Within that cell it may bind **one or more execution backends of +the same safety-class** (e.g. two Live brokers), so cross-broker [Position]s and +risk are evaluated in **one Core** over the canonical [InstrumentId]. A differing +temporal profile or safety-class **always** forces a separate Environment (a +Simulated or Paper [Fill] must never perturb Live risk); same-cell books may still +be split into separate Environments **by choice** for risk isolation. Brokers +co-bound into one Environment **share its fate** — a Core fault or [Emergency Halt] +touches all of them. _Avoid_: instance, deployment, tenant, session. **Simulated Broker**: @@ -156,7 +213,7 @@ _Avoid_: production, real-money mode. **Signal**: A Strategy's proposal of a _desired target_ — the position or exposure it wants -in a `Symbol` — submitted to Core for a decision, never an Order. Idempotent and +in an [InstrumentId] — submitted to Core for a decision, never an Order. Idempotent and nettable: Core reconciles actual → target across strategies under risk, deciding whether, when, and how much to act. Carries the as-of freshness it was decided under and the proposing Strategy's identity. @@ -179,7 +236,7 @@ _Avoid_: execution, trade, transaction. **Emergency Halt**: An operator-tripped switch that puts Core's Risk Engine into cancel-all / flatten -mode — operational safety, not operator trading: it picks no Symbol, Side, or +mode — operational safety, not operator trading: it picks no InstrumentId, Side, or Quantity, only invoking risk's existing authority (a control of the risk loop, not an Order). The Supervisor performs the effectful trip and emits a logged Core input, so it is deterministic and replayable. diff --git a/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md b/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md new file mode 100644 index 0000000..3ff0531 --- /dev/null +++ b/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md @@ -0,0 +1,69 @@ +# Environments bind multiple same-safety-class brokers; shadow is targeting, not a tag + +_Refines [ADR-0011](0011-execution-environments-mode-isolation.md)._ + +ADR-0011 isolated trading **modes** into Environments but described each as binding +an execution backend in the **singular**, implying one broker per Environment. That +was an unexamined wording, not a derived requirement. ADR-0011's isolation rationale +is entirely (a) **temporal homogeneity** (one clock profile) and (b) the **cardinal +collision** — a Simulated or Paper [Fill] must never perturb Live risk. Neither +rationale distinguishes two *Live* brokers from each other: they share the same +clock, the same safety-class, and both move real money. + +So an Environment occupies **one cell of the mode matrix** — `(temporal profile × +execution safety-class: Simulated | Paper | Live)` — and within that cell binds +**one or more execution backends of the same safety-class**. Multiple Live brokers +(e.g. IBKR + Coinbase) co-bind into one Core, which makes **cross-broker net +exposure, global per-`Symbol` position tracking, and cross-asset budget rules** +(e.g. FX-50 % / equity-50 %) **first-class, in-fold risk** over the canonical +`Symbol` — not a derived cross-Environment view. [Position]s remain keyed +`(Account, Symbol)` and are **never netted across [Account]s**; the risk fold +aggregates over them. + +Cross-cell still always separates: a differing temporal profile or safety-class +forces distinct Environments (Live vs Paper vs Shadow vs Backtest). Same-cell books +may still be split **by choice** for risk isolation; several Live Environments may +run at once. + +**Shadow stays a separate Environment** (live feed read-only × Simulated), **not a +per-signal tag in the Live Core.** The Frontend's per-strategy **live / shadow / +off** is realized as strategy-lifecycle **targeting** — register the Strategy-Node +instance against the Live Environment, the Shadow Environment, or neither — over +ADR-0017 operational control + ADR-0013 admission. "Routed to the Simulated Broker" +is therefore **structural** (which Environment the strategy is registered to), never +a conditional the Live Core must evaluate correctly. + +## Considered options + +- _One broker per Environment_ (ADR-0011 as literally worded) — rejected: it pushes + cross-broker risk, positions, and budget into a cross-Environment derived view, + when the isolation rationale never required the split. +- _Shadow as a `SHADOW` tag on signals inside the Live Core_ — rejected: this is + exactly ADR-0011's already-rejected "environment-tagged messages," lowered to the + signal layer. One tag-check bug routes a shadow signal to the live account (the + cardinal collision), and a runaway shadow strategy would share the Live Core's + fold, Bus, and CPU. +- _Multi-broker Environment + shadow-as-targeting_ (chosen): cross-broker risk in + one fold; mode/safety isolation preserved **structurally**; promotion shadow → live + is a deliberate, audited re-registration rather than a toggle. + +## Consequences + +- **Shared fate is explicit and accepted.** Brokers co-bound into one Environment + share a Core; a Core fault or [Emergency Halt] affects all of them at once. This is + *inherent* to unified cross-broker risk — wanting cross-broker netting **is** + wanting shared fate. Risk isolation remains available by running separate + Environments. +- **`Account` becomes a first-class key** (new glossary term): a normalized + composite that includes [Source], with several allowed per [Broker]; Positions are + keyed `(Account, Symbol)`, never netted across Accounts. +- **Canonical `Symbol` is promoted to an in-fold risk key**, not merely a Frontend + join key — which directly shapes the symbology layer (the next thread). +- **Promotion shadow → live is a lifecycle re-registration** (ADR-0013 admission, + ADR-0017 control), deliberately *not* a one-click flag — going from zero-capital to + real-money is an explicit audited act. +- **Reconciliation (ADR-0006) runs per Account;** one Environment may run several + reconciliation loops. The Event Log records every co-bound broker's inputs for the + book, fused by the event-time merge (ADR-0012). +- Refines ADR-0011; consistent with ADR-0013 (strategy registration) and ADR-0017 + (operational-only control). diff --git a/docs/adr/0025-symbology-instrument-id-reference-data.md b/docs/adr/0025-symbology-instrument-id-reference-data.md new file mode 100644 index 0000000..818455d --- /dev/null +++ b/docs/adr/0025-symbology-instrument-id-reference-data.md @@ -0,0 +1,162 @@ +# Symbology: self-identifying `InstrumentId`, off-wire `Instrument` record, deterministic mapping + +ADR-0023 deferred precision, tick, lot, and multiplier to "instrument metadata the +symbology layer must supply." This ADR is that layer. It defines how OATH names a +tradable instrument, where its reference data lives, how a broker's native id maps to +the canonical name, and how all of it stays replay-safe (ADR-0005) and POD on the +wire (ADR-0020 / 0023). MVP is **IBKR-first, Equity-only**; the harder asset classes +are designed-for but not built. + +## Decision + +### 1. Identity — `InstrumentId` (self-identifying), `Symbol` demoted to ticker + +The canonical identity is **`InstrumentId`**: venue-**independent** where a +venue-independent identity exists (so the same instrument at two brokers collapses to +one id, and positions/risk aggregate — ADR-0024), **venue-qualified** where it does +not, but **always normalized**, never a raw broker string. It is **self-identifying** +— a stable, standards-based (ISIN/FIGI/OCC) or deterministically-derived denotation +that needs **no OATH-private id registry** to say *which* instrument it is, and never +drifts. This is distinct from *self-explaining*: it pins identity, not attributes. + +`Symbol` is **demoted** to the human venue **ticker** ("AAPL", "ESM4") — a display +attribute on the `Instrument` record, never the identity (a ticker is reused across +companies and differs across venues). `Source` (the producing adapter) remains a +market-data **routing** coordinate, never part of identity: the same `InstrumentId` +from two Sources is two streams. + +### 2. Reference data — the `Instrument` record, off-wire, typed by class + +The **`Instrument`** record is resolved reference data keyed **`(InstrumentId, +Source)`** (tick/lot are venue facts, so one `InstrumentId` may resolve to two +`Instrument`s, parallel to two streams). It is the **single home** for the ADR-0023 +precision and **never travels on the wire or Event Log**. It is a **shared core** +that every class has and the money math needs day one — `InstrumentId`, `Source`, +asset-class, quote currency, **tick**, **lot/min size**, **multiplier** — plus a +**per-asset-class typed tail** (expiry/strike/right/underlying), **not** a flat struct +of optionals (illegal states stay unrepresentable). MVP implements only the `Equity` +variant. + +### 3. Wire form — the self-identifying name itself (Choice A) + +The `InstrumentId` travels on the Bus and Event Log as its **fixed-size +self-identifying name** (e.g. `EQ:US0378331005`), not an opaque surrogate. The Event +Log is therefore interpretable forever with no external table. Attributes stay +off-wire in the `Instrument` record. A process **may** intern the name to a local +integer for hot-path lookups **provided that integer never crosses the wire**. The +fixed length is **TBD against the real IBKR `contractDetails`** (24 bytes is a +placeholder; option/FIGI keys may need more). A logged-assignment wire `u64` is +**reserved** for the dense-sharding future (ADR-0021 bucket) and is **not** MVP. + +### 4. Mapping & agreement — deterministic rule + curated overrides; never guess + +Each adapter (the anti-corruption layer) derives `InstrumentId` from the broker's +reference data via a **shared, versioned normalization ruleset**, so two adapters +**agree by construction** when both report the same anchor. A small **operator-curated +override table** handles no-anchor and known-conflict cases. The OpenFIGI *lookup +service* is a **deferred** fallback. + +**Safety invariant (day one):** **no external anchor ⇒ venue-qualified id ⇒ no +collapse, ever, until curated.** A *missed* collapse is a degraded view; a *false* +collapse silently merges positions — a money bug. **Never guess a collapse.** + +A **cross-`Source` price-plausibility monitor** is a backstop against gross +bad-collapse: per `InstrumentId`, prices across its Sources should agree within a +**band over a sustained window**, **compared in a common currency** (read from the +`Instrument` record). On breach it **alerts and quarantines** the suspect mapping — +**out-of-fold** (Telemetry/alert plane, ADR-0014), never a silent trade or instant +halt. It is a **broad net** (catches order-of-magnitude mismatches, not same-price +collisions) and complements — does not replace — curation and the ADR-0006 +reconciliation backstop. (Distinct from the *in-fold* price-sanity risk guard that +stops Core acting on an implausible quote, which ADR-0023 anticipates.) + +### 5. Resolution — a logged Core input, split by determinism need + +Bringing an instrument into an Environment is a **logged Core input** ("instrument +registered"), before the first message referencing it. Metadata splits two ways: + +- **Fold-relevant** (multiplier, and anything the canonical fold reads) → **logged**, + so Replay reproduces P&L bit-exactly (ADR-0005). +- **Boundary/display-only** (the `Symbol` ticker, adapter-side rounding tick, display + precision) → cached in the `Instrument` record, **never logged**. + +Timing: a **config-declared universe resolved at Environment start**, plus +**logged on-demand additions** for dynamic universes (option chains, scans). Every +addition — boot or on-demand — is the same logged registration; **no implicit lazy +resolution** (mirrors ADR-0013's strategy-admission handshake). + +### 6. Lifecycle — immutable id, logged change + +- **Ticker change** (`FB → META`) is a **non-event for identity** (the id anchors on + ISIN): just refresh the `Symbol` field off-log. *This is the payoff of the + ISIN anchor.* +- **Identity-changing events** (ISIN change on re-domicile/merger): `InstrumentId` is + **immutable**; mint a **new** id plus a **logged succession link** (old → new). +- **Metadata changes** (tick regime, contract spec): the `Instrument` record is + **time-versioned**; the fold-relevant subset changes only via a **logged "instrument + updated"** input; boundary fields refresh off-log. +- **Position-moving corporate actions** (splits, dividends): **logged Core inputs**, + translated by the adapter from the venue notice, applied deterministically by the + fold. +- **Futures rollover is position lifecycle, not identity:** each contract is its own + `InstrumentId` (expiry in the id); a "continuous contract" is a *derived* + strategy-side concept. + +MVP scope: equity **splits** + **ticker change** only; mergers/spin-offs/rights, +succession UX, and futures roll are deferred (the seams above are fixed now). + +### 7. Derivatives — reference by `InstrumentId`, composition off-wire + +Options/futures/spreads reference their **underlying and legs by `InstrumentId`**, +recursively; the composition (underlying, legs+ratios, strike/expiry/right) lives in +the **off-wire `Instrument` typed tail**, so the wire id stays a compact anchor even +for complex instruments. Non-tradable underlyings (an index) get a **reference-only** +`InstrumentId` (no `Account`/`Position` ever keyed by it). This is *metadata about a +self-identifying instrument*, so it does not reintroduce a registry dependency. MVP +builds none of it; the seam is reserved. + +## Considered options + +- _Opaque `u64` surrogate on the wire_ — rejected: the id↔instrument binding is + arbitrary and OATH-assigned, so the Event Log's meaning would depend on a mutable + private registry that, if lost/drifted/collided, makes audit history unreadable or + silently wrong. Self-identifying names avoid this; local interning recovers the + speed. +- _Venue-in-identity (`Symbol.Venue`, à la NautilusTrader)_ — rejected: unambiguous + and always-works, but yields **no** cross-venue collapse, which OATH wants for + cross-broker positions/risk and cross-asset budget rules (ADR-0024). OATH anchors + the identity component on an external standard instead. +- _Fat `Symbol` carrying metadata inline_ — rejected: bloats every topic key and log + record and contradicts ADR-0020 / 0023. Metadata lives off-wire in `Instrument`. +- _Flat `Instrument` struct with optional fields_ — rejected: "valid-by-convention" + (an equity with a strike set); the per-class typed tail keeps illegal states + unrepresentable (ADR-0023's "misapplication won't compile" ethos). +- _Central registry / security master now_ — deferred, **not** rejected: production + grade will want one, but because `InstrumentId` is self-identifying it is an + **additive** evolution of the resolution/curation seam (it supplies attributes and + curation, never the identity binding), not a redesign. MVP ships the decentralized + degenerate form (rule + local overrides). + +## Consequences + +- **Discharges ADR-0023's coupling:** the `Instrument` record is the single typed home + for precision/tick/lot/multiplier, taken only through instrument-requiring + functions. +- **`InstrumentId` is the symbol component of the ADR-0020 topic instance-key**, and + the key for `Position` `(Account, InstrumentId)` and `Signal`s. +- **Resolution and lifecycle become Event-Log inputs** (ADR-0005): instrument + registration, metadata updates, succession links, and corporate actions are all + logged, deterministic, replayable; the off-bus instrument registry is recovery / + bootstrap substrate. +- **Cross-broker collapse over `InstrumentId` (ADR-0024)** is sound because the id is + venue-independent where it can be and the no-anchor fallback never guesses. +- **ADR-0006 reconciliation remains the money-moving backstop**; the price-plausibility + monitor is a cheap out-of-fold complement. +- **New glossary terms:** `InstrumentId`, `Instrument`; `Symbol` redefined; `Account`, + `Position` (added with ADR-0024). +- **Production central security master** is the anticipated evolution (parked). +- **Parked sub-questions:** fixed-size id length vs real IBKR contracts; **combo + identity** (structural-encode vs leg-decompose); the central security-master + service; OpenFIGI fallback; the corporate-action taxonomy + succession UX; and the + durable `Instrument` store as part of the repository-backend decision (ADR-0009 log↔ + repository split — pure-Rust, not Postgres). diff --git a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md index 1ceaf68..7e5076f 100644 --- a/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md +++ b/docs/superpowers/specs/2026-06-23-from-scratch-architecture-grilling.md @@ -8,7 +8,14 @@ parked-question cluster — **closed 2026-06-27** (ADRs 0020–0022): trait deli classes × access patterns, two-level routing, and the `Reliable` order-path failure model. **Numeric types** parked question — **closed 2026-06-27** (ADR-0023): fixed-point always-`i128`, exact/analytical two-domain split, -instrument-sourced precision, checked money ops. +instrument-sourced precision, checked money ops. **Symbology** parked question — +**closed 2026-06-27** (ADR-0025): self-_identifying_ `InstrumentId` (externally +anchored / venue-qualified fallback, never guess a collapse), off-wire `Instrument` +reference record, fixed-size name on the wire, deterministic rule + curated mapping, +logged resolution + lifecycle. It also reopened and refined **ADR-0011 → ADR-0024** +(an Environment binds **one-or-more same-safety-class** execution backends, so +cross-broker risk is in-fold; shadow/live/off is per-strategy **targeting**, not an +in-Core tag) and added glossary `Account` / `Position`. ## The question @@ -156,8 +163,24 @@ cross-process complexity to buy crash containment and hot-pluggability. - **Event Log / repository backend**: parquet + DataFusion / DuckDB candidate (keep the log↔repository split from ADR-0009). - **Snapshot** cadence and contents (recovery substrate). -- **Symbology** design: canonical identity (perm_id/OpenFIGI) + per-adapter - mapping. +- ~~**Symbology** design: canonical identity (perm_id/OpenFIGI) + per-adapter + mapping.~~ — **resolved (ADR-0025)**: **`InstrumentId`** = self-_identifying_ + canonical identity (externally-anchored ISIN/FIGI/OCC where it exists, + venue-qualified fallback, **never guess a collapse**); **`Symbol`** demoted to venue + ticker. Off-wire **`Instrument`** record keyed `(InstrumentId, Source)` (shared core + + per-asset-class typed tail) is the single home for ADR-0023 precision. Wire form = + fixed-size self-identifying name (Choice A; local-only interning). Mapping = + deterministic versioned rule + curated overrides + cross-`Source` price-plausibility + monitor. Resolution + lifecycle (immutable id + logged succession; corporate actions + as Core inputs; time-versioned metadata) are **logged Core inputs**. _Also drove + **ADR-0024** (multi-broker same-safety-class Environments; shadow = targeting) and + added glossary `Account` / `Position`._ + - **New parked sub-questions:** fixed-size id **length** vs real IBKR contracts; + **combo identity** (structural-encode vs leg-decompose); **central security-master** + service (production-grade evolution of the resolution/curation seam — additive, + does not change identity); **OpenFIGI** anchor-lookup fallback; the + **corporate-action taxonomy** + succession UX; durable **`Instrument` store** as + part of the Event-Log / repository backend decision. - ~~`Price`/`Quantity` numeric needs per asset class (crypto/wei) — when to move the inner type off `rust_decimal`.~~ — **resolved (ADR-0023)**: drop `rust_decimal`; fixed-point **always-`i128`** on the wire (`Price` signed, From 796dc01047849e1e7fbac28e0a26d1c0b20435c7 Mon Sep 17 00:00:00 2001 From: NotAProfDev <84450364+NotAProfDev@users.noreply.github.com> Date: Sat, 27 Jun 2026 18:38:58 +0000 Subject: [PATCH 8/8] docs(adr): resolve CodeRabbit review on #45 Address internal cross-ADR inconsistencies surfaced in review: - 0002: payload contract is `Serialize`-only; POD is backend-specific zero-copy discipline (ADR-0020), not the universal Bus contract. - 0003: drop stale `i64`/bignum numeric example; point at the settled fixed-point `i128`/`u128` model (ADR-0023). - 0012: clarify that the live path is buffering-free only when `L = 0`; `L > 0` buffers up to `L`. - 0017: record the "halt as-of seq N" fact before/with the effectful trip so the deterministic-replay claim survives a crash window. - 0020: loan boundary materializes to an owned payload; `Copy` POD is a backend fast-path discipline, not a public-API bound. - 0024: key risk/positions by `InstrumentId`, not `Symbol` (demoted to a venue ticker by ADR-0025), so cross-broker collapse holds. - 0009 + crate-topology plan: tag plain-text fences as `text`; add the missing `oath-model` edges for `oath-core`/`oath-strategy-host` so the Mermaid graph matches the dependency table. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...002-backend-agnostic-bus-canonical-message-model.md | 6 +++--- docs/adr/0003-canonical-model-adapter-translation.md | 7 +++---- ...09-crate-topology-spine-inverted-process-aligned.md | 2 +- .../0012-strategy-input-fusion-event-time-parity.md | 3 ++- .../0017-frontend-control-plane-operational-only.md | 6 +++--- .../0020-bus-trait-delivery-classes-access-patterns.md | 7 ++++--- ...nvironment-multi-broker-binding-shadow-targeting.md | 10 +++++----- .../plans/2026-06-27-crate-topology-adr-0009.md | 6 ++++-- 8 files changed, 25 insertions(+), 22 deletions(-) diff --git a/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md b/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md index 7fe9064..cec891e 100644 --- a/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md +++ b/docs/adr/0002-backend-agnostic-bus-canonical-message-model.md @@ -19,9 +19,9 @@ they move bytes, never in the data they carry. ## Consequences -- Message payloads are designed to the **intersection** of backend constraints: - fixed-layout, `#[repr(C)]` plain-old-data (for zero-copy) **and** serializable - (for network backends) — roughly a `Pod + Serialize` bound. +- Message payloads are `Serialize` (the shared trait bound); backends that + support zero-copy may additionally impose fixed-layout, `#[repr(C)]` + plain-old-data (POD) constraints as backend-specific discipline (ADR-0020). - The trait must be designed to the **stricter** ownership contract: zero-copy hands back a *loaned*, lifetime-bounded sample; network backends hand back an *owned* value. Modelling the borrowed/lifecycle-bounded case keeps zero-copy. diff --git a/docs/adr/0003-canonical-model-adapter-translation.md b/docs/adr/0003-canonical-model-adapter-translation.md index d2339cd..421f8d0 100644 --- a/docs/adr/0003-canonical-model-adapter-translation.md +++ b/docs/adr/0003-canonical-model-adapter-translation.md @@ -13,9 +13,8 @@ risk. - Every adapter carries a translation layer (precision conversion, symbol resolution, mapping tables) — accepted cost, paid to keep the core clean. -- `Price`/`Quantity` are newtypes over a swappable inner numeric type - (`rust_decimal` for the MVP — 16-byte, `Copy`, heap-free, so zero-copy-safe); - the inner type can later become fixed-point `i64` or a bignum at compile time - without touching call sites. +- `Price`/`Quantity` are newtypes over a swappable inner numeric type, so the + representation can change at compile time without touching call sites (later + settled on fixed-point `i128`/`u128` — ADR-0023). - Translation lives in the adapter process, so a malformed-message bug in one venue's translation cannot corrupt another's. diff --git a/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md b/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md index 0acb128..afe2632 100644 --- a/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md +++ b/docs/adr/0009-crate-topology-spine-inverted-process-aligned.md @@ -20,7 +20,7 @@ Two naming rules make the structure self-describing: ## Target layout -``` +```text crates/ model/ oath-model primitives + message payloads diff --git a/docs/adr/0012-strategy-input-fusion-event-time-parity.md b/docs/adr/0012-strategy-input-fusion-event-time-parity.md index 75f9afe..beee782 100644 --- a/docs/adr/0012-strategy-input-fusion-event-time-parity.md +++ b/docs/adr/0012-strategy-input-fusion-event-time-parity.md @@ -5,7 +5,8 @@ The strategy framework delivers a strategy's subscribed Bus topics as a single read-only **latest-value view** (the `StateView` analogue, ADR-0008); windowed / correlation joins are strategy-owned state for MVP. Live ordering uses a configurable per-Environment **lateness bound `L`** (default small; `L = 0` for -latency-critical Live), so the live decision path adds no buffering latency, and +latency-critical Live), so latency-critical Live can set `L = 0` to avoid +buffering while `L > 0` buffers the live decision path by up to `L`, and each Environment records its consumed input stream in **ingestion order** (a compact index over the durable Bus topics) so a recorded run replays bit-exactly regardless of `L`. diff --git a/docs/adr/0017-frontend-control-plane-operational-only.md b/docs/adr/0017-frontend-control-plane-operational-only.md index ef4db69..71f0b17 100644 --- a/docs/adr/0017-frontend-control-plane-operational-only.md +++ b/docs/adr/0017-frontend-control-plane-operational-only.md @@ -11,9 +11,9 @@ path ever bypasses risk. The one order-affecting control in MVP is an **emergenc halt**, modeled not as operator trading but as a **trip of the Risk Engine's existing cancel-all / flatten authority** (ADR-0004): the operator picks no instrument and no size, only trips the switch. Mechanically it follows ADR-0013's -registration template — the Supervisor performs the effectful trip, then emits a -logged "halt as-of seq N" fact into Core's Event Log — so the halt is -deterministic, replayable, and attributable. +registration template — the Supervisor durably records the "halt as-of seq N" +fact into Core's Event Log before (or as part of) the effectful trip — so the +halt is deterministic, replayable, and attributable even across a crash. ## Considered options diff --git a/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md b/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md index 6b377c2..eed06b4 100644 --- a/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md +++ b/docs/adr/0020-bus-trait-delivery-classes-access-patterns.md @@ -23,9 +23,10 @@ convention so they ride every backend including the fast path; user-defined messages need only `Serialize` (and opt into a backend's discipline for its fast path). **Receive is a loan guard** (`Sample: Deref`, drop returns the slot) — a real shared-memory loan where the backend supports it, an owned value -otherwise. A loan **may not cross three boundaries** — retention, thread hand-off, -`.await` — at each of which the consumer copies out to **owned `M: Copy` POD** and -drops the loan. (Core's drain — read field, append bytes to the Event Log, fold, +otherwise. A loaned `Sample` **may not cross** retention, thread hand-off, or +`.await`; the consumer **materializes it into an owned payload** before each +boundary, then drops the loan (owned payloads move normally — `Copy` POD is a +backend-specific fast-path discipline, not a public-API bound). (Core's drain — read field, append bytes to the Event Log, fold, drop — is the inspect-and-discard hot path that earns the guard its public place.) **Send is `publish(&M)`**; **loan-to-write (`SampleMut`) is an opt-in** zero-copy-construct path for large/bulk payloads (order-book depth, history pages). diff --git a/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md b/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md index 3ff0531..0ff9e66 100644 --- a/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md +++ b/docs/adr/0024-environment-multi-broker-binding-shadow-targeting.md @@ -14,10 +14,10 @@ So an Environment occupies **one cell of the mode matrix** — `(temporal profil execution safety-class: Simulated | Paper | Live)` — and within that cell binds **one or more execution backends of the same safety-class**. Multiple Live brokers (e.g. IBKR + Coinbase) co-bind into one Core, which makes **cross-broker net -exposure, global per-`Symbol` position tracking, and cross-asset budget rules** +exposure, global per-`InstrumentId` position tracking, and cross-asset budget rules** (e.g. FX-50 % / equity-50 %) **first-class, in-fold risk** over the canonical -`Symbol` — not a derived cross-Environment view. [Position]s remain keyed -`(Account, Symbol)` and are **never netted across [Account]s**; the risk fold +`InstrumentId` — not a derived cross-Environment view. [Position]s remain keyed +`(Account, InstrumentId)` and are **never netted across [Account]s**; the risk fold aggregates over them. Cross-cell still always separates: a differing temporal profile or safety-class @@ -56,8 +56,8 @@ a conditional the Live Core must evaluate correctly. Environments. - **`Account` becomes a first-class key** (new glossary term): a normalized composite that includes [Source], with several allowed per [Broker]; Positions are - keyed `(Account, Symbol)`, never netted across Accounts. -- **Canonical `Symbol` is promoted to an in-fold risk key**, not merely a Frontend + keyed `(Account, InstrumentId)`, never netted across Accounts. +- **Canonical `InstrumentId` is promoted to an in-fold risk key**, not merely a Frontend join key — which directly shapes the symbology layer (the next thread). - **Promotion shadow → live is a lifecycle re-registration** (ADR-0013 admission, ADR-0017 control), deliberately *not* a one-click flag — going from zero-capital to diff --git a/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md b/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md index 08440fb..a3e72d2 100644 --- a/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md +++ b/docs/superpowers/plans/2026-06-27-crate-topology-adr-0009.md @@ -1342,8 +1342,10 @@ graph TD core --> risk core --> exe core --> por + core --> model strathost[oath-strategy-host] --> stratapi + strathost --> model cli[oath-cli] --> model sup[oath-supervisor] --> model ``` @@ -1455,7 +1457,7 @@ Expected: all gates pass — crucially `typos` (new prose), `doc` (no broken int cargo metadata --format-version 1 --no-deps | jq -r '.packages[].name' | sort ``` Expected — exactly these 16 names: -``` +```text oath-adapter-api oath-adapter-net-api oath-bus-api @@ -1478,7 +1480,7 @@ And the directory layout: find crates -name Cargo.toml | sort ``` Expected: -``` +```text crates/adapter/api/Cargo.toml crates/adapter/net/api/Cargo.toml crates/bus/api/Cargo.toml