Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 44 additions & 2 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,14 @@ co-bound into one Environment **share its fate** — a Core fault or [Emergency
touches all of them.
_Avoid_: instance, deployment, tenant, session.

**EnvironmentId**:
The stable, operator-assigned identity of an [Environment] — the same handle that
names its Bus namespace — recorded at genesis so it is replay-stable, and
administered unique across any Environments that could target the same [Broker]
[Account]. It prefixes order identities so two Cores can never collide at a shared
venue.
_Avoid_: name, tag, instance id.

**Simulated Broker**:
A Broker-adapter backend that fills Orders internally against the Environment's
own market-data feed instead of routing to a real venue. The execution backend
Expand Down Expand Up @@ -226,10 +234,44 @@ decides; the Kernel performs the actions. Internal to Core, never sent on the Bu
_Avoid_: verdict, judgment, command, ruling.

**Order**:
An instruction Core sends to a broker to buy or sell, after the Risk Engine
approves. Carries the freshness/validity context it was decided under.
The logical buy/sell order Core works at a [Broker], identified by its [Order Id]
and stable across a lifecycle of [Order Instruction]s (place/amend/cancel) until it
reaches a terminal state (filled/cancelled/rejected). Core sends it only after the
Risk Engine approves; it carries the freshness/validity context it was decided
under.
_Avoid_: trade, transaction.

**Order Instruction**:
A single command Core issues against an [Order] — _place_ (open), _amend_ (modify
price/quantity), or _cancel_. Each is identified by its own [Order Instruction Id]
and supersedes the previous instruction on the same Order. The unit the Bus carries
from Core to a [Broker] adapter.
_Avoid_: order modification, request, message (for this command).

**Order Id**:
Core's stable, internal identity for an [Order] — constant across its whole
lifecycle of [Order Instruction]s. Derived deterministically so [Replay] regenerates
it identically; never sent on the wire. The anchor every per-instruction and
broker-assigned id resolves back to.
_Avoid_: client order id (that is per-instruction), broker order id (that is the
venue's).

**Order Instruction Id**:
The identity of one [Order Instruction] — unique per instruction, reused only on
retransmission of the same instruction. Derived deterministically (so [Replay]
regenerates it) and the join key for idempotent submission and crash
reconciliation: the broker's dedup key and the "what happened to this?" question.
The adapter renders it to the venue's per-message id (e.g. FIX `ClOrdID`).
_Avoid_: client order id (ambiguous — it is per-instruction, not per-order),
message id, event id (reserved for [Domain Event]).

**Broker Order Id**:
The venue-assigned identity for an [Order], learned from the broker's
acknowledgements — a logged Core input, so replay-stable _as data_, not derived. One
[Order] may accumulate several across its life, since some venues re-issue it on
each amend. Used for venue-keyed queries and cross-checking the broker's books.
_Avoid_: order id (that is Core's internal id), exchange ref, venue id.

**Fill**:
A partial or complete execution of an order, reported by a broker.
_Avoid_: execution, trade, transaction.
Expand Down
141 changes: 141 additions & 0 deletions docs/adr/0026-order-identity-three-ids-deterministic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Order identity: three deterministic ids, generation-stamped, Core-owned chaining

[ADR-0006](0006-broker-reconciliation-contract.md) made a deterministic "client
order id" the join key for crash reconciliation but left the **generation scheme**
open. This ADR fills that hole and, in doing so, splits the one "client order id"
into the **three** ids an order actually needs, fixes how each is made
replay-deterministic, and places the lifecycle **chaining** in Core.

## Three ids, one per role

A single id conflates three independent concerns. We model them separately:

| Id | Role | Lifecycle | Determinism class | Adapter renders to (FIX) |
|---|---|---|---|---|
| **Order Id** | Core's stable handle for the logical order | one per order, whole lifecycle | derived-deterministic | — (Core-internal, never on the wire) |
| **Order Instruction Id** | dedup + crash-reconciliation join key | one per place/amend/cancel; reused only on retransmit | derived-deterministic | `ClOrdID` (t11), chained via `OrigClOrdID` (t41) |
| **Broker Order Id** | venue-assigned handle; venue-keyed queries | 1‑to‑N per order (venues may re-id on amend) | **learned**, not derived | `OrderID` (t37) |

The **Order Instruction Id** is the join key ADR-0006 was pointing at — FIX
`ClOrdID` is per-message, which is exactly why a single "order id" was too coarse.
The **Broker Order Id** is a different determinism class: the broker mints it, so it
is never regenerated by computation — it arrives on an acknowledgement, which is a
logged Core input, and is therefore replay-stable **as data** (like the
`IncarnationStarted` marker below), not by derivation. Core cannot use it as a
primary key (it does not exist until the ack, and in-flight orders may never get
one), which is precisely why the internal **Order Id** must exist.

These are **roles, not wire formats.** OATH is venue-agnostic (ADR-0003); the FIX
column is one adapter's rendering. A non-FIX adapter maps the same roles onto its
own protocol (IBKR `orderId`/`permId`; crypto `client_order_id`). FIX `MsgSeqNum`
(t34) is **not** in this model at all — it is the FIX _session-transport_ sequence,
lives inside the adapter, and its Core↔adapter analogue is the Bus `Reliable` class
(ADR-0020), not an id we carry.

## Deterministic construction

Both internal ids share the shape **`(EnvironmentId, generation, counter)`**:

- **`EnvironmentId`** — the Environment's stable, operator-assigned identity
(the same handle that names its Bus namespace), recorded at genesis so it is
replay-stable, and administered unique across any Environments that could target
the same broker `Account`. It guarantees two Cores can never collide at a shared
venue.
- **`generation`** — a monotonic counter bumped **once per real boot** as an
effect, and **logged as an `IncarnationStarted { generation }` marker** before any
order is emitted. Replay never bumps it: each marker simply _sets_ the folded
generation, so orders are regenerated under the generation current at their log
position. This is what makes cross-session uniqueness deterministic — the boot
count is a real-world fact replay cannot recompute, so it is captured, not
derived. A crash between bumping and durably writing the marker is idempotent
(the next boot re-derives the same generation, and log-before-send guarantees no
order shipped under an un-durable generation).
- **`counter`** — a folded monotonic counter, **reset per incarnation** (the
generation already disambiguates across incarnations). There are **two**: an
order-counter (bumped per new logical order) and an instruction-counter (bumped
per emitted instruction). Both are owned by the single-writer Kernel.

An order created in generation `G` keeps its `(Env, G, …)` ids across a later
crash — replay re-runs `G`'s slice under generation `G` — while genuinely new work
after recovery uses `G+1`. Survivors are never renumbered.

## Why this composition

- **Counter, not hash.** A content-hash collides on _genuine_ duplicates (two
legitimately identical orders), and idempotent submit would then silently eat the
second as a retransmission. A folded counter never does; its determinism comes
from being _in the fold_, not from statelessness.
- **One Environment-wide order-counter, not per-`Account`.** One Environment is one
single-writer Kernel, so a single counter makes every order unique across _all_
bound brokers (ADR-0024), with deterministic emission order. Per-`Account`
counters would force `Account` into the id and add keyed state for no uniqueness
gain.
- **`Account`/`Source` is not in the id.** Uniqueness is already covered; outbound
reconciliation routing is folded on the order record; inbound attribution is
carried by the transport (one adapter process per venue). `Account` in the id
would buy only observability — so it may be rendered into the venue id _string_
for self-describing logs, without being part of the canonical identity.

## Two resolution indexes

Because instruction ids change per instruction and broker ids may change per amend,
Core maintains two many-to-one indexes into the stable Order Id, both rebuilt on
replay (`instruction` ids regenerated by the counter; `broker` ids read back from
logged acks):

- `order_instruction_id → order_id` (all of `M1…Mn → O`)
- `broker_order_id → order_id` (all of `B1…Bk → O`)

The broker index must hold the **set**, not just the current id: a late partial
fill on a pre-amend broker id must still attribute to the same order.

## Chaining: Core decides, the adapter renders

The supersession chain (`M1←M2←M3…`) is **folded state Core owns**. The canonical
`OrderInstruction` message Core sends to the adapter carries the reference
explicitly — `{ order_id, instruction_id, supersedes, broker_order_id? }` for
amend/cancel — where `supersedes` is the **prior Order Instruction Id** (a canonical
concept, not FIX `OrigClOrdID`). The adapter translates that reference to the
venue's mechanism: FIX `OrigClOrdID`/`OrderID`; IBKR same-`orderId` resend; crypto
`origClientOrderId`; or an emulated cancel-then-new (with venue-internal sub-ids
derived from the instruction id) where the venue lacks atomic replace.

This keeps a single source of truth (the chain is not duplicated in
non-replayable adapter state), keeps Core venue-agnostic, and lets the adapter stay
restart-resilient — its only persisted state is the ADR-0006 dedup / id-mapping
table, a separate concern from chaining. **Retransmit reuses the same Order
Instruction Id** (the case idempotent submit exists for); only new intent draws a
fresh one.

## Wire format

Rendering the canonical id to a venue-legal id is the **adapter's** job (ADR-0003).
Default is a **stateless deterministic encoding** (a compact base-N render of the
triple that fits common `ClOrdID` budgets, ~≤20 chars), so the common case needs no
per-order table. A **persisted bijective mapping** is the per-adapter fallback for
pathologically tight venues — the ADR-0006 emulation path, not the norm.

## Considered and rejected

- **UUIDv7 minted at decision time** — non-deterministic (embeds wall-clock +
randomness), so replay computes a different id and reconciliation shatters; would
also forfeit ADR-0012 fresh-backtest parity and overflows `ClOrdID` length. Its
one virtue, cross-session uniqueness, is captured by the logged `generation`
instead.
- **Content-hash ids** — collide on genuine duplicates (see above).
- **Raw Event Log sequence number as the id** — one logged input fans out into many
orders (flatten-across-N; ADR-0022 retries), so a single seq lacks granularity.
- **FIX `MsgSeqNum` (t34) as the join key** — session-transport, resets per session,
adapter-assigned; not replay-deterministic and below the canonical model.
- **Per-`Account` counters** — force `Account` into the id and add keyed Kernel
state for no uniqueness benefit under a single-writer Kernel.

## Relationships

Refines **ADR-0006** (its "client order id" = the **Order Instruction Id**). Rests
on **ADR-0005/0008** (folded single-writer state makes the ids deterministic),
**ADR-0003** (adapter renders venue ids), **ADR-0024** (Env-wide counter spans
co-bound brokers), and **ADR-0020** (Bus `Reliable` is the transport gap-detection
layer, the t34 analogue). Glossary: `EnvironmentId`, `Order`, `Order Instruction`,
`Order Id`, `Order Instruction Id`, `Broker Order Id` in
[CONTEXT.md](../../CONTEXT.md).
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ single-process, statically-composed monolith (`oath-engine` composes all layers
via `EngineBuilder`; dep graph `engine → {everything}`). The design we converged
on is a **single-host, multi-process, event-sourced system communicating over a
swappable message Bus**. The current crate graph is aimed at the monolith we
decided *not* to build — revising it is open branch (A) below.
decided _not_ to build — revising it is open branch (A) below.

## Decisions made (see docs/adr/)

Expand All @@ -50,7 +50,7 @@ decided *not* to build — revising it is open branch (A) below.
- **[ADR-0005](../../adr/0005-single-writer-event-sourced-core.md)** —
Single-writer, event-sourced, deterministic Core. MVP = single-threaded kernel +
offloaded async I/O (NautilusTrader-validated); disruptor pipeline is a
later, *measured* optimization. Replay = fold over the Event Log.
later, _measured_ optimization. Replay = fold over the Event Log.
- **[ADR-0006](../../adr/0006-broker-reconciliation-contract.md)** — Broker is
the source of truth. Recovery = replay + reconcile, joined by a client order
id. Log-before-send ordering invariant. Idempotent submit + queryable order
Expand Down Expand Up @@ -127,7 +127,7 @@ persistence/recovery, transport).

NautilusTrader independently uses nearly the same model (single-threaded
deterministic kernel, offloaded I/O, centralized cache, no sharding,
strategy→risk→exec). Key *deliberate* divergence: Nautilus is single-process
strategy→risk→exec). Key _deliberate_ divergence: Nautilus is single-process
with a non-swappable bus; OATH is multi-process with a swappable bus — paying
cross-process complexity to buy crash containment and hot-pluggability.

Expand Down Expand Up @@ -167,8 +167,8 @@ cross-process complexity to buy crash containment and hot-pluggability.
mapping.~~ — **resolved (ADR-0025)**: **`InstrumentId`** = self-_identifying_
canonical identity (externally-anchored ISIN/FIGI/OCC where it exists,
venue-qualified fallback, **never guess a collapse**); **`Symbol`** demoted to venue
ticker. Off-wire **`Instrument`** record keyed `(InstrumentId, Source)` (shared core
+ per-asset-class typed tail) is the single home for ADR-0023 precision. Wire form =
ticker. Off-wire **`Instrument`** record keyed `(InstrumentId, Source)` (shared core +
per-asset-class typed tail) is the single home for ADR-0023 precision. Wire form =
fixed-size self-identifying name (Choice A; local-only interning). Mapping =
deterministic versioned rule + curated overrides + cross-`Source` price-plausibility
monitor. Resolution + lifecycle (immutable id + logged succession; corporate actions
Expand All @@ -188,7 +188,19 @@ cross-process complexity to buy crash containment and hot-pluggability.
analytical `f64`, convert at the strategy boundary); precision is **instrument
metadata**, raw-only wire; money ops are checked / no-bare-arithmetic / widen-to-256
for notional; layered float-determinism scope (refines ADR-0012).
- **Deterministic client-order-id** generation scheme.
- ~~**Deterministic client-order-id** generation scheme.~~ — **resolved
(ADR-0026)**: three ids by role — **Order Id** (internal, lifecycle-stable),
**Order Instruction Id** (per place/amend/cancel; the dedup + reconciliation join
key), **Broker Order Id** (venue-assigned, learned). Internal ids are
derived-deterministic `(EnvironmentId, generation, counter)`; `generation` is
bumped once per boot and logged as an `IncarnationStarted` marker (reproduced by
folding, never recomputed). Env-wide order-counter; `Account`/`Source` not in the
id. Two many-to-one indexes (instruction→order, broker→order). Chaining is
Core-owned (`supersedes` in the canonical `OrderInstruction`); the adapter renders
to the venue (FIX `ClOrdID`/`OrigClOrdID`/`OrderID`). Refines ADR-0006.
- **New parked sub-question:** the **amend-before-ack** edge (amending an [Order]
whose place is not yet acknowledged, so no `Broker Order Id` exists) — order
state-machine territory, separate from identity.
- **Core failover** (Aeron-Cluster-style hot standby) — future, documented in
ADR-0005.
- **Strategy sandbox** (Branch B): MVP relies on process isolation (Strategy Node)
Expand Down