diff --git a/docs/adr/0029-network-adapter-stack-transport-split-compile-time-composition.md b/docs/adr/0029-network-adapter-stack-transport-split-compile-time-composition.md new file mode 100644 index 0000000..4bd97cf --- /dev/null +++ b/docs/adr/0029-network-adapter-stack-transport-split-compile-time-composition.md @@ -0,0 +1,151 @@ +# Network adapter stack: transport-split crates and compile-time `Service`/`Layer` composition + +[ADR-0009](0009-crate-topology-spine-inverted-process-aligned.md) placed +`oath-adapter-net-api` in the topology as "HTTP/WS composition primitives," and the +skeleton shipped a Tower-shaped composition core (`Service`, `Layer`, +`ServiceBuilder`, `Stack`, `Identity`) plus a coarse `ErrorKind` / `HasErrorKind` +classifier in one crate. Landing the first [Broker](../../CONTEXT.md) — Interactive +Brokers' Client Portal Web API — forces the question that skeleton deferred: *one net +crate, or many?* This ADR **splits the net layer by transport**, fixes what is +universal enough to live in a shared kernel versus what is transport-specific, and +states the binding-time discipline the whole stack inherits. The HTTP data plane and +the resilience/pacing layers are specified in +[ADR-0030](0030-http-transport-contract-wire-bytes-streaming-composition.md) and +[ADR-0031](0031-http-resilience-venue-pacing.md); this ADR is the structural, +**cross-transport** decision they both rest on. + +## Decision + +### 1. Split by transport, over a transport-neutral kernel + +A venue Adapter speaks more than one wire protocol — IBKR's Client Portal is REST +**and** a streaming WebSocket — and those protocols have *fundamentally different +interaction shapes* (§2). So the net layer is not one crate; it is a kernel plus one +contract crate per transport, plus leaf backends: + +```text +oath-adapter-net-api kernel — transport-neutral, std-only: + Layer, ServiceBuilder, Stack, Identity, + ErrorKind, HasErrorKind, Timer + ├── oath-adapter-net-http-api HTTP/REST contracts (depends on net-api) + │ └── oath-adapter-net-http-hyper leaf backend (hyper-util + rustls) + └── oath-adapter-net-ws-api WebSocket contracts (future; depends on net-api) + └── oath-adapter-net-ws- leaf backend (future) +``` + +This preserves ADR-0009's spine-inverted direction: `net-api` is the most-depended-on +contract, the per-transport `*-api` crates are narrower contracts on top, and concrete +backends implement them. A future gRPC/FIX/multicast transport is a new `*-api` crate +on the same kernel, never a fork. + +### 2. `Service` is not universal — it lives in `net-http-api`, not the kernel + +`Service` models **request → one reply**. That fits REST and unary RPC, but it +does **not** model the other transports' core operation: + +| Kernel symbol | REST | WebSocket | FIX / TCP session | UDP multicast (recv-only) | +|---|---|---|---|---| +| `Layer` / `ServiceBuilder` / `Stack` / `Identity` | ✓ | ✓ | ✓ | ✓ | +| `ErrorKind` / `HasErrorKind` | ✓ | ✓ | ✓ | ~ | +| `Service` (request→one reply) | ✓ | ✗ subscription yields *many* frames | ✗ async session | ✗ no request at all | + +A WebSocket subscription is "subscribe → stream of frames"; a multicast feed is pure +receive. Forcing those into request/reply is a lie. So **`Service` is a +*connection-shape* contract, not a kernel primitive**, and it lives in `net-http-api` +(the first request/reply transport). WS will define its own streaming contract in +`net-ws-api`. The explicit **no**: there is no shared `Service` in the kernel. + +`Service` is transport-*neutral* (it names no HTTP type), so were a second request/reply +transport to appear (gRPC, SSE), `Service` is hoisted into a `net-req-reply-api` crate +shared by both — *not* left in `net-http-api` forcing gRPC to depend on HTTP. We do not +build that crate now (YAGNI); we have one request/reply transport. + +### 3. The composition machinery is `Service`-free and stays in the kernel + +`Layer` carries **no `Service` bound** — it wraps *anything*. `ServiceBuilder`, +`Stack`, and `Identity` likewise compose an arbitrary `S`. That is exactly why they +belong in the kernel: the same machinery composes an HTTP `Service` stack today and a +WS subscription stack tomorrow. `ErrorKind` / `HasErrorKind` are coarse, wire-neutral +classifications (`Timeout`, `Connection`, `Throttled`, `Auth`, `Client`, `Server`, +`Unknown`) every transport's layers branch on; they stay in the kernel too. + +With `Service` removed, the kernel carries **no external dependencies** — std only. A +dependency-free kernel is the signal that the cut is clean. + +### 4. `Timer` is a kernel contract, not a runtime + +Timing layers (`Timeout`, `Retry` backoff, `RateLimit` refill, `CircuitBreaker` +cooldown — ADR-0031) need a clock, which collides with the `*-api` crates' zero-runtime +charter. Resolution: a minimal **`Timer` trait in the kernel** + +```rust +pub trait Timer: Clone + Send + Sync { + fn sleep(&self, dur: Duration) -> impl Future + Send; + fn now(&self) -> Instant; // token-bucket / cooldown elapsed-time reads +} +``` + +A trait is not a runtime, so the charter holds and the kernel stays std-only. Timing +*logic* lives with the transport that uses it (`Timeout`/`Retry`/`RateLimit` wrap a +`Service`, so they live in `net-http-api`), generic over `net-api::Timer`; the +**tokio-backed `Timer` impl lives in the leaf backend** (`net-http-hyper`). `Timer` is +in the kernel rather than `net-http-api` because WS reconnect/heartbeat will need the +same clock — it names no transport. Bonus: a mock `Timer` makes every timing layer +deterministically testable without real sleeps. + +### 5. Compile-time binding, no `dyn` — RPITIT throughout + +`Service::call` returns `impl Future` (RPITIT) — no `async-trait`, no `dyn`, no +per-call allocation. That makes `Service` (and the `HttpClient` seam built on it, +ADR-0030) **not object-safe**, which is correct here: the network backend is +**in-process**, and [ADR-0007](0007-binding-time-runtime-pluggable-iff-cross-process.md) +binds in-process collaborators at **compile time**, reserving runtime/`dyn` pluggability +for cross-process seams (the Bus). Adapters bind `impl HttpClient` statically. No +boxing, monomorphised stacks. + +## Considered options + +- *One net crate for all transports* — rejected: it would either force HTTP and WS to + share a `Service` that WS cannot honour, or accrete two unrelated core traits in one + crate. The transport split makes the request/reply-vs-streaming fault line a crate + boundary instead of a comment. +- *Keep `Service` in the kernel as "the" primitive* (the Tower bet) — rejected: it + reads as universal when it is request/reply-only, and a kernel that advertises + `Service` invites WS code to treat subscriptions as request/reply, which they are not. +- *Push `Service` into a `net-req-reply-api` crate now* — rejected as premature: correct + the day a second request/reply transport lands, but today it is one extra crate for a + generalisation we do not have. `Service` sits in `net-http-api` until then. +- *Timing layers in the backend on `tokio::time` directly* — rejected: it scatters the + layer logic across crates and couples it to tokio, forfeiting the mock-clock testability + and the WS reuse the `Timer` trait buys for one trait definition. +- *`dyn`-dispatched layers / `BoxFuture`* — rejected: an in-process seam under ADR-0007 + has no need for runtime pluggability, and `async-trait`'s per-call box is exactly the + allocation RPITIT removes. + +## Consequences + +- **The skeleton `oath-adapter-net-api` is repartitioned**: `Service` and the + `http`/`http-body`/`bytes` deps move out to `net-http-api`; the kernel loses all + external deps and gains `Timer`. New crates `oath-adapter-net-http-api`, + `oath-adapter-net-http-hyper`, and (later) `oath-adapter-net-ws-api` join the + workspace; the README dependency graph is updated to match. +- **The kernel is the single home for cross-transport vocabulary** — composition, + classification, and time. Each transport crate adds only its own connection-shape + trait and the layers/types that name its wire format. +- **`oath-adapter-api` is unaffected and independent.** The `Broker` / `DataProvider` + role traits + host harness speak `oath-model` and do **not** depend on the net crates; + the concrete `oath-adapter-ibkr` is the only place the inward role contract and the + outward net plumbing meet (ADR-0003 anti-corruption boundary). +- **WS is a deliberate later session** — its streaming contract, reconnect/heartbeat + layers, and backend are out of scope here; this ADR only guarantees the kernel is + ready for it (`Layer` machinery + `ErrorKind` + `Timer` all apply unchanged). + +## Relationships + +Refines **ADR-0009** (gives `oath-adapter-net-api` its internal structure and adds the +per-transport `*-api` crates). Rests on **ADR-0007** (in-process ⇒ compile-time/`impl` +binding, no `dyn`) and **ADR-0003** (adapter anti-corruption: the net layer is outward +plumbing, role translation stays in the concrete adapter). Is the base for +**ADR-0030** (HTTP transport contract) and **ADR-0031** (HTTP resilience & pacing). +Glossary: no change — `Service`, `Layer`, `HttpClient`, `Timer` are implementation +vocabulary, and [CONTEXT.md](../../CONTEXT.md) is domain-only. diff --git a/docs/adr/0030-http-transport-contract-wire-bytes-streaming-composition.md b/docs/adr/0030-http-transport-contract-wire-bytes-streaming-composition.md new file mode 100644 index 0000000..6c8c808 --- /dev/null +++ b/docs/adr/0030-http-transport-contract-wire-bytes-streaming-composition.md @@ -0,0 +1,186 @@ +# HTTP transport contract: untyped wire bytes, streaming-by-composition, unified `HttpError` + +[ADR-0029](0029-network-adapter-stack-transport-split-compile-time-composition.md) +placed `Service` and the HTTP-specific contracts in `oath-adapter-net-http-api` over +the transport-neutral kernel. This ADR fixes **what that HTTP `Service` carries** — +the request/response types, the body model, the error model, the leaf seam a backend +implements, and the backend itself — driven by the first [Broker](../../CONTEXT.md), +IBKR's Client Portal Web API. The resilience and venue-pacing layers that wrap this +contract are specified in [ADR-0031](0031-http-resilience-venue-pacing.md). + +## Decision + +### 1. The HTTP stack speaks untyped wire bytes; typing stays in the adapter + +```text +Service> → http::Response> +``` + +The whole net-http stack is pure transport: **bytes in, bytes out**. Typed +request-building and JSON (de)serialisation live **above** the net layer, in the +concrete adapter (`oath-adapter-ibkr`), never in `net-http-api`. This is the ADR-0003 +anti-corruption boundary made concrete: IBKR's JSON shapes must not leak into a shared +crate, and `serde` must never become a `net-http-api` dependency. The same stack is +therefore reusable for a [Data Provider](../../CONTEXT.md) whose payloads are not JSON +at all. + +### 2. Request body buffered, response body streaming — deliberately asymmetric + +The **request** body is a buffered `bytes::Bytes`: REST request payloads are tiny, and +`Retry` (ADR-0031) must be able to **replay** a request, which a consumed stream cannot +do. The **response** body is streaming-capable, so the adapter — not the net layer — +decides whether to buffer or stream a given call (SSE, large historical-bar downloads). +"Buffer it" is the caller invoking `.collect()`, not a property baked into the type. + +### 3. `ResponseBody` is a newtype over `Either, B>` + +The response body is assembled from `http-body-util` standard parts, not a hand-written +`Body` state machine, but wrapped so the vendor types do not leak into the canonical +contract: + +```rust +pub struct ResponseBody(Either, fn(Infallible) -> HttpError>, B>); +// Buffered = Full (one frame) Stream = B (live leaf body) +impl> http_body::Body for ResponseBody { … } +// delegating impl via pin-project-lite — no `unsafe` (workspace deny) +``` + +The newtype (over a public `type` alias) earns its keep: `Either` requires both sides to +share one `Body::Error`, so the `Full` side (`Error = Infallible`) must be +`MapErr`'d up to `HttpError` — plumbing that, as an alias, would have leaked +`Either, fn(…)>, B>` into every adapter signature. + +### 4. Buffering is a per-request directive, not a stack type — so one client serves both + +Encoding buffer-vs-stream in the stack type would force an adapter to build *two* +clients (the type differs), duplicating auth, pool, and rate-limit state. Instead the +choice is data on the request — an `http::Request` extension read by a single +`BufferOrStreamLayer`: + +```rust +enum BufferMode { Buffer, Stream } // Copy — survives Retry's request clone +``` + +- `Buffer` → the layer awaits inner and **collects the body to `Bytes` right there**; a + mid-read drop becomes an `Err` the surrounding `Retry` can replay. So the normal JSON + path keeps **full retry coverage including mid-stream failures**, because the body read + is part of the retried attempt. +- `Stream` → the layer returns the live body at headers; mid-stream recovery is the + adapter's job. + +One configured client, per-call choice. `BufferMode` is `Copy` so it survives the +request clone `Retry` makes; a test asserts replay preserves it. + +### 5. One concrete `HttpError` for service *and* body + +`HttpError` is the single error type across the stack — both `Service::Error` and the +`Body::Error` of every body in it (`B: Body`). It +implements `HasErrorKind` once. Backends map their native error (`hyper::Error`) into +`HttpError` at the leaf — the anti-corruption point we require regardless — with a boxed +`#[source]` variant preserving detail for logs without leaking the type. `net-http-api` +relaxes the kernel's "no `thiserror`" stance to derive `HttpError`, but stays free of +`tokio`/`hyper`/`reqwest`/`serde`. Errors are **not** generic: bodies are generic for +zero-alloc flow-through, but a single concrete error lets any layer *construct* one +(`Timeout`, retry-exhausted, body-read failure) without nested `enum`-wrappers or +`BoxError`. + +### 6. `HttpClient` is a blanket-impl'd `Service` sub-trait with `send` + +The named dependency-inversion seam the adapter codes against — but it **is** a +`Service`, so the `Layer` machinery composes it: + +```rust +pub trait HttpClient: + Service, Response = http::Response, Error = HttpError> +{ + type Body: http_body::Body; + fn send(&self, req: http::Request) -> impl Future<…> + Send { self.call(req) } +} +impl HttpClient for S +where S: Service, Response = http::Response, Error = HttpError>, + B: http_body::Body { type Body = B; } +``` + +Backends implement `Service` *once* and are `HttpClient` for free **once the backend +body's error is normalized to `HttpError`** — the leaf wraps `Incoming` (whose native +error is `hyper::Error`) so that `Body::Error = HttpError`, the same anti-corruption +mapping §5 requires for `Service::Error`. Both the normalized leaf (`Body = HyperBody`, +a `MapErr` over `Incoming`) and the fully-layered stack (`Body = ResponseBody`) +then satisfy it. `send` is sugar over `call`. Per ADR-0029 §5 it is a **compile-time +`impl HttpClient` seam**, not `dyn`. + +### 7. Backend: `hyper` + `hyper-util` + `rustls` + +The first leaf backend (`oath-adapter-net-http-hyper`) is hyper-util's pooled client +over rustls, **not reqwest**: + +- It fits the leaf natively — `hyper_util::client::legacy::Client` takes + `http::Request` and returns `http::Response` where `Incoming` is *already* + `http_body::Body`; the leaf is nearly an identity wrap plus the + `hyper::Error → HttpError` mapping on **both** the response result and the body + (`MapErr` — see §6). reqwest hands back a `Stream` needing re-wrapping. +- We build our own middleware (auth, retry, rate-limit), so reqwest's batteries + (redirect/cookie/decompression policy) partly **duplicate** the stack and add implicit + behaviour the anti-corruption ethos wants explicit. +- Smaller, more auditable dependency tree under the workspace `cargo-deny` gate. + +The cost is more wiring (we assemble the pooled HTTPS connector); it is contained behind +the `HttpClient` seam, so swapping to a future `net-http-reqwest` is zero churn to the +stack. + +### 8. Default assembled stack, data config, three-tier override + +`oath-adapter-net-http-hyper` ships the canonical stack behind a data-driven +constructor: + +```rust +pub struct HttpConfig { // plain data — no serde here + pub timeout: TimeoutConfig, pub retry: RetryConfig, pub headers: HeaderMap, /* … */ +} +pub fn build(cfg: HttpConfig, timer: T, auth: impl AuthSource, …) -> impl HttpClient; +``` + +- **Data vs dependencies:** `HttpConfig` is pure data; the `Timer` (generic `T: Timer`, + so a mock clock drives the timing layers in tests while production passes `TokioTimer`), + the `AuthSource`, and the keyed rate-limiter (ADR-0031) are passed as separate + constructor args — behaviour and credentials are not config. `serde` stays in the + adapter, which maps its own deserialised settings into these structs. +- **Three tiers:** *use* the default (`build`); *add* layers by wrapping the returned + `impl HttpClient` (e.g. IBKR's effectful session-keepalive `tickle`, which is **not** a + net-http layer); or *replace/reorder* by assembling `ServiceBuilder` from the public + parts, with the documented order as reference. Batteries included, batteries removable. + +## Considered options + +- *Typed request/response API in net-http* — rejected: pulls `serde` and venue JSON + shapes into the shared crate, breaching ADR-0003; serialisation belongs in the adapter. +- *Buffered `Bytes` response only* — rejected: forecloses SSE / large downloads for no + benefit once buffering is a one-frame `Either` arm. +- *Always-stream, caller `.collect()`s* — rejected: a streaming response returned at + headers escapes the `Retry` boundary, so the **common** JSON path loses retry coverage + on a mid-stream drop. The `BufferMode`-inside-retry design keeps it. +- *Public `type ResponseBody = Either<…>` alias* — rejected: leaks `Either`/`Full`/`MapErr` + into every adapter signature; the newtype hides the assembly behind a stable name. +- *Generic errors bounded by `HasErrorKind`* — rejected: error-producing layers would need + wrapper-`enum`s or `BoxError`, and the adapter inherits a `Stack`-deep error type. One + concrete `HttpError` is simpler and we map the backend error regardless. +- *reqwest backend* — rejected for the *first* backend (fit, control, supply chain above); + remains a viable future `net-http-reqwest` behind the same seam. + +## Consequences + +- `net-http-api` gains deps `http`, `http-body`, `http-body-util`, `bytes`, + `pin-project-lite`, `thiserror` (and `tracing`, ADR-0031), but **not** + `tokio`/`hyper`/`reqwest`/`serde` — it remains a zero-I/O contract crate. +- `net-http-hyper` owns the only `hyper`/`tokio`/`rustls` dependency and the + `hyper::Error → HttpError` and `Incoming → ResponseBody` mappings — the anti-corruption + point where the backend is sealed off. +- The adapter (`oath-adapter-ibkr`) owns `model ↔ JSON ↔ Bytes`, request building via + `http::request::Builder`, the `AuthSource`, and any effectful session management. + +## Relationships + +Builds on **ADR-0029** (`Service` in `net-http-api`, compile-time seam, `Timer`). +Enforces **ADR-0003** (serialisation/typing in the adapter, backend sealed at the leaf). +Is wrapped by **ADR-0031** (the resilience/pacing layers and `build()`'s default order). +Glossary unchanged — implementation vocabulary only. diff --git a/docs/adr/0031-http-resilience-venue-pacing.md b/docs/adr/0031-http-resilience-venue-pacing.md new file mode 100644 index 0000000..4d4d00b --- /dev/null +++ b/docs/adr/0031-http-resilience-venue-pacing.md @@ -0,0 +1,187 @@ +# HTTP resilience and venue pacing: the layer stack, order-safe retry, keyed rate/concurrency limits, circuit breaker + +[ADR-0030](0030-http-transport-contract-wire-bytes-streaming-composition.md) fixed the +HTTP transport contract (bytes in, streaming bytes out, `HttpClient`, hyper backend). +This ADR specifies the **middleware that wraps it** — the default layer stack and its +order, and the construction of each resilience layer — driven by the concrete pacing +rules of the first [Broker](../../CONTEXT.md), IBKR's Client Portal Web API. Every +timing layer is generic over +[`net-api::Timer`](0029-network-adapter-stack-transport-split-compile-time-composition.md); +the tokio impl lives in `net-http-hyper`. + +## The grounding case — IBKR Client Portal pacing + +IBKR enforces a **global 10 req/sec**, **per-endpoint** overrides, and a **429 → 15-minute +IP penalty box** (repeat violators permanently blocked). The per-endpoint column is not +uniform — it is rate *or* **concurrency**: + +| Shape | Examples | Models as | +|---|---|---| +| `1/sec`, `1/5s`, `10/s`, `1/min`, `1/15min` | `/tickle`, `/iserver/account/orders`, `/iserver/marketdata/snapshot`, `/sso/validate`, `/iserver/scanner/params` | `TokenBucket { rate, burst:1.. }` | +| **5 concurrent requests** | `/iserver/marketdata/history` | `Concurrency { max: 5 }` | +| unlisted | everything else | global only | + +This table drives §2–§4: a request counts against the global budget **and** its own +limit, that own limit is rate *xor* concurrency, and we must *never* hit a 429. + +## Decision + +### 1. The default stack and its order + +```text +Tracing → CircuitBreaker → Retry → RateLimit → Timeout → BufferOrStream → Auth → leaf +``` + +(First `.layer()` is outermost — ADR-0029's `ServiceBuilder` invariant.) Rationale for +the order: `Tracing` spans the whole logical request (including retries and pacing +waits); `CircuitBreaker` short-circuits *before* `Retry` runs (§5); `RateLimit` is +*inside* `Retry` so each attempt spends budget; `Timeout` bounds the send, not the +permit wait; `BufferOrStream` is inside `Retry` so retry sees the buffered outcome +(ADR-0030 §4); `Auth` re-stamps current credentials per attempt. A trivial `SetHeaders` +(pure request-header stamp) folds in near `Auth`. + +### 2. Order-safe retry — the wire layer never retransmits an order + +A blind wire retry in front of `POST /order` duplicates the order on a timeout-then-retry +— a funded incident, not a bug. So `Retry` is **retryability-aware**: + +- It decides per request from a `Retryability` request extension (`Copy`, survives + replay), defaulting to **retry idempotent methods only** (`GET`/`HEAD`/`PUT`/`DELETE`); + **never `POST`** unless explicitly marked. +- **Order retransmission is Core's job, not the transport's.** A timed-out order surfaces + as a classified `HttpError`; Core decides whether to reconcile or re-issue under the + *same* `Order Instruction Id`, per + [ADR-0022](0022-reliable-order-path-graduated-failure.md) (graduated failure), + [ADR-0006](0006-broker-reconciliation-contract.md) (reconciliation), and + [ADR-0026](0026-order-identity-three-ids-deterministic.md) (the deterministic + instruction id as the venue dedup key — the FIX `ClOrdID` role). +- It **never retries a `429`** (§5): retrying compounds IBKR's penalty box. + +### 3. `RateLimit` — one keyed layer, rate *and* concurrency as policies + +Two separate rate/concurrency layers cannot model an endpoint whose limit is concurrency +(`/iserver/marketdata/history`) while its neighbour's is rate — the per-endpoint policy +would split across two unsynced maps. So they are **one layer**, and concurrency is just +another policy: + +```rust +enum LimitPolicy { // closed enum — NO dyn (extend by variant) + TokenBucket { rate: f64, burst: u32 }, // every IBKR rate row, by parameters + Concurrency { max: u32 }, // /iserver/marketdata/history = 5 +} +enum Permit { Rate, Concurrency(OwnedSemaphorePermit) } // acquire → guard +``` + +- **Buckets:** a `frozen Arc>` — the key set is known from config + at construction, never mutated, so **lookup is lock-free and there is no map-wide lock**; + each `Bucket` owns its **own `Mutex`** over its policy state, so contention is scoped to + one endpoint. The lock is **released before any `await`** (compute deficit → unlock → + `Timer::sleep` → retry), so a throttled request never blocks other acquirers of its + bucket. +- **Per-request directive** (`http::Request` extension, replacing a classifier closure — + the adapter knows the endpoint when it builds the request): + + ```rust + struct RateLimit { scope: Scope, key: Option } + enum Scope { None, Global, Local, Both } // full 2×2 state space + ``` + + `None` → unlimited (acquire nothing — the *explicit* opt-out); `Global`/`Local`/`Both` + → the obvious bucket sets; **absent directive defaults to `Global`** (you cannot bypass + the global budget by forgetting to stamp). A `Global`/`Local`/`Both` request that + references a bucket **missing from the map is a configuration error, not "no limit"**: + the config is **validated at construction** (every key the adapter stamps must have a + bucket), and any gap that still reaches runtime **fails closed** — the request is + rejected as `Throttled`, never sent unthrottled. A silent fail-open path would bypass + pacing straight into IBKR's 429 penalty box; only the explicit `None` is unlimited. +- **Acquire order:** rate-type buckets **before** concurrency-type, global-first — a + request never holds a scarce concurrency permit while merely *waiting* on a rate token. +- **Permit lifetime:** a rate `Permit` is a ZST (acquire-and-go); a concurrency permit is + **held for the request** and released at `call`-return for a **buffered** response (the + real IBKR `/history` case — work done when the buffered fetch returns) or **attached to + the response body** (released at stream-end/drop) for a **streaming** response. No caller + discipline, no permit-handoff via extension. +- **Wait, with `max_wait`:** an exhausted bucket waits (backpressure, not failure) up to + `max_wait`, then returns `Throttled`. + +### 4. `LimitPolicy` is a closed enum — no `dyn` + +Both `net-http-api` and adapters are first-party, so a new venue's limit shape is a new +enum variant we add when we meet it (YAGNI), dispatched by `match` — no vtable, no alloc. +A `Custom(Arc)` escape hatch was explicitly rejected for reintroducing the dynamic +dispatch the whole stack avoids (ADR-0029 §5). `FixedWindow` is *not* added now — IBKR +needs only `TokenBucket` + `Concurrency`. + +### 5. `CircuitBreaker` — the 429 backstop + +`RateLimit` is the **proactive** guard (never hit 429); `CircuitBreaker` is the +**reactive** one (stop cold if we do). Three states, `Timer`-driven: + +```rust +struct CircuitBreakerConfig { + failure_threshold: u32, // consecutive Connection/Server/Timeout → Open + cooldown: Duration, // general outage, e.g. 30s + throttle_cooldown: Duration, // penalty box ≈ 15 min, on Throttled/429 + half_open_probes: u32, // 1 +} +``` + +- **Closed → Open** on `failure_threshold` consecutive `Connection`/`Server`/`Timeout` + (consecutive-count for v1; rolling-window later), **or immediately on `Throttled`/429** + with the long `throttle_cooldown` (IBKR's 15-min box). +- **Open** rejects fast with a **non-retryable `CircuitOpen`** `HttpError` — and it sits + **outside `Retry`**, so it counts *logical* (post-retry) outcomes and short-circuits + before `Retry`/`RateLimit` run. +- **Half-Open** after cooldown admits `half_open_probes`; success closes, failure re-opens. +- **Single per-host breaker** (v1): IBKR's penalty box is **per-IP, venue-wide**, so one + breaker for the whole gateway matches reality. State shared behind `Arc`. + +### 6. `Tracing` — a Telemetry source, secret-safe + +`TracingLayer` (in `net-http-api`, on the zero-runtime `tracing` facade) is **outermost**: +one span per logical request covering retries and pacing waits, with `Retry` emitting +per-attempt events within it. It records method / route / status / `ErrorKind` / latency / +attempt count and **never** logs auth material or bodies (no `Authorization`/`Cookie`/API +keys/query tokens) — the Auth layer injects secrets, so Tracing is the one place certain +not to leak them. Its output is the +[ADR-0014](0014-observability-three-planes-deterministic-boundary.md) **Telemetry** plane: +the net stack runs in the Adapter process, outside Core's deterministic fold, so this is +machinery metrics (latency/throughput), never canonical state. The layer only instruments; +aggregation is a subscriber's job. It is **always-on but pay-per-use** — an omitted +`TracingLayer` is zero code in a hand-rolled stack; only `build()`'s default includes it. + +## Considered options + +- *Two layers (rate + concurrency)* — rejected: cannot place an endpoint's rate-xor-concurrency + limit in one synced acquire pass; the IBKR table forces the merge. +- *Concurrency permit handed to the caller via response extension* — rejected: makes release + depend on caller discipline and breaks when the response is destructured; body-attachment + releases automatically. +- *`Custom(Arc)`* — rejected: smuggles `dyn` back in; the closed enum is the + first-party extensibility mechanism. +- *Map-wide lock / per-clone buckets* — rejected: a map lock serialises all endpoints; a + per-clone bucket silently multiplies the real rate. Frozen `Arc` + per-bucket + `Mutex` is the only correct shape. +- *`CircuitBreaker` inside `Retry`* — rejected: `Retry` would retry the open-circuit + rejections, defeating it; the breaker must wrap `Retry`. +- *Retry keyed purely on `ErrorKind`* — rejected: it duplicates orders on `POST` retry and + hammers the 429 penalty box; safety must be structural (retryability-aware + no-retry-429). + +## Consequences + +- `RateLimit`, `CircuitBreaker`, `Timeout`, `Retry` are generic over `net-api::Timer`; + `RateLimit`/`CircuitBreaker` use `Timer::now()` (ADR-0029 §4 added it for exactly this). + All are mockable with a fake clock. +- The IBKR adapter supplies the `RateLimit` config map + `RateKey` type from the pacing + table, stamps `RateLimit`/`Retryability`/`BufferMode` extensions when building each + request, owns the effectful `tickle` keepalive as a wrapping layer, and provides the + `CircuitBreaker`/`Retry`/`Timeout` configs. +- `ConcurrencyLimit` as a *separate* layer is dropped — concurrency is a `LimitPolicy` + inside `RateLimit`. + +## Relationships + +Wraps **ADR-0030** (the HTTP contract) and rests on **ADR-0029** (`Timer`, composition, +no `dyn`). Defers order retransmission to **ADR-0022 / 0006 / 0026**. Routes net-layer +observability to the **ADR-0014** Telemetry plane. Glossary unchanged — implementation +vocabulary only; IBKR pacing values are reference data for the adapter, not domain terms.