-
Notifications
You must be signed in to change notification settings - Fork 0
docs(adr): network adapter stack design (ADR-0029–0031) #56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+524
−0
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
151 changes: 151 additions & 0 deletions
151
docs/adr/0029-network-adapter-stack-transport-split-compile-time-composition.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| # Network adapter stack: transport-split crates and compile-time `Service`/`Layer` composition | ||
|
|
||
| [ADR-0009](0009-crate-topology-spine-inverted-process-aligned.md) placed | ||
| `oath-adapter-net-api` in the topology as "HTTP/WS composition primitives," and the | ||
| skeleton shipped a Tower-shaped composition core (`Service`, `Layer`, | ||
| `ServiceBuilder`, `Stack`, `Identity`) plus a coarse `ErrorKind` / `HasErrorKind` | ||
| classifier in one crate. Landing the first [Broker](../../CONTEXT.md) — Interactive | ||
| Brokers' Client Portal Web API — forces the question that skeleton deferred: *one net | ||
| crate, or many?* This ADR **splits the net layer by transport**, fixes what is | ||
| universal enough to live in a shared kernel versus what is transport-specific, and | ||
| states the binding-time discipline the whole stack inherits. The HTTP data plane and | ||
| the resilience/pacing layers are specified in | ||
| [ADR-0030](0030-http-transport-contract-wire-bytes-streaming-composition.md) and | ||
| [ADR-0031](0031-http-resilience-venue-pacing.md); this ADR is the structural, | ||
| **cross-transport** decision they both rest on. | ||
|
|
||
| ## Decision | ||
|
|
||
| ### 1. Split by transport, over a transport-neutral kernel | ||
|
|
||
| A venue Adapter speaks more than one wire protocol — IBKR's Client Portal is REST | ||
| **and** a streaming WebSocket — and those protocols have *fundamentally different | ||
| interaction shapes* (§2). So the net layer is not one crate; it is a kernel plus one | ||
| contract crate per transport, plus leaf backends: | ||
|
|
||
| ```text | ||
| oath-adapter-net-api kernel — transport-neutral, std-only: | ||
| Layer, ServiceBuilder, Stack, Identity, | ||
| ErrorKind, HasErrorKind, Timer | ||
| ├── oath-adapter-net-http-api HTTP/REST contracts (depends on net-api) | ||
| │ └── oath-adapter-net-http-hyper leaf backend (hyper-util + rustls) | ||
| └── oath-adapter-net-ws-api WebSocket contracts (future; depends on net-api) | ||
| └── oath-adapter-net-ws-<backend> leaf backend (future) | ||
| ``` | ||
|
|
||
| This preserves ADR-0009's spine-inverted direction: `net-api` is the most-depended-on | ||
| contract, the per-transport `*-api` crates are narrower contracts on top, and concrete | ||
| backends implement them. A future gRPC/FIX/multicast transport is a new `*-api` crate | ||
| on the same kernel, never a fork. | ||
|
|
||
| ### 2. `Service` is not universal — it lives in `net-http-api`, not the kernel | ||
|
|
||
| `Service<Req>` models **request → one reply**. That fits REST and unary RPC, but it | ||
| does **not** model the other transports' core operation: | ||
|
|
||
| | Kernel symbol | REST | WebSocket | FIX / TCP session | UDP multicast (recv-only) | | ||
| |---|---|---|---|---| | ||
| | `Layer` / `ServiceBuilder` / `Stack` / `Identity` | ✓ | ✓ | ✓ | ✓ | | ||
| | `ErrorKind` / `HasErrorKind` | ✓ | ✓ | ✓ | ~ | | ||
| | `Service<Req>` (request→one reply) | ✓ | ✗ subscription yields *many* frames | ✗ async session | ✗ no request at all | | ||
|
|
||
| A WebSocket subscription is "subscribe → stream of frames"; a multicast feed is pure | ||
| receive. Forcing those into request/reply is a lie. So **`Service` is a | ||
| *connection-shape* contract, not a kernel primitive**, and it lives in `net-http-api` | ||
| (the first request/reply transport). WS will define its own streaming contract in | ||
| `net-ws-api`. The explicit **no**: there is no shared `Service` in the kernel. | ||
|
|
||
| `Service` is transport-*neutral* (it names no HTTP type), so were a second request/reply | ||
| transport to appear (gRPC, SSE), `Service` is hoisted into a `net-req-reply-api` crate | ||
| shared by both — *not* left in `net-http-api` forcing gRPC to depend on HTTP. We do not | ||
| build that crate now (YAGNI); we have one request/reply transport. | ||
|
|
||
| ### 3. The composition machinery is `Service`-free and stays in the kernel | ||
|
|
||
| `Layer<S>` carries **no `Service` bound** — it wraps *anything*. `ServiceBuilder`, | ||
| `Stack`, and `Identity` likewise compose an arbitrary `S`. That is exactly why they | ||
| belong in the kernel: the same machinery composes an HTTP `Service` stack today and a | ||
| WS subscription stack tomorrow. `ErrorKind` / `HasErrorKind` are coarse, wire-neutral | ||
| classifications (`Timeout`, `Connection`, `Throttled`, `Auth`, `Client`, `Server`, | ||
| `Unknown`) every transport's layers branch on; they stay in the kernel too. | ||
|
|
||
| With `Service` removed, the kernel carries **no external dependencies** — std only. A | ||
| dependency-free kernel is the signal that the cut is clean. | ||
|
|
||
| ### 4. `Timer` is a kernel contract, not a runtime | ||
|
|
||
| Timing layers (`Timeout`, `Retry` backoff, `RateLimit` refill, `CircuitBreaker` | ||
| cooldown — ADR-0031) need a clock, which collides with the `*-api` crates' zero-runtime | ||
| charter. Resolution: a minimal **`Timer` trait in the kernel** | ||
|
|
||
| ```rust | ||
| pub trait Timer: Clone + Send + Sync { | ||
| fn sleep(&self, dur: Duration) -> impl Future<Output = ()> + Send; | ||
| fn now(&self) -> Instant; // token-bucket / cooldown elapsed-time reads | ||
| } | ||
| ``` | ||
|
|
||
| A trait is not a runtime, so the charter holds and the kernel stays std-only. Timing | ||
| *logic* lives with the transport that uses it (`Timeout`/`Retry`/`RateLimit` wrap a | ||
| `Service`, so they live in `net-http-api`), generic over `net-api::Timer`; the | ||
| **tokio-backed `Timer` impl lives in the leaf backend** (`net-http-hyper`). `Timer` is | ||
| in the kernel rather than `net-http-api` because WS reconnect/heartbeat will need the | ||
| same clock — it names no transport. Bonus: a mock `Timer` makes every timing layer | ||
| deterministically testable without real sleeps. | ||
|
|
||
| ### 5. Compile-time binding, no `dyn` — RPITIT throughout | ||
|
|
||
| `Service::call` returns `impl Future` (RPITIT) — no `async-trait`, no `dyn`, no | ||
| per-call allocation. That makes `Service` (and the `HttpClient` seam built on it, | ||
| ADR-0030) **not object-safe**, which is correct here: the network backend is | ||
| **in-process**, and [ADR-0007](0007-binding-time-runtime-pluggable-iff-cross-process.md) | ||
| binds in-process collaborators at **compile time**, reserving runtime/`dyn` pluggability | ||
| for cross-process seams (the Bus). Adapters bind `impl HttpClient` statically. No | ||
| boxing, monomorphised stacks. | ||
|
|
||
| ## Considered options | ||
|
|
||
| - *One net crate for all transports* — rejected: it would either force HTTP and WS to | ||
| share a `Service` that WS cannot honour, or accrete two unrelated core traits in one | ||
| crate. The transport split makes the request/reply-vs-streaming fault line a crate | ||
| boundary instead of a comment. | ||
| - *Keep `Service` in the kernel as "the" primitive* (the Tower bet) — rejected: it | ||
| reads as universal when it is request/reply-only, and a kernel that advertises | ||
| `Service` invites WS code to treat subscriptions as request/reply, which they are not. | ||
| - *Push `Service` into a `net-req-reply-api` crate now* — rejected as premature: correct | ||
| the day a second request/reply transport lands, but today it is one extra crate for a | ||
| generalisation we do not have. `Service` sits in `net-http-api` until then. | ||
| - *Timing layers in the backend on `tokio::time` directly* — rejected: it scatters the | ||
| layer logic across crates and couples it to tokio, forfeiting the mock-clock testability | ||
| and the WS reuse the `Timer` trait buys for one trait definition. | ||
| - *`dyn`-dispatched layers / `BoxFuture`* — rejected: an in-process seam under ADR-0007 | ||
| has no need for runtime pluggability, and `async-trait`'s per-call box is exactly the | ||
| allocation RPITIT removes. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - **The skeleton `oath-adapter-net-api` is repartitioned**: `Service` and the | ||
| `http`/`http-body`/`bytes` deps move out to `net-http-api`; the kernel loses all | ||
| external deps and gains `Timer`. New crates `oath-adapter-net-http-api`, | ||
| `oath-adapter-net-http-hyper`, and (later) `oath-adapter-net-ws-api` join the | ||
| workspace; the README dependency graph is updated to match. | ||
| - **The kernel is the single home for cross-transport vocabulary** — composition, | ||
| classification, and time. Each transport crate adds only its own connection-shape | ||
| trait and the layers/types that name its wire format. | ||
| - **`oath-adapter-api` is unaffected and independent.** The `Broker` / `DataProvider` | ||
| role traits + host harness speak `oath-model` and do **not** depend on the net crates; | ||
| the concrete `oath-adapter-ibkr` is the only place the inward role contract and the | ||
| outward net plumbing meet (ADR-0003 anti-corruption boundary). | ||
| - **WS is a deliberate later session** — its streaming contract, reconnect/heartbeat | ||
| layers, and backend are out of scope here; this ADR only guarantees the kernel is | ||
| ready for it (`Layer` machinery + `ErrorKind` + `Timer` all apply unchanged). | ||
|
|
||
| ## Relationships | ||
|
|
||
| Refines **ADR-0009** (gives `oath-adapter-net-api` its internal structure and adds the | ||
| per-transport `*-api` crates). Rests on **ADR-0007** (in-process ⇒ compile-time/`impl` | ||
| binding, no `dyn`) and **ADR-0003** (adapter anti-corruption: the net layer is outward | ||
| plumbing, role translation stays in the concrete adapter). Is the base for | ||
| **ADR-0030** (HTTP transport contract) and **ADR-0031** (HTTP resilience & pacing). | ||
| Glossary: no change — `Service`, `Layer`, `HttpClient`, `Timer` are implementation | ||
| vocabulary, and [CONTEXT.md](../../CONTEXT.md) is domain-only. |
186 changes: 186 additions & 0 deletions
186
docs/adr/0030-http-transport-contract-wire-bytes-streaming-composition.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,186 @@ | ||
| # HTTP transport contract: untyped wire bytes, streaming-by-composition, unified `HttpError` | ||
|
|
||
| [ADR-0029](0029-network-adapter-stack-transport-split-compile-time-composition.md) | ||
| placed `Service` and the HTTP-specific contracts in `oath-adapter-net-http-api` over | ||
| the transport-neutral kernel. This ADR fixes **what that HTTP `Service` carries** — | ||
| the request/response types, the body model, the error model, the leaf seam a backend | ||
| implements, and the backend itself — driven by the first [Broker](../../CONTEXT.md), | ||
| IBKR's Client Portal Web API. The resilience and venue-pacing layers that wrap this | ||
| contract are specified in [ADR-0031](0031-http-resilience-venue-pacing.md). | ||
|
|
||
| ## Decision | ||
|
|
||
| ### 1. The HTTP stack speaks untyped wire bytes; typing stays in the adapter | ||
|
|
||
| ```text | ||
| Service<http::Request<Bytes>> → http::Response<ResponseBody<B>> | ||
| ``` | ||
|
|
||
| The whole net-http stack is pure transport: **bytes in, bytes out**. Typed | ||
| request-building and JSON (de)serialisation live **above** the net layer, in the | ||
| concrete adapter (`oath-adapter-ibkr`), never in `net-http-api`. This is the ADR-0003 | ||
| anti-corruption boundary made concrete: IBKR's JSON shapes must not leak into a shared | ||
| crate, and `serde` must never become a `net-http-api` dependency. The same stack is | ||
| therefore reusable for a [Data Provider](../../CONTEXT.md) whose payloads are not JSON | ||
| at all. | ||
|
|
||
| ### 2. Request body buffered, response body streaming — deliberately asymmetric | ||
|
|
||
| The **request** body is a buffered `bytes::Bytes`: REST request payloads are tiny, and | ||
| `Retry` (ADR-0031) must be able to **replay** a request, which a consumed stream cannot | ||
| do. The **response** body is streaming-capable, so the adapter — not the net layer — | ||
| decides whether to buffer or stream a given call (SSE, large historical-bar downloads). | ||
| "Buffer it" is the caller invoking `.collect()`, not a property baked into the type. | ||
|
|
||
| ### 3. `ResponseBody<B>` is a newtype over `Either<Full<Bytes>, B>` | ||
|
|
||
| The response body is assembled from `http-body-util` standard parts, not a hand-written | ||
| `Body` state machine, but wrapped so the vendor types do not leak into the canonical | ||
| contract: | ||
|
|
||
| ```rust | ||
| pub struct ResponseBody<B>(Either<MapErr<Full<Bytes>, fn(Infallible) -> HttpError>, B>); | ||
| // Buffered = Full<Bytes> (one frame) Stream = B (live leaf body) | ||
| impl<B: http_body::Body<Data = Bytes, Error = HttpError>> http_body::Body for ResponseBody<B> { … } | ||
| // delegating impl via pin-project-lite — no `unsafe` (workspace deny) | ||
| ``` | ||
|
|
||
| The newtype (over a public `type` alias) earns its keep: `Either` requires both sides to | ||
| share one `Body::Error`, so the `Full<Bytes>` side (`Error = Infallible`) must be | ||
| `MapErr`'d up to `HttpError` — plumbing that, as an alias, would have leaked | ||
| `Either<MapErr<Full<Bytes>, fn(…)>, B>` into every adapter signature. | ||
|
|
||
| ### 4. Buffering is a per-request directive, not a stack type — so one client serves both | ||
|
|
||
| Encoding buffer-vs-stream in the stack type would force an adapter to build *two* | ||
| clients (the type differs), duplicating auth, pool, and rate-limit state. Instead the | ||
| choice is data on the request — an `http::Request` extension read by a single | ||
| `BufferOrStreamLayer`: | ||
|
|
||
| ```rust | ||
| enum BufferMode { Buffer, Stream } // Copy — survives Retry's request clone | ||
| ``` | ||
|
|
||
| - `Buffer` → the layer awaits inner and **collects the body to `Bytes` right there**; a | ||
| mid-read drop becomes an `Err` the surrounding `Retry` can replay. So the normal JSON | ||
| path keeps **full retry coverage including mid-stream failures**, because the body read | ||
| is part of the retried attempt. | ||
| - `Stream` → the layer returns the live body at headers; mid-stream recovery is the | ||
| adapter's job. | ||
|
|
||
| One configured client, per-call choice. `BufferMode` is `Copy` so it survives the | ||
| request clone `Retry` makes; a test asserts replay preserves it. | ||
|
|
||
| ### 5. One concrete `HttpError` for service *and* body | ||
|
|
||
| `HttpError` is the single error type across the stack — both `Service::Error` and the | ||
| `Body::Error` of every body in it (`B: Body<Data = Bytes, Error = HttpError>`). It | ||
| implements `HasErrorKind` once. Backends map their native error (`hyper::Error`) into | ||
| `HttpError` at the leaf — the anti-corruption point we require regardless — with a boxed | ||
| `#[source]` variant preserving detail for logs without leaking the type. `net-http-api` | ||
| relaxes the kernel's "no `thiserror`" stance to derive `HttpError`, but stays free of | ||
| `tokio`/`hyper`/`reqwest`/`serde`. Errors are **not** generic: bodies are generic for | ||
| zero-alloc flow-through, but a single concrete error lets any layer *construct* one | ||
| (`Timeout`, retry-exhausted, body-read failure) without nested `enum`-wrappers or | ||
| `BoxError`. | ||
|
|
||
| ### 6. `HttpClient` is a blanket-impl'd `Service` sub-trait with `send` | ||
|
|
||
| The named dependency-inversion seam the adapter codes against — but it **is** a | ||
| `Service`, so the `Layer` machinery composes it: | ||
|
|
||
| ```rust | ||
| pub trait HttpClient: | ||
| Service<http::Request<Bytes>, Response = http::Response<Self::Body>, Error = HttpError> | ||
| { | ||
| type Body: http_body::Body<Data = Bytes, Error = HttpError>; | ||
| fn send(&self, req: http::Request<Bytes>) -> impl Future<…> + Send { self.call(req) } | ||
| } | ||
| impl<S, B> HttpClient for S | ||
| where S: Service<http::Request<Bytes>, Response = http::Response<B>, Error = HttpError>, | ||
| B: http_body::Body<Data = Bytes, Error = HttpError> { type Body = B; } | ||
| ``` | ||
|
|
||
| Backends implement `Service` *once* and are `HttpClient` for free **once the backend | ||
| body's error is normalized to `HttpError`** — the leaf wraps `Incoming` (whose native | ||
| error is `hyper::Error`) so that `Body::Error = HttpError`, the same anti-corruption | ||
| mapping §5 requires for `Service::Error`. Both the normalized leaf (`Body = HyperBody`, | ||
| a `MapErr` over `Incoming`) and the fully-layered stack (`Body = ResponseBody<HyperBody>`) | ||
| then satisfy it. `send` is sugar over `call`. Per ADR-0029 §5 it is a **compile-time | ||
| `impl HttpClient` seam**, not `dyn`. | ||
|
|
||
| ### 7. Backend: `hyper` + `hyper-util` + `rustls` | ||
|
|
||
| The first leaf backend (`oath-adapter-net-http-hyper`) is hyper-util's pooled client | ||
| over rustls, **not reqwest**: | ||
|
|
||
| - It fits the leaf natively — `hyper_util::client::legacy::Client` takes | ||
| `http::Request<B>` and returns `http::Response<Incoming>` where `Incoming` is *already* | ||
| `http_body::Body<Data = Bytes>`; the leaf is nearly an identity wrap plus the | ||
| `hyper::Error → HttpError` mapping on **both** the response result and the body | ||
| (`MapErr<Incoming>` — see §6). reqwest hands back a `Stream` needing re-wrapping. | ||
| - We build our own middleware (auth, retry, rate-limit), so reqwest's batteries | ||
| (redirect/cookie/decompression policy) partly **duplicate** the stack and add implicit | ||
| behaviour the anti-corruption ethos wants explicit. | ||
| - Smaller, more auditable dependency tree under the workspace `cargo-deny` gate. | ||
|
|
||
| The cost is more wiring (we assemble the pooled HTTPS connector); it is contained behind | ||
| the `HttpClient` seam, so swapping to a future `net-http-reqwest` is zero churn to the | ||
| stack. | ||
|
|
||
| ### 8. Default assembled stack, data config, three-tier override | ||
|
|
||
| `oath-adapter-net-http-hyper` ships the canonical stack behind a data-driven | ||
| constructor: | ||
|
|
||
| ```rust | ||
| pub struct HttpConfig { // plain data — no serde here | ||
| pub timeout: TimeoutConfig, pub retry: RetryConfig, pub headers: HeaderMap, /* … */ | ||
| } | ||
| pub fn build<T: Timer>(cfg: HttpConfig, timer: T, auth: impl AuthSource, …) -> impl HttpClient; | ||
| ``` | ||
|
|
||
| - **Data vs dependencies:** `HttpConfig` is pure data; the `Timer` (generic `T: Timer`, | ||
| so a mock clock drives the timing layers in tests while production passes `TokioTimer`), | ||
| the `AuthSource`, and the keyed rate-limiter (ADR-0031) are passed as separate | ||
| constructor args — behaviour and credentials are not config. `serde` stays in the | ||
| adapter, which maps its own deserialised settings into these structs. | ||
| - **Three tiers:** *use* the default (`build`); *add* layers by wrapping the returned | ||
| `impl HttpClient` (e.g. IBKR's effectful session-keepalive `tickle`, which is **not** a | ||
| net-http layer); or *replace/reorder* by assembling `ServiceBuilder` from the public | ||
| parts, with the documented order as reference. Batteries included, batteries removable. | ||
|
|
||
| ## Considered options | ||
|
|
||
| - *Typed request/response API in net-http* — rejected: pulls `serde` and venue JSON | ||
| shapes into the shared crate, breaching ADR-0003; serialisation belongs in the adapter. | ||
| - *Buffered `Bytes` response only* — rejected: forecloses SSE / large downloads for no | ||
| benefit once buffering is a one-frame `Either` arm. | ||
| - *Always-stream, caller `.collect()`s* — rejected: a streaming response returned at | ||
| headers escapes the `Retry` boundary, so the **common** JSON path loses retry coverage | ||
| on a mid-stream drop. The `BufferMode`-inside-retry design keeps it. | ||
| - *Public `type ResponseBody = Either<…>` alias* — rejected: leaks `Either`/`Full`/`MapErr` | ||
| into every adapter signature; the newtype hides the assembly behind a stable name. | ||
| - *Generic errors bounded by `HasErrorKind`* — rejected: error-producing layers would need | ||
| wrapper-`enum`s or `BoxError`, and the adapter inherits a `Stack`-deep error type. One | ||
| concrete `HttpError` is simpler and we map the backend error regardless. | ||
| - *reqwest backend* — rejected for the *first* backend (fit, control, supply chain above); | ||
| remains a viable future `net-http-reqwest` behind the same seam. | ||
|
|
||
| ## Consequences | ||
|
|
||
| - `net-http-api` gains deps `http`, `http-body`, `http-body-util`, `bytes`, | ||
| `pin-project-lite`, `thiserror` (and `tracing`, ADR-0031), but **not** | ||
| `tokio`/`hyper`/`reqwest`/`serde` — it remains a zero-I/O contract crate. | ||
| - `net-http-hyper` owns the only `hyper`/`tokio`/`rustls` dependency and the | ||
| `hyper::Error → HttpError` and `Incoming → ResponseBody` mappings — the anti-corruption | ||
| point where the backend is sealed off. | ||
| - The adapter (`oath-adapter-ibkr`) owns `model ↔ JSON ↔ Bytes`, request building via | ||
| `http::request::Builder`, the `AuthSource`, and any effectful session management. | ||
|
|
||
| ## Relationships | ||
|
|
||
| Builds on **ADR-0029** (`Service` in `net-http-api`, compile-time seam, `Timer`). | ||
| Enforces **ADR-0003** (serialisation/typing in the adapter, backend sealed at the leaf). | ||
| Is wrapped by **ADR-0031** (the resilience/pacing layers and `build()`'s default order). | ||
| Glossary unchanged — implementation vocabulary only. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.