Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Network adapter stack: transport-split crates and compile-time `Service`/`Layer` composition

[ADR-0009](0009-crate-topology-spine-inverted-process-aligned.md) placed
`oath-adapter-net-api` in the topology as "HTTP/WS composition primitives," and the
skeleton shipped a Tower-shaped composition core (`Service`, `Layer`,
`ServiceBuilder`, `Stack`, `Identity`) plus a coarse `ErrorKind` / `HasErrorKind`
classifier in one crate. Landing the first [Broker](../../CONTEXT.md) — Interactive
Brokers' Client Portal Web API — forces the question that skeleton deferred: *one net
crate, or many?* This ADR **splits the net layer by transport**, fixes what is
universal enough to live in a shared kernel versus what is transport-specific, and
states the binding-time discipline the whole stack inherits. The HTTP data plane and
the resilience/pacing layers are specified in
[ADR-0030](0030-http-transport-contract-wire-bytes-streaming-composition.md) and
[ADR-0031](0031-http-resilience-venue-pacing.md); this ADR is the structural,
**cross-transport** decision they both rest on.

## Decision

### 1. Split by transport, over a transport-neutral kernel

A venue Adapter speaks more than one wire protocol — IBKR's Client Portal is REST
**and** a streaming WebSocket — and those protocols have *fundamentally different
interaction shapes* (§2). So the net layer is not one crate; it is a kernel plus one
contract crate per transport, plus leaf backends:

```text
oath-adapter-net-api kernel — transport-neutral, std-only:
Layer, ServiceBuilder, Stack, Identity,
ErrorKind, HasErrorKind, Timer
├── oath-adapter-net-http-api HTTP/REST contracts (depends on net-api)
│ └── oath-adapter-net-http-hyper leaf backend (hyper-util + rustls)
└── oath-adapter-net-ws-api WebSocket contracts (future; depends on net-api)
└── oath-adapter-net-ws-<backend> leaf backend (future)
```

This preserves ADR-0009's spine-inverted direction: `net-api` is the most-depended-on
contract, the per-transport `*-api` crates are narrower contracts on top, and concrete
backends implement them. A future gRPC/FIX/multicast transport is a new `*-api` crate
on the same kernel, never a fork.

### 2. `Service` is not universal — it lives in `net-http-api`, not the kernel

`Service<Req>` models **request → one reply**. That fits REST and unary RPC, but it
does **not** model the other transports' core operation:

| Kernel symbol | REST | WebSocket | FIX / TCP session | UDP multicast (recv-only) |
|---|---|---|---|---|
| `Layer` / `ServiceBuilder` / `Stack` / `Identity` | ✓ | ✓ | ✓ | ✓ |
| `ErrorKind` / `HasErrorKind` | ✓ | ✓ | ✓ | ~ |
| `Service<Req>` (request→one reply) | ✓ | ✗ subscription yields *many* frames | ✗ async session | ✗ no request at all |

A WebSocket subscription is "subscribe → stream of frames"; a multicast feed is pure
receive. Forcing those into request/reply is a lie. So **`Service` is a
*connection-shape* contract, not a kernel primitive**, and it lives in `net-http-api`
(the first request/reply transport). WS will define its own streaming contract in
`net-ws-api`. The explicit **no**: there is no shared `Service` in the kernel.

`Service` is transport-*neutral* (it names no HTTP type), so were a second request/reply
transport to appear (gRPC, SSE), `Service` is hoisted into a `net-req-reply-api` crate
shared by both — *not* left in `net-http-api` forcing gRPC to depend on HTTP. We do not
build that crate now (YAGNI); we have one request/reply transport.

### 3. The composition machinery is `Service`-free and stays in the kernel

`Layer<S>` carries **no `Service` bound** — it wraps *anything*. `ServiceBuilder`,
`Stack`, and `Identity` likewise compose an arbitrary `S`. That is exactly why they
belong in the kernel: the same machinery composes an HTTP `Service` stack today and a
WS subscription stack tomorrow. `ErrorKind` / `HasErrorKind` are coarse, wire-neutral
classifications (`Timeout`, `Connection`, `Throttled`, `Auth`, `Client`, `Server`,
`Unknown`) every transport's layers branch on; they stay in the kernel too.

With `Service` removed, the kernel carries **no external dependencies** — std only. A
dependency-free kernel is the signal that the cut is clean.

### 4. `Timer` is a kernel contract, not a runtime

Timing layers (`Timeout`, `Retry` backoff, `RateLimit` refill, `CircuitBreaker`
cooldown — ADR-0031) need a clock, which collides with the `*-api` crates' zero-runtime
charter. Resolution: a minimal **`Timer` trait in the kernel**

```rust
pub trait Timer: Clone + Send + Sync {
fn sleep(&self, dur: Duration) -> impl Future<Output = ()> + Send;
fn now(&self) -> Instant; // token-bucket / cooldown elapsed-time reads
}
```

A trait is not a runtime, so the charter holds and the kernel stays std-only. Timing
*logic* lives with the transport that uses it (`Timeout`/`Retry`/`RateLimit` wrap a
`Service`, so they live in `net-http-api`), generic over `net-api::Timer`; the
**tokio-backed `Timer` impl lives in the leaf backend** (`net-http-hyper`). `Timer` is
in the kernel rather than `net-http-api` because WS reconnect/heartbeat will need the
same clock — it names no transport. Bonus: a mock `Timer` makes every timing layer
deterministically testable without real sleeps.

### 5. Compile-time binding, no `dyn` — RPITIT throughout

`Service::call` returns `impl Future` (RPITIT) — no `async-trait`, no `dyn`, no
per-call allocation. That makes `Service` (and the `HttpClient` seam built on it,
ADR-0030) **not object-safe**, which is correct here: the network backend is
**in-process**, and [ADR-0007](0007-binding-time-runtime-pluggable-iff-cross-process.md)
binds in-process collaborators at **compile time**, reserving runtime/`dyn` pluggability
for cross-process seams (the Bus). Adapters bind `impl HttpClient` statically. No
boxing, monomorphised stacks.

## Considered options

- *One net crate for all transports* — rejected: it would either force HTTP and WS to
share a `Service` that WS cannot honour, or accrete two unrelated core traits in one
crate. The transport split makes the request/reply-vs-streaming fault line a crate
boundary instead of a comment.
- *Keep `Service` in the kernel as "the" primitive* (the Tower bet) — rejected: it
reads as universal when it is request/reply-only, and a kernel that advertises
`Service` invites WS code to treat subscriptions as request/reply, which they are not.
- *Push `Service` into a `net-req-reply-api` crate now* — rejected as premature: correct
the day a second request/reply transport lands, but today it is one extra crate for a
generalisation we do not have. `Service` sits in `net-http-api` until then.
- *Timing layers in the backend on `tokio::time` directly* — rejected: it scatters the
layer logic across crates and couples it to tokio, forfeiting the mock-clock testability
and the WS reuse the `Timer` trait buys for one trait definition.
- *`dyn`-dispatched layers / `BoxFuture`* — rejected: an in-process seam under ADR-0007
has no need for runtime pluggability, and `async-trait`'s per-call box is exactly the
allocation RPITIT removes.

## Consequences

- **The skeleton `oath-adapter-net-api` is repartitioned**: `Service` and the
`http`/`http-body`/`bytes` deps move out to `net-http-api`; the kernel loses all
external deps and gains `Timer`. New crates `oath-adapter-net-http-api`,
`oath-adapter-net-http-hyper`, and (later) `oath-adapter-net-ws-api` join the
workspace; the README dependency graph is updated to match.
- **The kernel is the single home for cross-transport vocabulary** — composition,
classification, and time. Each transport crate adds only its own connection-shape
trait and the layers/types that name its wire format.
- **`oath-adapter-api` is unaffected and independent.** The `Broker` / `DataProvider`
role traits + host harness speak `oath-model` and do **not** depend on the net crates;
the concrete `oath-adapter-ibkr` is the only place the inward role contract and the
outward net plumbing meet (ADR-0003 anti-corruption boundary).
- **WS is a deliberate later session** — its streaming contract, reconnect/heartbeat
layers, and backend are out of scope here; this ADR only guarantees the kernel is
ready for it (`Layer` machinery + `ErrorKind` + `Timer` all apply unchanged).

## Relationships

Refines **ADR-0009** (gives `oath-adapter-net-api` its internal structure and adds the
per-transport `*-api` crates). Rests on **ADR-0007** (in-process ⇒ compile-time/`impl`
binding, no `dyn`) and **ADR-0003** (adapter anti-corruption: the net layer is outward
plumbing, role translation stays in the concrete adapter). Is the base for
**ADR-0030** (HTTP transport contract) and **ADR-0031** (HTTP resilience & pacing).
Glossary: no change — `Service`, `Layer`, `HttpClient`, `Timer` are implementation
vocabulary, and [CONTEXT.md](../../CONTEXT.md) is domain-only.
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# HTTP transport contract: untyped wire bytes, streaming-by-composition, unified `HttpError`

[ADR-0029](0029-network-adapter-stack-transport-split-compile-time-composition.md)
placed `Service` and the HTTP-specific contracts in `oath-adapter-net-http-api` over
the transport-neutral kernel. This ADR fixes **what that HTTP `Service` carries** —
the request/response types, the body model, the error model, the leaf seam a backend
implements, and the backend itself — driven by the first [Broker](../../CONTEXT.md),
IBKR's Client Portal Web API. The resilience and venue-pacing layers that wrap this
contract are specified in [ADR-0031](0031-http-resilience-venue-pacing.md).

## Decision

### 1. The HTTP stack speaks untyped wire bytes; typing stays in the adapter

```text
Service<http::Request<Bytes>> → http::Response<ResponseBody<B>>
```

The whole net-http stack is pure transport: **bytes in, bytes out**. Typed
request-building and JSON (de)serialisation live **above** the net layer, in the
concrete adapter (`oath-adapter-ibkr`), never in `net-http-api`. This is the ADR-0003
anti-corruption boundary made concrete: IBKR's JSON shapes must not leak into a shared
crate, and `serde` must never become a `net-http-api` dependency. The same stack is
therefore reusable for a [Data Provider](../../CONTEXT.md) whose payloads are not JSON
at all.

### 2. Request body buffered, response body streaming — deliberately asymmetric

The **request** body is a buffered `bytes::Bytes`: REST request payloads are tiny, and
`Retry` (ADR-0031) must be able to **replay** a request, which a consumed stream cannot
do. The **response** body is streaming-capable, so the adapter — not the net layer —
decides whether to buffer or stream a given call (SSE, large historical-bar downloads).
"Buffer it" is the caller invoking `.collect()`, not a property baked into the type.

### 3. `ResponseBody<B>` is a newtype over `Either<Full<Bytes>, B>`

The response body is assembled from `http-body-util` standard parts, not a hand-written
`Body` state machine, but wrapped so the vendor types do not leak into the canonical
contract:

```rust
pub struct ResponseBody<B>(Either<MapErr<Full<Bytes>, fn(Infallible) -> HttpError>, B>);
// Buffered = Full<Bytes> (one frame) Stream = B (live leaf body)
impl<B: http_body::Body<Data = Bytes, Error = HttpError>> http_body::Body for ResponseBody<B> { … }
// delegating impl via pin-project-lite — no `unsafe` (workspace deny)
```

The newtype (over a public `type` alias) earns its keep: `Either` requires both sides to
share one `Body::Error`, so the `Full<Bytes>` side (`Error = Infallible`) must be
`MapErr`'d up to `HttpError` — plumbing that, as an alias, would have leaked
`Either<MapErr<Full<Bytes>, fn(…)>, B>` into every adapter signature.

### 4. Buffering is a per-request directive, not a stack type — so one client serves both

Encoding buffer-vs-stream in the stack type would force an adapter to build *two*
clients (the type differs), duplicating auth, pool, and rate-limit state. Instead the
choice is data on the request — an `http::Request` extension read by a single
`BufferOrStreamLayer`:

```rust
enum BufferMode { Buffer, Stream } // Copy — survives Retry's request clone
```

- `Buffer` → the layer awaits inner and **collects the body to `Bytes` right there**; a
mid-read drop becomes an `Err` the surrounding `Retry` can replay. So the normal JSON
path keeps **full retry coverage including mid-stream failures**, because the body read
is part of the retried attempt.
- `Stream` → the layer returns the live body at headers; mid-stream recovery is the
adapter's job.

One configured client, per-call choice. `BufferMode` is `Copy` so it survives the
request clone `Retry` makes; a test asserts replay preserves it.

### 5. One concrete `HttpError` for service *and* body

`HttpError` is the single error type across the stack — both `Service::Error` and the
`Body::Error` of every body in it (`B: Body<Data = Bytes, Error = HttpError>`). It
implements `HasErrorKind` once. Backends map their native error (`hyper::Error`) into
`HttpError` at the leaf — the anti-corruption point we require regardless — with a boxed
`#[source]` variant preserving detail for logs without leaking the type. `net-http-api`
relaxes the kernel's "no `thiserror`" stance to derive `HttpError`, but stays free of
`tokio`/`hyper`/`reqwest`/`serde`. Errors are **not** generic: bodies are generic for
zero-alloc flow-through, but a single concrete error lets any layer *construct* one
(`Timeout`, retry-exhausted, body-read failure) without nested `enum`-wrappers or
`BoxError`.

### 6. `HttpClient` is a blanket-impl'd `Service` sub-trait with `send`

The named dependency-inversion seam the adapter codes against — but it **is** a
`Service`, so the `Layer` machinery composes it:

```rust
pub trait HttpClient:
Service<http::Request<Bytes>, Response = http::Response<Self::Body>, Error = HttpError>
{
type Body: http_body::Body<Data = Bytes, Error = HttpError>;
fn send(&self, req: http::Request<Bytes>) -> impl Future<…> + Send { self.call(req) }
}
impl<S, B> HttpClient for S
where S: Service<http::Request<Bytes>, Response = http::Response<B>, Error = HttpError>,
B: http_body::Body<Data = Bytes, Error = HttpError> { type Body = B; }
```

Backends implement `Service` *once* and are `HttpClient` for free **once the backend
body's error is normalized to `HttpError`** — the leaf wraps `Incoming` (whose native
error is `hyper::Error`) so that `Body::Error = HttpError`, the same anti-corruption
mapping §5 requires for `Service::Error`. Both the normalized leaf (`Body = HyperBody`,
a `MapErr` over `Incoming`) and the fully-layered stack (`Body = ResponseBody<HyperBody>`)
then satisfy it. `send` is sugar over `call`. Per ADR-0029 §5 it is a **compile-time
`impl HttpClient` seam**, not `dyn`.

### 7. Backend: `hyper` + `hyper-util` + `rustls`

The first leaf backend (`oath-adapter-net-http-hyper`) is hyper-util's pooled client
over rustls, **not reqwest**:

- It fits the leaf natively — `hyper_util::client::legacy::Client` takes
`http::Request<B>` and returns `http::Response<Incoming>` where `Incoming` is *already*
`http_body::Body<Data = Bytes>`; the leaf is nearly an identity wrap plus the
`hyper::Error → HttpError` mapping on **both** the response result and the body
(`MapErr<Incoming>` — see §6). reqwest hands back a `Stream` needing re-wrapping.
- We build our own middleware (auth, retry, rate-limit), so reqwest's batteries
(redirect/cookie/decompression policy) partly **duplicate** the stack and add implicit
behaviour the anti-corruption ethos wants explicit.
- Smaller, more auditable dependency tree under the workspace `cargo-deny` gate.

The cost is more wiring (we assemble the pooled HTTPS connector); it is contained behind
the `HttpClient` seam, so swapping to a future `net-http-reqwest` is zero churn to the
stack.

### 8. Default assembled stack, data config, three-tier override

`oath-adapter-net-http-hyper` ships the canonical stack behind a data-driven
constructor:

```rust
pub struct HttpConfig { // plain data — no serde here
pub timeout: TimeoutConfig, pub retry: RetryConfig, pub headers: HeaderMap, /* … */
}
pub fn build<T: Timer>(cfg: HttpConfig, timer: T, auth: impl AuthSource, …) -> impl HttpClient;
```
Comment thread
NotAProfDev marked this conversation as resolved.

- **Data vs dependencies:** `HttpConfig` is pure data; the `Timer` (generic `T: Timer`,
so a mock clock drives the timing layers in tests while production passes `TokioTimer`),
the `AuthSource`, and the keyed rate-limiter (ADR-0031) are passed as separate
constructor args — behaviour and credentials are not config. `serde` stays in the
adapter, which maps its own deserialised settings into these structs.
- **Three tiers:** *use* the default (`build`); *add* layers by wrapping the returned
`impl HttpClient` (e.g. IBKR's effectful session-keepalive `tickle`, which is **not** a
net-http layer); or *replace/reorder* by assembling `ServiceBuilder` from the public
parts, with the documented order as reference. Batteries included, batteries removable.

## Considered options

- *Typed request/response API in net-http* — rejected: pulls `serde` and venue JSON
shapes into the shared crate, breaching ADR-0003; serialisation belongs in the adapter.
- *Buffered `Bytes` response only* — rejected: forecloses SSE / large downloads for no
benefit once buffering is a one-frame `Either` arm.
- *Always-stream, caller `.collect()`s* — rejected: a streaming response returned at
headers escapes the `Retry` boundary, so the **common** JSON path loses retry coverage
on a mid-stream drop. The `BufferMode`-inside-retry design keeps it.
- *Public `type ResponseBody = Either<…>` alias* — rejected: leaks `Either`/`Full`/`MapErr`
into every adapter signature; the newtype hides the assembly behind a stable name.
- *Generic errors bounded by `HasErrorKind`* — rejected: error-producing layers would need
wrapper-`enum`s or `BoxError`, and the adapter inherits a `Stack`-deep error type. One
concrete `HttpError` is simpler and we map the backend error regardless.
- *reqwest backend* — rejected for the *first* backend (fit, control, supply chain above);
remains a viable future `net-http-reqwest` behind the same seam.

## Consequences

- `net-http-api` gains deps `http`, `http-body`, `http-body-util`, `bytes`,
`pin-project-lite`, `thiserror` (and `tracing`, ADR-0031), but **not**
`tokio`/`hyper`/`reqwest`/`serde` — it remains a zero-I/O contract crate.
- `net-http-hyper` owns the only `hyper`/`tokio`/`rustls` dependency and the
`hyper::Error → HttpError` and `Incoming → ResponseBody` mappings — the anti-corruption
point where the backend is sealed off.
- The adapter (`oath-adapter-ibkr`) owns `model ↔ JSON ↔ Bytes`, request building via
`http::request::Builder`, the `AuthSource`, and any effectful session management.

## Relationships

Builds on **ADR-0029** (`Service` in `net-http-api`, compile-time seam, `Timer`).
Enforces **ADR-0003** (serialisation/typing in the adapter, backend sealed at the leaf).
Is wrapped by **ADR-0031** (the resilience/pacing layers and `build()`'s default order).
Glossary unchanged — implementation vocabulary only.
Loading