Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,20 @@ linters:
deny:
- pkg: github.com/MustardSeedNetworks/stem/internal/api
desc: "ratelimit is a leaf — per-IP limiter takes injected dependencies, so it never imports the api transport layer"
# sse is a leaf of internal/api (ADR-0011): it depends only on
# stdlib — never on the api transport layer itself. This depguard
# rule enforces that boundary so a future accidental upward import
# fails CI rather than silently coupling the leaf back into the
# transport layer.
# Test files (broadcaster_test.go) are excluded: the _test.go
# external package imports the leaf itself, which is expected.
"api-sse-isolated":
files:
- "**/internal/api/sse/**"
- "!$test"
deny:
- pkg: github.com/MustardSeedNetworks/stem/internal/api
desc: "sse is a leaf — the broadcaster engine must not import the api transport layer; wire at the api layer"

embeddedstructfieldcheck:
# Checks that sync.Mutex and sync.RWMutex are not used as embedded fields.
Expand Down
120 changes: 53 additions & 67 deletions docs/adr/0011-internal-api-sub-package-decomposition.md
Original file line number Diff line number Diff line change
@@ -1,77 +1,63 @@
# ADR 0011: Decompose `internal/api` into isolated sub-packages
# ADR-0011: internal/api sub-package decomposition

**Status:** Accepted (2026-06-16)
**Status:** Accepted
**Date:** 2026-06-16

## Context

`internal/api` is the Stem HTTP transport layer (handlers, routing, the
capability registry). It has accreted several **self-contained infrastructure
concerns** into the same flat package alongside the transport code:

- the per-client rate limiter (`ratelimit.go`, ~334 LOC);
- the SSE hub + streaming engine (`sse.go` / `handlers_sse.go` /
`sse_publishers.go`, ~345 LOC);
- the TLS listener setup + client-fingerprint logic (`tls.go` /
`tls_fingerprint.go`, ~462 LOC).

Because everything is one Go package, any file can reach any other: there is no
signal or boundary stopping, say, the SSE engine from coupling to an auth
handler. The dependency-direction depguard (ADR-0003) protects the domain core
from importing the API layer, but it cannot express boundaries *within*
`internal/api`. The flat package is also the root cause of the snake_case
filename violations (`handlers_*.go`, `sse_*.go`, `tls_*.go`, …): the prefixes
are a package emulated inside one directory.

This mirrors the situation seed (ADR-0001/0016/0020 strangle) and niac
(**ADR-0006**, Accepted 2026-06-08) already addressed. Stem aligns to the same
fleet convention. Note: stem's per-session CSRF manager already lives in its own
package (`internal/auth/csrf.go`), so it is **not** part of this decomposition.
`internal/api` is a single ~6 000-line package that contains the HTTP server,
all route handlers, middleware, the SSE broadcaster, and the rate limiter in
one flat namespace. Cross-cutting concerns (rate limiting, SSE) have no formal
boundary: any file can import any symbol, and the dependency direction from
leaf concerns back into the HTTP transport is invisible to static analysis.

The first decomposition slice (PR #451) extracted the rate-limiter into
`internal/api/ratelimit`. This ADR records the second slice and the general
decomposition plan.

## Decision

Extract the cohesive infrastructure concerns into sibling **leaf packages** under
`internal/api/<concern>/`, one at a time, lowest-coupling first
(`ratelimit` → `sse` → `tls`). Then strangle the remaining capability handlers
(`mfa`, `reflector`, `settings`, `license`, …) out of the flat layer so that
`internal/api` becomes pure transport + composition, with role-named files
(`handler.go`, `types.go`, …) and **no snake_case filenames**.

Each leaf package:

- owns its type(s), unexported helpers, and tests;
- imports **only** the standard library and inward domain packages — never the
`internal/api` transport layer;
- takes its transport-specific dependencies (client-IP extraction, error
rendering, event sources) as **injected function values**, so the dependency
arrow points inward;
- is composed by the `Server` (and `internal/daemon` where relevant), which
holds the concrete manager and wires it into the declarative route registry
(`register()`/`registerAll()` in `route.go`). The registry, `/__capabilities`,
the middleware composition order, and all security invariants
(CSRF, rate-limit, role/scope gates) are **unchanged** — only *where the
building blocks live* changes.

Each extraction adds a depguard `api-<concern>-isolated` rule (modelled on the
existing inward-only rule and niac's ADR-0006 rules) that denies the sub-package
importing `github.com/MustardSeedNetworks/stem/internal/api`, so the leaf
boundary is **CI-enforced**, not convention.

Once the flat layer's snake_case filenames are eliminated, port seed's
`check-filename-policy.sh` gate into stem CI so the convention cannot regress.
Extract isolated, self-contained concerns from `internal/api` into **leaf
sub-packages** of the form `internal/api/<concern>`. A leaf:

- imports ONLY stdlib and other inward packages (never the api transport layer)
- exports a constructor (`New(...)` or equivalent) returning an exported type
- is depguard-gated: a rule named `api-<concern>-isolated` prevents accidental
upward imports in CI

The api transport layer wires leaves at construction time; no leaf knows about
`internal/api`.

### Completed slices

| Slice | Package | PR |
|-------|---------|-----|
| Rate limiter | `internal/api/ratelimit` | #451 |
| SSE broadcaster | `internal/api/sse` | this ADR |

### SSE slice (this ADR)

`internal/api/sse` holds the `Broadcaster` type (fan-out engine), `Frame`
(wire type + `Encode`), and the subscriber bookkeeping. It has zero HTTP
imports. The HTTP handler (`handleSSEEvents`), the publisher loop
(`runReflectorStatsPublisher`), and the endpoint wiring stay in `internal/api`.

The `sse.HeartbeatInterval` constant is exported so the transport layer can
reference it without duplicating the value; the api-layer file (`sse.go`)
re-derives its own constant from the same numeric literal to avoid an import
dependency in the other direction.

### Future slices (candidates)

| Concern | Notes |
|---------|-------|
| TLS utilities | `ensureSelfSignedCert`, `createTLSConfig`, ACME helpers |
| CORS logic | RFC 1918 origin validation |

## Consequences

- Boundaries within the API layer become explicit and CI-enforced; a future
change cannot silently couple a leaf to the transport layer.
- The snake_case filename violations disappear as a *consequence* of
decomposition (role-named files inside real packages), not via a cosmetic
rename — matching the fleet "guards encode best practice" stance.
- Behaviour is preserved at every step: extractions are verbatim moves with
inward-injected dependencies; the route registry and security middleware are
untouched. Each slice is independently gated (build, vet, full golangci,
tests, route-policy + output-escaping gates) before merge.
- Slices are **serial**, not parallel: they share `server.go` wiring and
`.golangci.yml`, so concurrent extraction PRs collide. Land one, rebase the
next.
- Stem reaches the same enforced-decomposition state as seed and niac, closing
the fleet-parity gap for `internal/api`.
- The leaf boundary is statically enforced by depguard (`api-sse-isolated`,
`api-ratelimit-isolated` rules in `.golangci.yml`).
- `go vet` + `golangci-lint` catch upward imports at CI time.
- `internal/api` package size decreases incrementally with each slice.
- No behaviour change: endpoints, event types, and publish sites are identical.
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ survives the people and the diffs. Format mirrors the sibling repos (seed/niac).
| [0006](0006-at-rest-encryption-device-keyed.md) | At-rest encryption device-keyed; DEK/JWT separation N/A | Accepted |
| [0007](0007-ed25519-signed-licenses.md) | Ed25519-signed license tokens | Accepted |
| [0008](0008-dataplane-parser-memory-safety.md) | Memory-safety gate for the C dataplane packet parser | Accepted |
| [0011](0011-internal-api-sub-package-decomposition.md) | internal/api sub-package decomposition (ratelimit + sse leaves) | Accepted |
| [0009](0009-csrf-defense-in-depth-edges.md) | CSRF defense-in-depth at the edges | Accepted |
| [0010](0010-json-wire-casing-convention.md) | JSON wire-casing convention (camelCase API) | Accepted |
| [0011](0011-internal-api-sub-package-decomposition.md) | Decompose `internal/api` into isolated sub-packages | Accepted |
Expand Down
8 changes: 5 additions & 3 deletions internal/api/background_internal_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import (
"context"
"testing"
"time"

"github.com/MustardSeedNetworks/stem/internal/api/sse"
)

// TestBackgroundComponentsStartStop verifies the holder starts its goroutine
Expand All @@ -15,7 +17,7 @@ import (
func TestBackgroundComponentsStartStop(t *testing.T) {
t.Parallel()

s := &Server{sseBroadcaster: NewSSEBroadcaster()}
s := &Server{sseBroadcaster: sse.New()}
bg := newBackgroundComponents(s)

bg.Start(context.Background())
Expand Down Expand Up @@ -46,7 +48,7 @@ func TestBackgroundComponentsStopBeforeStart(t *testing.T) {
func TestBackgroundComponentsStopIdempotent(t *testing.T) {
t.Parallel()

s := &Server{sseBroadcaster: NewSSEBroadcaster()}
s := &Server{sseBroadcaster: sse.New()}
bg := newBackgroundComponents(s)
bg.Start(context.Background())
bg.Stop()
Expand All @@ -59,7 +61,7 @@ func TestBackgroundComponentsStopIdempotent(t *testing.T) {
func TestBackgroundComponentsCtxCancelStops(t *testing.T) {
t.Parallel()

s := &Server{sseBroadcaster: NewSSEBroadcaster()}
s := &Server{sseBroadcaster: sse.New()}
bg := newBackgroundComponents(s)

ctx, cancel := context.WithCancel(context.Background())
Expand Down
5 changes: 3 additions & 2 deletions internal/api/handlers_settings.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package api
import (
"net/http"

"github.com/MustardSeedNetworks/stem/internal/api/sse"
"github.com/MustardSeedNetworks/stem/internal/logging"
"github.com/MustardSeedNetworks/stem/internal/netif"
reflectorDP "github.com/MustardSeedNetworks/stem/internal/reflector/dataplane"
Expand Down Expand Up @@ -161,7 +162,7 @@ func (s *Server) handleModeUpdate(w http.ResponseWriter, r *http.Request) {
}
// Broadcast even on "unchanged" so a browser tab that missed
// the original change still gets a confirmation frame.
s.sseBroadcaster.Publish(SSEFrame{Type: "mode_changed", Payload: resp})
s.sseBroadcaster.Publish(sse.Frame{Type: "mode_changed", Payload: resp})
writeJSON(w, resp)
return
}
Expand Down Expand Up @@ -190,7 +191,7 @@ func (s *Server) handleModeUpdate(w http.ResponseWriter, r *http.Request) {
}
// Push to every connected SSE subscriber so other browser tabs /
// CLI watchers see the mode change in real time (#296).
s.sseBroadcaster.Publish(SSEFrame{Type: "mode_changed", Payload: resp})
s.sseBroadcaster.Publish(sse.Frame{Type: "mode_changed", Payload: resp})
writeJSON(w, resp)
}

Expand Down
4 changes: 3 additions & 1 deletion internal/api/handlers_sse.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
// SPDX-License-Identifier: BUSL-1.1

package api

import (
Expand All @@ -10,7 +12,7 @@ import (

// handleSSEEvents serves the long-lived SSE stream at /api/v1/events.
//
// Wire protocol: each frame is one JSON object preceded by `data: ` and
// Wire protocol: each frame is one JSON object preceded by "data: " and
// followed by a blank line. Periodic SSE-comment heartbeats keep
// intermediaries (load balancers, reverse proxies) from idling the
// connection — they discard comments so the UI never sees them.
Expand Down
5 changes: 3 additions & 2 deletions internal/api/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ import (
"time"

"github.com/MustardSeedNetworks/stem/internal/api/ratelimit"
"github.com/MustardSeedNetworks/stem/internal/api/sse"
"github.com/MustardSeedNetworks/stem/internal/auth"
"github.com/MustardSeedNetworks/stem/internal/license"
"github.com/MustardSeedNetworks/stem/internal/logging"
Expand Down Expand Up @@ -139,7 +140,7 @@ var staticFiles embed.FS
type Server struct {
port int
mux *http.ServeMux
sseBroadcaster *SSEBroadcaster // Fan-out for /api/v1/events subscribers; nil-safe via getSSE()
sseBroadcaster *sse.Broadcaster // Fan-out for /api/v1/events subscribers; nil when SSE not available.
httpServer *http.Server
stats *Stats
statsMu sync.RWMutex
Expand Down Expand Up @@ -261,7 +262,7 @@ func NewServer(port int) (*Server, error) {
s := &Server{}
s.port = port
s.mux = http.NewServeMux()
s.sseBroadcaster = NewSSEBroadcaster()
s.sseBroadcaster = sse.New()
s.statsMu = sync.RWMutex{}
s.stats = &Stats{
PacketsReceived: 0,
Expand Down
Loading