Skip to content

fix(observe): serve the off-path pool's counters at /stats - #50

Merged
OsherElhadad merged 1 commit into
mainfrom
fix/wire-pool-stats
Aug 10, 2026
Merged

fix(observe): serve the off-path pool's counters at /stats#50
OsherElhadad merged 1 commit into
mainfrom
fix/wire-pool-stats

Conversation

@OsherElhadad

Copy link
Copy Markdown
Collaborator

Found by the docs audit in #49, which diffed every documented /stats field against the actual Snapshot struct.

Problem

modes.Pool tracked queued/pending/processed/dropped/errors correctly — and nothing ever read them. metrics.Snapshot had no field for them and proxy.Handler.stats never called Stats().

Meanwhile docs/how-to/operating-modes.md (added by #43) instructs operators to watch dropped and errors, and makes a point of contrasting that with a dashboard which reports only queue depth. The doc described a counter no consumer could reach.

Both halves were individually correct, which is why neither the code review nor the docs review caught it — the gap only exists between them.

Why dropped in particular matters

A drop is an observation silently given up. So a rising dropped means the potential_*/projected_* figures understate what compaction would have saved. Serving queue depth while hiding drops is exactly the failure the doc calls out in the competitor's dashboard.

Fix

Exposed as observe_queue, filled by the host at serve time (the pool lives in modes, which sits above metrics, so metrics cannot read it). omitempty, so a sync-only deployment shows no phantom queue.

metrics declares its own QueueStats rather than importing modes — the dependency runs the other way and importing it would invert the layering.

Purely additive: no existing /stats key renamed or removed, so deploy/harbor/*.py keeps parsing.

Test

TestObserveQueueCountersReachStats asserts the counters arrive over HTTP — it tests the wiring, not the pool, which already had coverage.

Verified non-vacuous. With the wiring removed:

modes_test.go:571: observe_queue absent from /stats: the pool's counters are still
unreachable, so the documented `dropped` cannot be read by any consumer

Gates

go build -tags cg_skeleton ./... · go test -tags cg_skeleton ./... · go test -race ./proxy/... ./metrics/... · gofmt -l · go vet — all clean.

modes.Pool tracked queued/pending/processed/dropped/errors correctly and nothing
ever read them: metrics.Snapshot had no field and the /stats handler never called
Stats(). So docs/how-to/operating-modes.md instructed operators to watch dropped
and errors -- and made a point of contrasting that with a dashboard which reports
only queue depth -- while describing a counter no consumer could reach.

dropped is the one that changes a reader's conclusion. A drop is an observation
silently given up, so a rising dropped means the potential_*/projected_* figures
UNDERSTATE what compaction would have saved. Reporting depth while hiding drops
is precisely the gap the doc calls out elsewhere.

Exposed as observe_queue, omitted when no pool is running so a sync-only
deployment shows no phantom queue. metrics declares its own QueueStats rather
than importing modes, because the dependency runs the other way. Purely additive:
no existing /stats key renamed or removed, so deploy/harbor/*.py keeps parsing.

Found by a docs audit that diffed every documented field against the Snapshot
struct -- the kind of gap neither a code review nor a docs review finds alone,
since each half was individually correct.

The test asserts the counters arrive over HTTP, not that the pool counts; it
fails without the wiring with the field absent.

Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants