fix(observe): gate the filter ledger behind Ctx.Stats() so observe records nothing enforced - #54
Merged
Merged
Conversation
…cords nothing enforced cmdfilter called c.FilterStats with no mode check, so an observe-only run -- zero enforced requests, sync_enforced 0 -- still reported real-looking cmdfilter_families, cmdfilter_filters and cmdfilter_selector_misses entries. Those tokens were never saved: nothing was forwarded. Unlike the two fields observe deliberately shares (cg_added_ms_avg, a true measurement of the enforced path whose reading of ~0 IS the headline result; and context-guru's own model spend, which is real money and labelled by observe_llm_notice), these three sit in the enforced namespace with no mode label and no potential_* counterpart. A consumer cannot tell them from real savings. That is the failure #31 named as its primary correctness risk -- a mislabelled hypothetical is worse than no number, because it silently inflates the product's own headline claim. The gate is a Ctx.Stats() accessor rather than a check at cmdfilter's two call sites. A component author reaching for c.FilterStats has no reason to think about operating modes, so the next sink added to Ctx would reproduce this exactly; an accessor makes the safe path the only convenient one. New leakage from #42 and #43 composing -- neither PR's own review could see it, because the stats sink and observe mode landed independently. The test asserts sync DOES record, so it proves the gate rather than a dead sink, and fails without the gate with the reported symptom (acts=1 in observe mode). Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
This was referenced Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #46.
The leak
cmdfiltercalledc.FilterStatswith no mode check, so an observe-only run — zero enforced requests,sync_enforced: 0— still reported real-looking entries:Those tokens were never saved — nothing was forwarded.
Why these three fields, and not the two observe deliberately shares
#43correctly keeps two enforced-namespace fields populated in observe mode:cg_added_ms_avg— a true measurement of the enforced path, and its reading of ~0 is observe's headline result. Zeroing it would hide a real number.llm_calls/llm_*_tokens— real money actually spent measuring off-path. Relabelling real spend aspotential_*would be a worse lie. Labelled viaobserve_llm_notice.These three are different: no mode label, no
potential_*counterpart. A consumer cannot tell them from real savings. That is exactly the failure #31 named as its primary correctness risk — a mislabelled hypothetical is worse than no number, because it silently inflates the product's own headline claim.The fix is an accessor, not a call-site check
Gating at
Ctxrather than atcmdfilter's two call sites is deliberate: a component author reaching forc.FilterStatshas no reason to think about operating modes, so the next sink added toCtxwould reproduce this bug exactly. An accessor makes the safe path the only convenient one. The field's doc comment now says to read it throughStats().Provenance
This is new leakage from #42 × #43 composing — neither PR's own review could see it, because the stats sink and observe mode landed independently and each was correct in isolation. Found by the integration review of the five merges, which is the class of defect per-PR review structurally cannot catch.
Test
TestObserveModeDoesNotRecordFilterStatsdrives the component in both modes and asserts sync does record — otherwise a broken sink would pass it for the wrong reason.Verified non-vacuous. With the gate removed it reproduces the reported symptom:
Gates
go build -tags cg_skeleton ./...·go test -tags cg_skeleton ./...·go test -race ./components/... ./proxy/...·gofmt -l·go vet— all clean.Note on the remaining integration findings
#45 (the volatile-tail split is a silent no-op on Bedrock Converse), #47 (pin-budget exhaustion reopening the
TailOnlyfail-open on/compactonly) and #48 (the selector-miss ledger bounding key count but not key size) are still open and unaddressed by this PR.