Skip to content

fix(cache): delete both breakpoint spellings from the split's volatile half - #44

Merged
OsherElhadad merged 1 commit into
mainfrom
fix/split-cachepoint
Aug 10, 2026
Merged

fix(cache): delete both breakpoint spellings from the split's volatile half#44
OsherElhadad merged 1 commit into
mainfrom
fix/split-cachepoint

Conversation

@OsherElhadad

Copy link
Copy Markdown
Collaborator

Closes nothing on its own — this is a blocker found by integration review of the five cache/filter/observe merges (#33, #36, #40, #42, #43). Neither per-PR review could see it.

The defect

splitVolatileTail copies the system block it splits, then deletes cache_control from the churning half so the breakpoint covers only the stable text (apply/prefixsplit.go:118).

It did not delete cachePoint — the Bedrock/Vertex spelling of the same thing, which wireBreakpoints counts alongside cache_control (apply/metawrite.go:117).

So on those providers, a splittable system block carrying an inline cachePoint gets its breakpoint duplicated — one on each half, 1 → 2. That silently spends a slot from the provider's cap of four, and from a request already at the cap it puts 5 on the wire → 400.

Why both reviews missed it

The two halves landed in the same PR (#36) but in different files: the split predates it, the cachePoint counting was added by it. And the Anthropic path — the only one the benchmarks exercise — has no cachePoint to duplicate, so no run could have surfaced it.

This is the same shape as the bug #36 was written to fix (46 breakpoints applied, 0 forwarded): a provider-specific wire detail invisible to the code path under test.

Fix

One line, plus a comment explaining why both spellings must go.

delete(volatile, "cache_control")
delete(volatile, "cachePoint")

Test

TestSplitDoesNotDuplicateBedrockCachePoint asserts the wire breakpoint count is unchanged across the split, and that the volatile half keeps no breakpoint in either spelling.

Verified non-vacuous — with the fix reverted it fails with the exact reported symptom:

--- FAIL: TestSplitDoesNotDuplicateBedrockCachePoint
    prefixsplit_test.go:353: split changed the wire breakpoint count 1 -> 2;
    a copied cachePoint burns a slot from the provider's cap of four

Gates

go build -tags cg_skeleton ./... · go test -tags cg_skeleton ./... · go test -race ./apply/... · gofmt -l · go vet — all clean.

Related findings from the same review, NOT fixed here

Tracked separately so this stays a one-line blocker fix:

  1. The volatile-tail split is a silent no-op on Bedrock Converse. explicitBreakpointProvider admits Bedrock/Vertex, but on real Converse shape cachePoint is its own array entry after the block it terminates — so the split inserts the volatile half before it and the breakpoint still covers the churn. The split reports Changed: true and cachesplit looks active in /stats, yet the −34.1% win it is credited with cannot materialize there. Same class of defect, one provider over.
  2. Observe mode leaks hypotheticals into unlabelled /stats fields. cmdfilter's FilterStats sink has no Mode check, so an observe-only run reports real cmdfilter_families/cmdfilter_filters/cmdfilter_selector_misses with no mode label and no potential_* counterpart — a consumer cannot tell enforced from hypothetical. New leakage from feat(cmdfilter): port rtk's filter coverage (3 -> 24) and fix three DSL loss-typing bugs #42 × feat(proxy): add observe mode — measure what compaction would save, without enforcing it #43 composing.
  3. Pin-budget exhaustion can reopen the TailOnly fail-open on the /compact route only (the chat path uses modes.Tracker and never consults cg:len:). Measurement-accuracy risk for offline eval, not live traffic.
  4. The selector-miss ledger bounds count, not key size — replaying multimodal traffic filled the top slots with base64 PNG blobs, making the "which filter to write next" ledger unusable on that traffic.

…e half

splitVolatileTail COPIES the system block it splits, then deletes cache_control
from the churning half so the breakpoint covers only the stable text. It did not
delete cachePoint -- the Bedrock/Vertex spelling of the same thing, which
wireBreakpoints counts alongside cache_control.

So on those providers a splittable system block carrying an inline cachePoint had
its breakpoint DUPLICATED: one on each half, 1 -> 2. That silently spends a slot
from the provider's cap of four, and from a request already at the cap it puts
five on the wire and takes a 400.

Found by an integration review of the five cache/filter/observe merges: neither
per-PR review could see it, because the split and the wire-breakpoint counter
landed in the same PR but in different files, and the Anthropic path -- the only
one the benchmarks exercise -- has no cachePoint to duplicate.

The regression test asserts the wire count is unchanged across the split and that
the volatile half keeps no breakpoint in either spelling. It fails without the
fix with the exact symptom (1 -> 2).

Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants