Skip to content

fix(cache): the volatile-tail split is a silent no-op on Bedrock Converse — it reports success and cannot deliver #45

Description

@OsherElhadad

Found by an integration review of the five cache/filter/observe merges (#33, #36, #40, #42, #43). Related to but distinct from #44, which fixes the duplication of cachePoint across the split.

Problem

explicitBreakpointProvider admits Bedrock, BedrockMantle and Vertex, so splitVolatileTail runs on those providers. But on the real Bedrock Converse shape, cachePoint is its own array entry placed after the block it terminates — not a field on that block:

"system": [ {"text": "…stable…\nRecent commits:\n"}, {"cachePoint": {"type": "default"}} ]

The split inserts the volatile half before that trailing entry, producing:

system: [stable, volatile, {cachePoint}]

So the breakpoint still covers the churn, which is precisely what the split exists to prevent.

Why this is worse than an ordinary no-op

It reports success. splitVolatileTail returns Changed: true, cachesplit shows as active in /stats, and nothing distinguishes "split and it worked" from "split and it achieved nothing." The −34.1% cost win the split is credited with cannot materialize on these providers, yet every signal says the mechanism ran.

This is the same class of defect #36 was written to kill — cacheinject applying 46 breakpoints and forwarding 0 — reappearing one provider over. It survived because the Anthropic path is the only one the benchmarks exercise, and on Anthropic the field-on-block shape makes the split correct.

Per the project's F-1 rule (an aggregate moving the predicted way is not evidence the predicted mechanism operated): a component that reports acting is not evidence it acted usefully either.

Desired behavior

Detect the Converse array-entry shape and split correctly: the stable half keeps the trailing cachePoint entry immediately after it, and the volatile half goes after that entry.

Alternatively, if handling the shape properly is not worth it, stop claiming to act: return Changed: false for it so /stats tells the truth and the credited saving is not attributed on providers where it cannot occur. Silently doing nothing while reporting success is the one option that must not stand.

Relevant code

  • apply/prefixsplit.gosplitVolatileTail, explicitBreakpointProvider, and the block-copy that assumes a field-on-block breakpoint.
  • apply/metawrite.go:106-119breakpointPaths, which already knows cachePoint can be a system.# entry in its own right.

Testing plan

  • Construct a real Converse-shaped system array (block followed by a standalone {"cachePoint":…} entry) with a git-tail block, and assert the breakpoint no longer covers the volatile half after the split.
  • Assert Changed is false when the split cannot achieve separation, so /stats cannot report a phantom action.
  • Keep the Anthropic field-on-block case passing unchanged.

Acceptance criteria

  • On Converse shape, either the split separates the churn correctly, or it declines and reports Changed: false.
  • A test that fails on today's code, showing the breakpoint still covering the tail.
  • The docs stop attributing the −34.1% figure to providers where the mechanism cannot fire, or state the provider scope explicitly.
  • go test -race green.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status
    New/ToDo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions