Skip to content

fix(gate/cache): assign every gate:matrix flag, and wire the cache write path - #169

Merged
myselfsiddharth merged 2 commits into
mainfrom
claude/pr-review-htvs6t
Aug 16, 2026
Merged

fix(gate/cache): assign every gate:matrix flag, and wire the cache write path#169
myselfsiddharth merged 2 commits into
mainfrom
claude/pr-review-htvs6t

Conversation

@myselfsiddharth

Copy link
Copy Markdown
Contributor

Summary

Closes #165. Closes #166.

Two wiring bugs in the same area: gate:matrix accepted flags it dropped, and the cache had a read path with no write path. Both had the same shape — a surface that looks connected and is not — and both produced numbers that look measured.

Track

  • Track 1 — churn harness / gate measurement
  • Tooling / CI / housekeeping

Changes

#165 — four flags accepted, consumed, and dropped

parseArgs held a valued set; assignValue held an else if chain. They disagreed about --from-cache, --site-key, --task-key and --repair-model: the parser consumed the value, handed it over, and the chain fell off its end. No error, and no unknown argument either — the flags were in valued, which is exactly what made them invisible.

Closed structurally, per the issue's second bullet, rather than by adding four branches. One VALUED_FLAGS table maps each flag to its assigner; the parser derives its accepted set from those keys; assignValue throws on an unrecognised key instead of returning. A flag cannot be accepted-but-unassigned any more, because there is only one list.

--repair-model was dropped twice, and the issue names only the first. Assigning it is not sufficient — nothing in run-matrix.ts ever read args.repairModel, so ReplayRunner still fell back to StubRepairModelClient. Fixing only assignValue would have closed the issue's letter and left its entire stated harm in place: a run asking for the real model would still report a self-heal rate of 0 and a cost_repair of zero, both looking like measurements. The flag now constructs an AnthropicRepairModelClient and passes it through to runVersionLive, before anything boots — so a missing ANTHROPIC_API_KEY is a named refusal rather than a stub run, which is the whole reason that client throws at construction. --repair-model under --dry-run is refused outright: no step fails, so the repair loop never runs and no token is ever spent.

Affected artifacts: any past run invoked with --repair-model reported stub numbers. As #165 says, check the run's own rows rather than the command that was typed — a stub run has cost_repair all-zero on every row and no repair_context_level.

#166 — nothing populated the cache

writeCacheRow() had no caller outside tests/. The read half landed with #118, so gate:matrix --from-cache <dir> resolved programs out of a directory no shipped code path could fill, and cacheHitRate() could only ever report no_data for a reason that had nothing to do with the experiment.

Ownership decision — the compiler CLI writes (paragent compile --to-cache <dir>, via the new src/cache/ingest.ts):

  • The compiler is the only stage holding steps_total. resolveProgram refuses to return anything it cannot prove complete (ADR-0013), and the ProgramRef carrying that proof is written by the compiler. Whoever writes rows must already hold the whole bundle.
  • A measurement harness should not also be a data producer. Had gate:matrix populated on first run, the populating run would be a file replay and every run after it a cache hit — inside one reported sample. That is the "comparing two different things" hazard Measure the fresh-reasoning baseline cost (the §9 denominator) #39 warns about.
  • A separate paragent cache write would need a third reader of the bundle shape.

All-or-nothing. The authority runs over the whole bundle before anything is persisted, so a rejection on step 7 cannot leave steps 0–6 on disk. A prefix would not be a correctness hazard — resolveProgram fails closed and reports incomplete — but a cache that half-holds a program is a worse thing to debug than one that does not hold it.

src/cache/ stays library-only and takes a structurally-typed bundle, so the dependency runs one way and the compiler still does not import the cache's types.

Finding: the pre-check and the authority disagree, in the safe direction

Requested by #166 item 3. Now that the authority actually runs on real data, the two can be compared. On the committed 12-step live bundle the compiler marks 1 row poolable and writeCacheRow marks 7:

rows=12 pool_eligible=1
cached 12 steps to .cache/paragent (7 pool-eligible, 5 tenant-only)
  note: authority pooled 6 step(s) the compiler pre-check did not: 2, 3, 4, 5, 8, 9

One rule accounts for the whole gap. Where a row's entire locator chain is tainted but the row carries flow_topology, buildPoolRow degrades it to a topology_only pool row — carrying no locator at all, only "a click happened here, in main, between a click and a fill" — while decidePoolEligibility refuses it outright as topology_only_degraded.

Legal (a pre-check may be stricter, never looser; the dangerous direction stays pinned by live-bundle-pool.test.ts) and nothing tenant-derived escapes either way. Deliberately not reconciled here: changing decidePoolEligibility changes what every committed bundle artifact claims about pool eligibility, which is a privacy-adjacent decision that wants its own ADR, not a rider on a wiring fix. It is reported on every --to-cache run and recorded in docs/gate/compiler.md and docs/architecture.md rather than smoothed over.

Docs

  • docs/architecture.md: break 1 marked closed, the diagram edge drawn solid, the red linkStyle removed, the src/cache/ and src/compiler/ rows updated. Scoped to break 1 — the full re-derivation is docs/architecture.md is 62 commits stale and now states things that are false #168's job and is deliberately not attempted here.
  • docs/gate/compiler.md: --to-cache in the CLI section, and the measured pre-check/authority divergence.
  • docs/gate/runner.md: corrects a claim that was false — it said the real repair client was "opt-in via gate:matrix --repair-model", which did not work until this PR.

Test plan

New tests:

  • tests/unit/gate-matrix.test.ts — table-driven over the flag list, plus a coverage check so a new flag cannot slip past untested, plus the unknown-key throw.
  • tests/integration/cache-ingest-bundle.test.ts — round trip to a complete resolveProgram HIT read by a separate store on the same directory; rows in both pool.jsonl and tenant.jsonl; the authority overriding the pre-check; whole-bundle rejection leaving an empty directory; the real committed 12-step bundle.

Both guards were checked by reverting rather than only by passing:

  • reintroducing the accepted-but-unassigned flag shape fails 2 tests;
  • without the two-pass write, the rejection test finds a prefix on disk.

End-to-end, the loop the issue asks about:

$ npm run compile -- --in <trajectory>.json --to-cache .cache/paragent
cached 12 steps to .cache/paragent (7 pool-eligible, 5 tenant-only)

$ npm run gate:matrix -- --dry-run --from-cache .cache/paragent \
    --site-key grafana-oss@9.5.21 --task-key create-stat-dashboard-from-testdata ...
gate:matrix: cache HIT prog-traj-gate-live-create-stat-dashboard-from-testdata-9.5.21 (12 steps)

Rows carry program_source: cache. That line was unreachable from any shipped path before this PR.

Hard rules

  • No secrets.
  • No invented metrics. The only numbers here are counts read out of a run (1 vs 7 poolable rows, 12 steps) and test counts. No gate number is claimed; --dry-run rows stay all-zero and --repair-model is now refused under --dry-run rather than appearing to price a repair.
  • Claims are sourced. Every claim about what was wired was verified by running it, not by reading a sibling doc.
  • Docs ship with the code, carrying the standard YAML frontmatter. No document was added, so docs/README.md needs no new row; npm run lint:docs is clean (65 docs).

Contracts

  • This PR does not change contracts/.

Note for the reviewer

The --repair-model client construction is wired but unexercised against a real key — no Docker daemon or ANTHROPIC_API_KEY in this environment, so the live path stops at the pre-flight. What is proven is that the flag now reaches a client constructor at all; the self-heal rate stays structurally 0 until someone runs it with a key, which is the exit criterion #27 names.

🤖 Generated with Claude Code


Generated by Claude Code

…ite path

Closes #165. Closes #166.

## #165 — gate:matrix silently dropped four flags

`parseArgs` held a `valued` set and `assignValue` held an `else if` chain, and
the two disagreed: `--from-cache`, `--site-key`, `--task-key` and
`--repair-model` were consumed by the parser, handed over, and fell off the end
of the chain. No error — they were in `valued`, so they never tripped
`unknown argument` either.

Closed structurally rather than by adding four branches: one `VALUED_FLAGS`
table maps each flag to its assigner, the parser derives its accepted set from
those keys, and `assignValue` throws on a key it does not recognise instead of
returning. A flag cannot now be accepted-but-unassigned. The test walks the
table rather than naming the four that were broken, and a second test asserts
the sample map covers every flag so a new one cannot slip past uncovered.

**`--repair-model` was dropped twice, and the issue names only the first.**
Assigning it is not sufficient: nothing in `run-matrix.ts` ever read
`args.repairModel`, so `ReplayRunner` still defaulted to
`StubRepairModelClient` and the run still reported a self-heal rate of 0 and a
`cost_repair` of zero that both look measured. The flag now builds an
`AnthropicRepairModelClient` and passes it to `runVersionLive`. Constructed
before anything boots, so a missing `ANTHROPIC_API_KEY` is a named refusal
rather than a stub run — the whole reason that client throws at construction.
`--repair-model` under `--dry-run` is refused outright: no step fails, so the
repair loop never runs and no token is ever spent.

Any pre-existing artifact produced with `--repair-model` reported stub numbers.

## #166 — nothing populated the cache

`writeCacheRow()` had no caller outside `tests/`. The read half landed with
#118, so `gate:matrix --from-cache <dir>` resolved programs out of a directory
no shipped code path could fill, and `cacheHitRate()` could only report
`no_data` for a reason unrelated to the experiment.

`paragent compile --to-cache <dir>` closes it, via `src/cache/ingest.ts`. The
compiler CLI owns the write because the compiler is the only stage holding
`steps_total` — the `ProgramRef` `resolveProgram` needs to prove completeness —
and because a measurement harness that populated on first run would make its
populating run differ from every run after it, inside one reported sample.

All-or-nothing: the authority runs over the whole bundle before anything is
persisted, so a rejected step cannot leave a resolvable-looking prefix on disk.
`src/cache/` stays library-only and takes a structurally-typed bundle, so the
dependency runs one way and the compiler still does not import the cache's types.

Verified end to end: compile --to-cache -> `gate:matrix --from-cache` reports
`cache HIT prog-traj-gate-live-... (12 steps)` and rows carry
`program_source: cache`.

### Finding: the pre-check and the authority disagree, in the safe direction

Now that the authority actually runs on real data, the two can be compared. On
the committed 12-step live bundle the compiler marks 1 row poolable and
`writeCacheRow` marks 7. One rule accounts for the gap: where a row's whole
locator chain is tainted but the row carries `flow_topology`, `buildPoolRow`
degrades it to a `topology_only` pool row carrying no locator at all, while
`decidePoolEligibility` refuses it as `topology_only_degraded`.

Legal (a pre-check may be stricter, never looser) and nothing tenant-derived
escapes either way. Deliberately not reconciled here: changing it changes what
every committed bundle artifact claims about pool eligibility, which wants its
own ADR rather than a rider on a wiring fix. Reported by `--to-cache` on every
run and recorded in docs/gate/compiler.md and docs/architecture.md.

## Tests

- Table-driven flag coverage in `tests/unit/gate-matrix.test.ts`.
- `tests/integration/cache-ingest-bundle.test.ts`: round trip to a complete
  resolveProgram HIT, rows on disk in both files, the authority overriding the
  pre-check, whole-bundle rejection leaving an empty directory, and the real
  committed 12-step bundle.
- Canary suite unmodified and green, as #166 requires.

Both guards were checked by reverting: reintroducing the accepted-but-unassigned
flag shape fails 2 tests; dropping the two-pass write leaves a prefix on disk.

npm run ci green (534 unit, 33 integration); npm run test:canary green (52).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016CAt2F7SR83iuqvnM4GzAf
@myselfsiddharth
myselfsiddharth requested a review from a team as a code owner August 16, 2026 07:02
@github-actions github-actions Bot added the size/XL > 600 changed lines — consider splitting label Aug 16, 2026
@github-actions
github-actions Bot requested a review from OM152002 August 16, 2026 07:02
@github-actions github-actions Bot added documentation Improvements or additions to documentation gate PRD section 9 gate measurement area: compiler Touches compiler area: cache Touches cache area: experiments Touches experiments labels Aug 16, 2026
@myselfsiddharth myselfsiddharth changed the title fix(gate,cache): assign every gate:matrix flag, and wire the cache write path fix(gate/cache): assign every gate:matrix flag, and wire the cache write path Aug 16, 2026
The divergence between decidePoolEligibility and writeCacheRow was recorded as
an open finding with no tracking issue to reach. Filed as #170, which lays out
the three ways it could go; both docs now link it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016CAt2F7SR83iuqvnM4GzAf
@myselfsiddharth
myselfsiddharth merged commit e4ab318 into main Aug 16, 2026
12 checks passed
@myselfsiddharth
myselfsiddharth deleted the claude/pr-review-htvs6t branch August 16, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cache Touches cache area: compiler Touches compiler area: experiments Touches experiments documentation Improvements or additions to documentation gate PRD section 9 gate measurement size/XL > 600 changed lines — consider splitting

Projects

None yet

2 participants