Context
experiments/gate-v1/run-matrix.ts declares ten value-taking flags in parseArgs's valued set. assignValue() has branches for only six of them:
versions program param runs port cost-fresh → assigned
from-cache site-key task-key repair-model → silently dropped
The parser consumes the flag's value (const value = argv[++i]), hands it to assignValue, and assignValue falls off the end of its else if chain without matching. No error, no warning. The flag looks accepted — it is in the valued set, so it does not trip the unknown argument throw, and it is documented in usage().
Found while wiring --cost-fresh in #164 (which added the sixth branch, and is why the bug is visible by contrast rather than by symptom).
Why this is worse than a normal dropped flag
--repair-model is the serious one. It is the switch that opts a gate run into the real Anthropic repair client instead of StubRepairModelClient. Dropped silently, a run invoked as:
npm run gate:matrix -- --versions <...> --repair-model claude-opus-5
runs the stub. The stub returns corrected_action: null and zero tokens, so that run reports a self-heal rate of 0 and a cost_repair of zero — and both look like measurements. §9's kill line is mean repair cost ≥ 70% of fresh, so a zero numerator does not merely score badly; it reads as a spectacularly passing repair cost against whatever denominator #39 eventually measures.
This is precisely the failure mode AnthropicRepairModelClient was built to prevent. From its own module docstring:
This client is constructed explicitly, and throws at construction when ANTHROPIC_API_KEY is unset rather than degrading to a no-op — a gate run that silently used the stub would produce a wrong number that looks real, which is worse than a run that failed.
The client refuses to degrade silently. The CLI in front of it degrades silently anyway, and never reaches the constructor that would have objected.
The other three matter less but are not harmless:
What to do
- Add the four missing branches to
assignValue().
- Make this class of bug impossible rather than fixing four instances of it. The
valued set and assignValue's branches are two lists that must agree and currently do not. Derive one from the other, or make assignValue throw on an unrecognized key instead of falling through — a flag the parser accepted but nobody consumed is a caller error, and the parser already throws for unknown argument, so silence here is inconsistent with its own posture.
- Unit-test that every flag in
valued actually lands somewhere in Args. tests/unit/gate-matrix.test.ts already exercises assignValue directly, so this is a table-driven test over the flag list, not new scaffolding.
Checking whether a past run was affected
Any completed matrix run invoked with --repair-model reported stub numbers. Before quoting a self-heal rate or a repair cost from an existing artifact, confirm from the run's own rows rather than from the command that was typed: a stub run has cost_repair all-zero on every row, and repair_context_level absent.
How to test
npm run test # table-driven: every flag in `valued` is consumed by assignValue
npm run ci
Before you open the PR
Context
experiments/gate-v1/run-matrix.tsdeclares ten value-taking flags inparseArgs'svaluedset.assignValue()has branches for only six of them:The parser consumes the flag's value (
const value = argv[++i]), hands it toassignValue, andassignValuefalls off the end of itselse ifchain without matching. No error, no warning. The flag looks accepted — it is in thevaluedset, so it does not trip theunknown argumentthrow, and it is documented inusage().Found while wiring
--cost-freshin #164 (which added the sixth branch, and is why the bug is visible by contrast rather than by symptom).Why this is worse than a normal dropped flag
--repair-modelis the serious one. It is the switch that opts a gate run into the real Anthropic repair client instead ofStubRepairModelClient. Dropped silently, a run invoked as:runs the stub. The stub returns
corrected_action: nulland zero tokens, so that run reports a self-heal rate of 0 and acost_repairof zero — and both look like measurements. §9's kill line ismean repair cost ≥ 70% of fresh, so a zero numerator does not merely score badly; it reads as a spectacularly passing repair cost against whatever denominator #39 eventually measures.This is precisely the failure mode
AnthropicRepairModelClientwas built to prevent. From its own module docstring:The client refuses to degrade silently. The CLI in front of it degrades silently anyway, and never reaches the constructor that would have objected.
The other three matter less but are not harmless:
--from-cache— the cache read path (Cache read path: resolve a program from the cache and record hit/miss — the amortization claim has no mechanism without it #118/ADR-0014) never engages, soprogram_sourcestaysfileand cache hit-rate (feat(metrics): report cache hit-rate, the fifth §9 secondary metric #154) has no denominator.--site-key/--task-key— identity fields fall back to defaults, so rows can be attributed to the wrong task.task_keymismatch is exactly what Measure the fresh-reasoning baseline cost (the §9 denominator) #39 warns makes the fresh/repair ratio compare two different things.What to do
assignValue().valuedset andassignValue's branches are two lists that must agree and currently do not. Derive one from the other, or makeassignValuethrow on an unrecognized key instead of falling through — a flag the parser accepted but nobody consumed is a caller error, and the parser already throws forunknown argument, so silence here is inconsistent with its own posture.valuedactually lands somewhere inArgs.tests/unit/gate-matrix.test.tsalready exercisesassignValuedirectly, so this is a table-driven test over the flag list, not new scaffolding.Checking whether a past run was affected
Any completed matrix run invoked with
--repair-modelreported stub numbers. Before quoting a self-heal rate or a repair cost from an existing artifact, confirm from the run's own rows rather than from the command that was typed: a stub run hascost_repairall-zero on every row, andrepair_context_levelabsent.How to test
Before you open the PR