Make the developer's own wrong account one of the options

[imkp1]

What

On explained_it_back — and only there — stage 3 now has to build one distractor from the mechanism as the developer stated it.

Every other signal leaves stage 3 guessing at what a half-understanding looks like, working from the taxonomy of wrongness in the prompt. Rank 0 already has one in the transcript, stated by the developer and known to be wrong. The design puts the whole burden of multiple choice on the distractors — "the dangerous failure is a fluent answer describing a different mechanism, and the distractors are the only place left to catch it" — and an option someone actually believed is not a guess at that.

Two constraints keep it from backfiring:

• Paraphrased, never quoted. grask does not show a developer their own mistake; it is why the hypothesis is internal. An option reading as your words, wrong is the accusation this design refuses to make.
• Required to be false. Rank 0 covers accounts that were incomplete as well as wrong, and an incomplete account is often true as far as it goes — which is exactly the probe stage 4 discards for having two true options. Those get sharpened into the false claim they would imply if taken as the whole mechanism, rather than restated.

Prompt-only, and deliberately not a fifth structural gate: the four gates are all mechanically decidable, and "is this option the developer's stated mechanism" is a semantic judgement that could only be checked with a model call in the capture path — a judge by another name.

Why not thread shows through

Moment.shows names the wrong part and is the strictest-gated field in triage (triage.py:319 rejects a rank-0 moment without it), but it reaches nothing downstream — it is not in stage 2's prompt, not on Seed, not in the seeds table. Threading it end-to-end was the alternative, and it is deferred: stage 3 already receives the developer's verified quotes and the full dialogue, so the misconception is in front of it either way. If rank 0 fires and the distractor still comes out invented rather than recovered, that is the evidence for threading shows — and not before.

Measuring it

The corpus holds zero explained_it_back seeds:

asked_why            43
pushed_back          17
explained_at_length   2
new_pattern           1
explained_it_back     0

The signal shipped in #22, after every stored seed was captured, so "inspect the next N rank-0 probes" currently has N=0. The prior question is whether rank 0 fires at all — and the old report could not answer it, because a rare signal and an over-strictly gated one looked identical. Those need opposite fixes.

So triage now records every rejected moment, not only the rejections that emptied a session. demoted_from_ask answers "did this session lose all of its moments"; it names no gate and says nothing about a session that kept one moment and threw another away — which is most of them. Those rejections were computed and dropped.

triage_run reports:

• surviving moments by signal, every ranked signal including those at zero — a signal absent from a report reads as not measured rather than never fired;
• a tally of rejections by reason, with the turn N: prefix stripped, or one gate firing on four turns reads as four unrelated one-offs.

Run it over the whole corpus, not --limit N: that flag takes the newest sessions, which measures the deployment window rather than the signal.

Repairs the end-to-end test

test_capture_smoke is the only test that proves the four stages compose rather than each working alone, and it had been failing since stage 4 landed — broken twice over, invisibly, because calibration is deselected by default:

1. The autouse no_real_model_calls guard that arrived with stage 4 refused the calibration run too, so the one test meant to spend money could not.
2. Underneath it, the test asserted probes.criteria was non-empty — a NOT NULL column left from the judge design that add_probe writes [] into on every row (storage.py:847, and test_storage.py:181 asserts exactly that). No probe has satisfied it since multiple choice replaced the judge.

It now asserts the answer key that replaced criteria: option count, no duplicates, an index naming a real option, a non-empty explanation.

Verification

• 478 passed on 3.12 and on the 3.8 floor; ruff and mypy clean; coverage 87.5% (gate 84%).
• Real end-to-end run against a non-grask transcript: ask → seed → probe → verify, valid key, $0.73.
• Twelve new tests, each watched failing first.

Version bumped 0.1.0-rc7 → 0.1.0-rc8 in pyproject.toml and .claude-plugin/plugin.json.

🤖 Generated with Claude Code