Name the misconception the evidence supports, and let stage 2 say there isn't one

[imkp1]

An external review's central objection, which is correct: stage 2 was asked to state "what this developer may have accepted without fully understanding," and nothing in the pipeline observes understanding. Six changes from that review, plus one of my own that did not survive its own measurement and was reverted in the same branch.

Everything here was run against the real model before being claimed to work. Two changes came back with null results and are documented as unproven rather than quietly kept as wins.

The framing

grask sees turns, agent replies, and diffs. It never sees a mind, and asking for a hidden mental state gets one — fluently, on any session at all. Stage 2 now names the most plausible mechanism misconception the evidence supports, and the prompt says what is not evidence for one: the agent introduced the mechanism, the developer asked a question, the developer accepted a suggestion, the pattern looks unfamiliar. Each is evidence only that something happened.

Stage 2 may decline

Triage decides a session is worth looking at. It never decides a misconception is there, and stage 2 — the first stage to see the agent's side and the diff — is the first thing that can tell. It can now answer {"decline": "..."}, recorded as the new terminal verdict declined.

Before this, a stage 2 that found nothing had two exits: invent a hypothesis, which teaches the developer the tool is guessing, or trip a structural gate and be filed as a broken pipeline. declined is deliberately neither error (nothing malfunctioned, and that rate is what says whether the prompt works) nor silent (triage kept this session, and the decline rate is the only way to see it collapsing yield rather than trimming it). Invisible to /grask: no question was written, so there is nothing to explain.

Two triage changes, both from the review's own counterexample

A developer asks why jitter is needed, gets an answer, and says "right, so the clients don't all retry at the same instant." asked_why fires, and it is wrong to.

• A correct restatement now disqualifies the moment. It is the strongest evidence available that there is no gap, and asking anyway spends the session's one question on the topic with the most evidence they already know it.
• A wrong restatement is the new signal explained_it_back, ranked above everything including asked_why. It had nowhere to land before: asked_why is gated on the quote being a question, and a confident wrong statement corrects nobody, so it was not pushed_back either. Everything below rank 0 is a reason to suspect a gap; rank 0 is a gap.

Answerability is stage 4's first question

A stem may name a local file as its setting, but if choosing between the options needs the contents of a local file the question does not quote, no option is a true answer and the judge marks them all false. The no-option-true rejection now carries every option's reason — without it, "the mechanisms are all broken" and "only its author could answer this" are the same string in the log.

What was measured

Check	Result
Disqualifier, on the review's transcript	silent, zero candidates
Same transcript, restatement made wrong	explained_it_back; shows named the error
Decline, on demonstrated understanding	fires
Decline, on a real gap	does not fire
Unquoted scripts/release.sh question	discarded, and the reason says why
Stem anchored on capture.py, portable mechanism	survives
Re-verifying 15 probes that had already passed stage 4	12 kept, 3 discarded — all three local-file recall
20 previously-scored sessions, re-run end to end	17 silent, 3 ask, 0 error, 0 declined

The 3 discards are correct catches, not over-rejection: the twelve kept include probes anchored on cmd/bd/create.go, ticks/reaper.py, and .github/dependabot.yml. It is a ~20% yield cut on the current corpus and a bug report against stage 3 — those questions should never have been written. The decline never fired in 3 opportunities, which shows it does not collapse yield and nothing more.

Two null results, kept as such

The stage-3 distractor taxonomy. A blind A/B — three seeds, both arms, six questions interleaved and classified by a judge not told which arm wrote which — came back 9 distractors per arm with exactly one banned shape in each. At n=3 that detects nothing. Kept because it costs ten lines and sits where the design says the lever is; the design doc says it is unproven rather than claiming a win.

Feeding the discard reason back into stage 3 (added in 3097bc1, reverted in 3245235). Both arms on the two discarded probes whose transcripts survived: all four re-asked questions passed stage 4, the two blind re-runs included. The premise — that a re-run needs to be told what went wrong — was not observed to hold, and an unproven keyword argument threaded through three modules costs more than unproven prompt text.

sessions.discard_reason survives the revert on its own merits: stage 4's judgment previously reached grask.log and stopped, a file rotated at 1 MB and written by a detached worker. It is what makes the locality rate queryable, which is the number the answerability check exists to move.

Notes for review

• New terminal verdict declined and new nullable column discard_reason, both migrated for existing databases.
• SIGNAL_RANK renumbers: adding explained_it_back at 0 shifts the other four down. Selection on an existing corpus will change where a session carries both signals.
• 461 tests, ruff and mypy clean, coverage 84.7%.
• Live-call scripts stayed in a scratchpad and out of tests/ — the suite must never reach the real CLI.
• What would settle the two nulls is the worth-asking vote, which is still unbuilt.

🤖 Generated with Claude Code