Context
Found while fixing #8820. The fleet-calibration export (FLEET_QUERY, src/selfhost/orb-collector.ts) inner-joins gate_decision to pr_outcome in review_audit — a PR with no pr_outcome row is never exported at all, so it contributes to neither the numerator nor the denominator of any published accuracy metric.
Measured on the live self-host (2026-07-26):
This is not a stale-data artifact. Spot-checked two of the missing targets against GitHub:
| PR |
GitHub state |
pr_outcome row |
| #8728 |
CLOSED 2026-07-26T01:19:47Z |
none |
| #8748 |
CLOSED 2026-07-26T02:36:34Z |
none |
Both were genuinely bot-closed (they have agent.action.close audit events — that is exactly how recordSupersededReversals found them), yet no pr_outcome was ever written. So the outcome writer is missing cases the close actuator completes.
Because the dropped rows are disproportionately errors, their absence biases the published accuracy upward. With #8822 the number reads 93.6%; folding in the missing 40 would lower it further.
Requirements
- Determine why
pr_outcome is not written for these bot-closed PRs. Start at recordPrOutcome/the pull_request closed webhook path in src/review/outcomes-wire.ts (the writer is documented as firing "on a pull_request closed webhook") and compare against agent.action.close completions — a self-closed PR may not deliver a webhook the instance processes, or the handler may bail before the write.
- Fix the gap so every terminal gate action records its realized outcome. A bot-initiated close must not depend on a webhook round-trip the instance might never see.
- Backfill the missing
pr_outcome rows for targets that have a completed agent.action.close/agent.action.merge but no outcome, so historical calibration is corrected (the fleet export re-sends on cursor rewind, and ingest upserts, so a backfill propagates safely).
- Add an invariant test: a completed terminal gate action always yields a
pr_outcome row.
- Regression test for the specific shape above (bot-closed PR, no inbound webhook → outcome still recorded).
Test Coverage Requirements
99%+ patch coverage, branch-counted, on all changed src/** lines — both arms of every new conditional.
Expected Outcome
Every bot-closed and bot-merged PR carries a pr_outcome row; the fleet export stops dropping ~61% of detected reversals; published decision accuracy reflects the full error population rather than the subset that happened to get an outcome row.
Context
Found while fixing #8820. The fleet-calibration export (
FLEET_QUERY,src/selfhost/orb-collector.ts) inner-joinsgate_decisiontopr_outcomeinreview_audit— a PR with nopr_outcomerow is never exported at all, so it contributes to neither the numerator nor the denominator of any published accuracy metric.Measured on the live self-host (2026-07-26):
reversal_supersededmarker (calibration: detect one-shot-culture reversals — bot-closed PR superseded by a merged successor (reversal_superseded) #8166 — the one-shot culture's dominant "the bot was wrong" shape).pr_outcomerow and therefore export.This is not a stale-data artifact. Spot-checked two of the missing targets against GitHub:
pr_outcomerowBoth were genuinely bot-closed (they have
agent.action.closeaudit events — that is exactly howrecordSupersededReversalsfound them), yet nopr_outcomewas ever written. So the outcome writer is missing cases the close actuator completes.Because the dropped rows are disproportionately errors, their absence biases the published accuracy upward. With #8822 the number reads 93.6%; folding in the missing 40 would lower it further.
Requirements
pr_outcomeis not written for these bot-closed PRs. Start atrecordPrOutcome/thepull_requestclosedwebhook path insrc/review/outcomes-wire.ts(the writer is documented as firing "on apull_requestclosedwebhook") and compare againstagent.action.closecompletions — a self-closed PR may not deliver a webhook the instance processes, or the handler may bail before the write.pr_outcomerows for targets that have a completedagent.action.close/agent.action.mergebut no outcome, so historical calibration is corrected (the fleet export re-sends on cursor rewind, and ingest upserts, so a backfill propagates safely).pr_outcomerow.Test Coverage Requirements
99%+ patch coverage, branch-counted, on all changed
src/**lines — both arms of every new conditional.Expected Outcome
Every bot-closed and bot-merged PR carries a
pr_outcomerow; the fleet export stops dropping ~61% of detected reversals; published decision accuracy reflects the full error population rather than the subset that happened to get an outcome row.