fix(orb): record PR outcomes from the bot's own action, not the webhook alone

[JSONbored]

What

Makes a PR's realized outcome depend on the action the bot actually completed, not on GitHub redelivering a webhook. Closes #8823.

Why — a silent, measurable regression that biased accuracy upward

pr_outcome was written only from the inbound pull_request.closed webhook. Any delivery this instance never processed — a dropped brokered relay event, a restart mid-delivery — lost that PR's ground truth permanently. Because the fleet export inner-joins gate_decision to pr_outcome, an outcome-less PR falls out of calibration entirely: it contributes to neither the numerator nor the denominator of any accuracy metric.

Measured on the live self-host (2026-07-26), bot closes with no pr_outcome row, by day:

day	bot closes	missing outcome
07-15	120	0
07-16	110	0
07-17	105	0
07-18	57	8
07-19	64	28
07-20	103	53
07-21	197	123
07-24	132	93

A clean regression starting 2026-07-18, running at ~55% loss since. Spot-checked against GitHub — the PRs are genuinely closed:

PR	GitHub state	pr_outcome
#8428	CLOSED 2026-07-24T13:00:02Z	none
#8728	CLOSED 2026-07-26T01:19:47Z	none

It is not specific to one close path (contributor-cap closes and ordinary gate closes are both affected), which is why the fix targets the dependency itself rather than a call site.

This is the same root cause behind 40 of the 66 reversal_superseded markers never reaching fleet calibration (#8820). Since the lost rows skew toward the gate's mistakes — a superseded close is by definition a wrong close — their absence pushed published accuracy up.

How

• outcomes-wire.ts: recordTerminalActionOutcome(env, repo, pr, decision) writes the realized outcome directly. Idempotent — skips when a pr_outcome row already exists, so whichever of the two paths wins the race, exactly one row survives. Best-effort throughout: bookkeeping can never fail an action that already succeeded against GitHub, and if the idempotency probe itself fails it writes anyway (a duplicate is strictly better than a lost outcome; both readers take the latest row).
• agent-action-executor.ts: call it after a successful merge and close.
• scripts/backfill-pr-outcomes{,-core}.ts: closes the historical hole. The completed terminal action is the ground truth — GitHub rejects a merge the actor may not perform, and a completed close is the repository's realized decision. Pure planning core + thin IO wrapper, dry-run by default, idempotent, and it skips the legacy project:pull_request:owner/repo#N target shape the fleet export already excludes rather than fabricating outcomes for pre-cutover rows.

Ops (after merge, on the self-host)

node --experimental-strip-types scripts/backfill-pr-outcomes.ts --dry-run
node --experimental-strip-types scripts/backfill-pr-outcomes.ts --apply

then rewind orb_export_cursor so the corrected rows re-export to fleet calibration (ingest upserts, so replay is safe).

Verification

• TSC clean. 660 tests green across the full executor-importing suite set (outcomes-wire, agent-action-executor, agent-approval-queue, routes-agent-approval, queue-lifecycle-guards, queue, queue-2, processors-public-comment-merge-facts), plus 12 new backfill tests.
• New regression tests cover: webhook-free recording of both decisions, idempotency against the webhook path, a failing idempotency probe still writing, and a failing audit_events mirror never breaking the canonical review_audit write.
• Changed-line coverage: 0 uncovered statements/branches across changed and new files.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.83%. Comparing base (f84ab1d) to head (339cb59).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8824   +/-   ##
=======================================
  Coverage   93.83%   93.83%           
=======================================
  Files         800      800           
  Lines       79831    79842   +11     
  Branches    24196    24197    +1     
=======================================
+ Hits        74908    74919   +11     
  Misses       3558     3558           
  Partials     1365     1365           

Flag	Coverage Δ
backend	95.11% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/review/outcomes-wire.ts	93.11% <100.00%> (+0.23%)	⬆️
src/services/agent-action-executor.ts	97.28% <100.00%> (+0.01%)	⬆️