feat(scripts): backfill the risk-control calibration set from decision history

[JSONbored]

Advances #8828.

The risk-control guarantee (#8835) needs adjudicated (confidence, correct?) pairs and the naive path is weeks of live label accrual. But ai_review_cache has retained per-finding confidence since 2026-06-28 and review_audit holds every acted gate decision — this backfill stages calibration candidates from that existing evidence, and adds the pre-registered α(n) tightening schedule so the published guarantee strengthens itself as labels accrue.

• Pure core (scripts/backfill-decision-labels-core.ts): acted closes stage as close_arm (a close whose realized outcome is merged = reopened-and-merged = the definitive incorrect class). Holds whose only blockers were AI-judgment findings stage as holdout_close for ANALYSIS only — adjudicating the backfill proved surviving holds are not valid close counterfactuals (a hold that survived means a non-confidence criterion blocked the close), so apply inserts close_arm rows only; the live ε-holdout (calibration: randomized ε-holdout on would-close PRs with propensity logging #8831) samples the would-close population and stays the valid source going forward.
• Thin IO wrapper: stdin candidates → {records, labels, worklist}. Records reuse buildDecisionRecord/canonicalJson (byte-identical digests); configDigest: "backfill:unavailable" is the honest provenance sentinel; no decision-ledger writes (the chain attests live finalizations only). Labels stage status='pending' — invisible to loadCalibrationPairs until adjudication flips them.
• Pre-registered α(n) schedule (src/review/risk-control-wire.ts): when no env α pins an arm, the close arm's budget follows a deterministic function of usable-label count — 0.05 under 350 pairs, 0.025 from 350, 0.015 from 700. Sample size is information, not the test statistic, so the walk is not outcome-snooping and each daily recalibration still runs exactly one level-δ test; the homepage claim self-tightens 95% → 97.5% → 98.5% with zero human steps. (The α-grid alternative needs a Bonferroni δ-split that raises every floor ~35% and certifies nothing at today's volume.)

Ran against production history: 653 candidates → 331 staged (232 close_arm, 99 analysis-only holdout_close), 280 mixed-blocker holds excluded, 42 duplicates deduped. Adjudicated close_arm set: 226 usable pairs, 9 incorrect (96.0% acted-close precision, coherent with the published 95.7% baseline); at today's volume the schedule's first tier certifies λ̂=0.62 at 57% coverage.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	loopover-ui	d8bf3f8	Commit Preview URL Branch Preview URL	Jul 26 2026, 02:13 PM

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
21936	1	21935	21

View the top 1 failed test(s) by shortest run time

test/unit/miner-worktree-allocator-collisions.test.ts > loopover-miner worktree allocator collisions (#4298) > returns distinct worktree paths when multiple processes acquire simultaneously

Stack Traces | 0.453s run time

AssertionError: expected 2 to be 5 // Object.is equality

- Expected
+ Received

- 5
+ 2

 ❯ test/unit/miner-worktree-allocator-collisions.test.ts:148:41

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}