fix(gate): never close a PR for visual evidence the pipeline cannot produce - #9934
Conversation
…roduce (#9881) `screenshotTableGate.action: "close"` closed PRs when no visual evidence was found. Correct when evidence is merely absent; wrong when it was never obtainable. A repo with `review.visual.enabled` but no preview deploys at all (JSONbored/awesome-claude) exhausts its poll budget, never sets visualCaptureSatisfiedSha, and every contributor PR touching a visual-scoped path is closed one-shot -- for evidence no contributor action could ever supply, against a misconfiguration nobody was told about. The proof was already in the capture: build state `absent` means no preview check-run was found AT ALL, and the file already noted this was "confirmed live on a repo whose UI has no preview-deploy CI configured". Paired with a spent poll budget, that distinguishes "no pipeline" from a late deploy or a renderer blip, which is exactly the distinction the close was missing. The violation is UNCHANGED -- the PR really does lack visual evidence and the finding still says so. Only enforcement degrades, and the contributor-visible reason names the maintainer-side remedy, since no contributor action can resolve it. Marked per head SHA like its siblings, so a later push re-arms the requirement and a repo that gains preview deploys stops matching on its very next commit.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
loopover-ui | 2ea08bb | Commit Preview URL Branch Preview URL |
Jul 30 2026, 05:07 PM |
|
Caution 🛑 LoopOver review result - fixes requiredReview updated: 2026-07-30 17:30:24 UTC
Review summary Nits — 6 non-blocking
CI checks failing
Decision drivers
Context & advisory signals — never blocks the verdict
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Decision record
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #9934 +/- ##
==========================================
- Coverage 91.88% 91.87% -0.02%
==========================================
Files 923 923
Lines 113407 113451 +44
Branches 27332 27338 +6
==========================================
+ Hits 104205 104232 +27
- Misses 7915 7930 +15
- Partials 1287 1289 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Closes #9881.
The defect
screenshotTableGate.action: "close"closes a PR when no visual evidence is found. That is correct when evidence is merely absent. It is wrong when evidence was never obtainable.A repo with
review.visual.enabled: truebut no preview deployments at all —JSONbored/awesome-claudeis the live case — exhausts its preview-poll budget, never setsvisualCaptureSatisfiedSha, and every contributor PR touching a visual-scoped path is then closed one-shot. No contributor action other than hand-authoring the table could ever change that, and nothing had told the maintainer their configuration was unsatisfiable.Current blast radius: latent, not firing. I checked the six most recently closed-unmerged PRs on
awesome-claude— all touch zero visual-scoped files, so the gate did not close them. It triggers on the next PR that touchesapps/web/src/{components,routes}/**without a hand-authored table.The proof was already being computed
capture.tsdistinguishes preview build states, andabsentmeans no preview check-run was found at all — the file already notes this was "confirmed live on a repo whose UI has no preview-deploy CI configured: every PR's 'after' cell spun forever."absentplus a spent poll budget (5 polls over 7.5 minutes) is conclusive: not a late deploy, not a renderer blip, no pipeline. That pairing is surfaced asCaptureResult.previewUnobtainable, persisted per head SHA asvisual_capture_unobtainable_sha(migration 0207), and read by the gate.What changes, and what deliberately does not
action: "close"becomes advisory for that head, with an audit event (screenshot_table_close_degraded_capture_unobtainable) recording it.requireScreenshotTable.action: advisory— because no contributor action can resolve it.visualCaptureSatisfiedShaandvisualCaptureRetryPendingSha. A later push re-arms the requirement; a repo that gains preview deploys stops matching on its very next commit; a successful capture clears the marker.Decisions I made
The issue offered three directions and raised a fourth question; I took the narrow, reversible ones:
ScreenshotTableGateActiontier. The issue notes"close" | "advisory"has no middle setting, which is a fair point — but it is full config-as-code parity work (engine type → default → GATE key → normalizer → resolver →types.ts→openapi/schemas.ts→ both example YAMLs → docs drift) and this fix does not need it. Left for its own issue.Validation
tsc --noEmit,db:migrations:check(contiguous 0001..0207),db:schema-drift:check,engine-parity:drift-check,dead-exports:check,docs:drift-check,git diff --checkall clean.