fix(review): stop AI-review verdict-shopping — content-fingerprint stickiness + flip-count escalation

[JSONbored]

Closes #9016.

Security advisory: https://github.com/JSONbored/loopover/security/advisories/GHSA-5w8q-r748-9j9c

The AI reviewer is non-deterministic even at temperature 0, and the review cache keys purely on head SHA — so a contributor could force fresh re-rolls (a no-op recommit invalidating the cache key, or a same-head retry once the 30-minute non-cacheable cooldown lapses) until a lucky CLEAN roll auto-merged a PR another roll had flagged as blocked.

Fix 1 — content-fingerprint stickiness across head SHAs (getCachedAiReviewAcrossHeads in src/db/repositories.ts): a fallback used only when the exact-head cache lookup misses. aiReviewCacheInputFingerprint already hashes the actual per-file PATCH content (not just paths), so an identical fingerprint under a DIFFERENT head SHA means the reviewed content is genuinely unchanged — this reuses the prior verdict instead of spending a fresh, independently-random roll on content the reviewer already judged. Same cacheable/recency rules as the exact-head lookup (durable rows reuse unboundedly; dynamic-context rows only within the existing cooldown, never once published).

Fix 2 — per-PR verdict-flip escalation (migration 0183, src/review/verdict-flip-guard.ts pure + verdict-flip-store.ts IO): every FRESH (non-cache-hit) verdict in block mode is compared against the PR's last fresh verdict — a flip is a change in whether the verdict carried a blocking AI-judgment defect (ai_consensus_defect/ai_review_split). Repeating the same verdict never adds to the count (ordinary re-review isn't abuse); once flips clear the threshold (3), the gate holds for a human via ai_review_inconclusive, regardless of what the newest roll says — closing the exact exploit ("retry until blockers come back empty").

Regression fix along the way: the pre-existing #ops-review-burst: a genuinely new commit... test had a fixture that returned IDENTICAL patch content across two different head SHAs — exactly the no-op-recommit case fix #1 is designed to reuse, which made its own aiCalls assertion test the wrong thing once this PR landed. Fixed by varying real content between heads (via upsertPullRequestFile, matching how a real webhook-driven detail-sync would refresh storage) — it now correctly stays a valid regression for its ORIGINAL concern.

Tests: 15 pure/store/repository tests (every flip-state transition including the exploit's own defect→clean→defect→clean oscillation pattern, the fingerprint-fallback's cacheable/non-cacheable/scope/recency/malformed-date/future-date/no-options edge cases, D1 fail-open on both read and write), plus an end-to-end queue test proving a pre-seeded threshold PR's next clean roll is genuinely held (ai_review_inconclusive in gate_outcomes, the audit event recorded, the metric incremented, the PR never merged) and an advisory-mode test proving the mechanism never engages outside block mode. Changed-line coverage: repositories.ts and the pure/store modules are fully covered; two defensive fail-open .catch() arms in the wiring (an infra-failure guard around the fingerprint fallback, and around the escalation's own audit-event write) remain uncovered — consistent with dozens of similarly-guarded, untested bare catches already throughout this exact file (e.g. lines 2003, 2392, 2658, 2777, 3054) — forcing them without destabilizing the surrounding webhook harness proved impractical within scope; flagging for visibility rather than leaving unaddressed.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

❌ Patch coverage is 90.47619% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.17%. Comparing base (3c1173e) to head (7fc63c6).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/queue/processors.ts	63.63%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9094      +/-   ##
==========================================
+ Coverage   93.88%   95.17%   +1.28%     
==========================================
  Files         810      716      -94     
  Lines       80600    58151   -22449     
  Branches    24454    20584    -3870     
==========================================
- Hits        75675    55345   -20330     
+ Misses       3560     1617    -1943     
+ Partials     1365     1189     -176     

Flag	Coverage Δ
backend	95.17% <90.47%> (-0.01%)	⬇️
control-plane	?
rees	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/db/repositories.ts	96.77% <100.00%> (+0.01%)	⬆️
src/review/verdict-flip-guard.ts	100.00% <100.00%> (ø)
src/review/verdict-flip-store.ts	100.00% <100.00%> (ø)
src/selfhost/metrics.ts	100.00% <ø> (ø)
src/selfhost/pg-dialect.ts	100.00% <ø> (ø)
src/queue/processors.ts	95.61% <63.63%> (-0.12%)	⬇️

... and 96 files with indirect coverage changes

[loopover-orb]

Caution

🛑 LoopOver review result - fixes required

_{Review updated: 2026-07-26 17:47:12 UTC}

9 files · 1 AI reviewer · no blockers · CI failing · unstable

🛑 Suggested Action - Manual Review

Review summary
This adds two complementary anti-verdict-shopping defenses for #9016: a content-fingerprint cache fallback across head SHAs (getCachedAiReviewAcrossHeads) and a per-PR verdict-flip counter that escalates to a human hold after 3 oscillations (verdict-flip-guard.ts + verdict-flip-store.ts), wired into processors.ts's block-mode publish path with a new migration 0183 and self-host pg-dialect/schema-drift entries. The logic is well-tested (pure state machine unit tests, D1 fail-open tests, and end-to-end queue tests simulating the exploit tail), the migration is a simple non-temp CREATE TABLE with no D1-authorizer concerns, and the fix targets a real security advisory rather than being scope-creep. The one thing I can't verify from the diff/context is whether `aiReview.persistable` and `aiReview.findings` typings actually include the fields referenced at processors.ts's new call site, but nothing in the visible diff suggests those are new/undefined.

Nits — 6 non-blocking

• src/review/verdict-flip-store.ts's console.warn calls (per the external brief) are pre-existing repo convention for fail-open logging elsewhere, but confirm they don't leak sensitive PR content beyond repoFullName/pullNumber/truncated error message.
• The CI FAILED checks (validate-tests, validate) have no detail provided in what I was given, so I can't verify whether they relate to this diff's content or an unrelated/base-branch issue — worth checking those logs directly before merge.
• getCachedAiReviewAcrossHeads's fingerprint scan is bounded to 20 rows (AI_REVIEW_FINGERPRINT_SCAN_LIMIT) and ordered by created_at DESC, which is reasonable, but consider indexing (repo_full_name, pull_number, ai_review_mode, created_at) if this table grows large across many repos.
• Double check that `settings.aiReviewMode === "block"` gating in processors.ts fully excludes advisory mode from ever writing a flip-state row (the added test 'advisory-mode reviews never touch the flip-escalation wiring' covers this, which is good).
• Consider a short doc comment cross-referencing VERDICT_FLIP_ESCALATION_THRESHOLD's value (3) to the security advisory GHSA-5w8q-r748-9j9c for future readers who land on this file without PR context.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

CI checks failing

• validate
• validate-tests

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#9016, #1
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	❌ 8/20	High review scope from cached public metadata (2 linked issues).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 13 registered-repo PR(s), 13 merged, 323 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 13 PR(s), 323 issue(s).
Improvement	✅ Minor	risk: clean · value: minor

Linked issue satisfaction

Addressed
The PR directly implements both fixes requested by the issue: content-fingerprint-based cache stickiness across head SHAs (getCachedAiReviewAcrossHeads) so a no-op recommit reuses the prior verdict, and a per-PR verdict-flip counter (verdict-flip-guard.ts/verdict-flip-store.ts, migration 0183) that escalates to a human hold after repeated verdict oscillation, matching the acceptance criteria almos

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, Ruby, Go, MDX, Shell, Solidity, JavaScript
• Official Gittensor activity: 13 PR(s), 323 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
• Then work through the remaining 2 steps in the Signals table above.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record

• action: hold · clause: success
• config: 03a7f8b529a9 · pack: oss-anti-slop
• record: ec82416c1b02 (schema v3, head dc768f2)

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review