Skip to content

fix(ai-review): the incoherent-diff bail misfires on title-vs-diff scope mismatch, turning reviewable PRs into inconclusive holds #8789

Description

@JSONbored

Context

Confirmed production incident (loopover PRs #8735/#8737, 2026-07-26 ~01:40Z): the primary reviewer returned the deliberate bail (`INCOHERENT_DIFF_ASSESSMENT`, "Cannot review — the diff appears out of sync with the PR head", #7518/#7520 machinery) with this rationale captured in its own JSON: "The PR title/description describes a trivial test-fixture typing fix, but the diff shows a substantive production change (needsMinerDetection: true added at 7 call sites in processors.ts)".

The diff was CORRECT — #8735 genuinely adds `needsMinerDetection: true` at 7 `authorizePrActionActor` call sites; the PR's title merely undersells its own diff. Title-vs-diff scope mismatch is a REVIEWABLE quality signal (the same class as linked-issue scope mismatch), not evidence of a mechanically stale diff. The bail turned a reviewable PR into `ai_review_inconclusive` → manual-review hold (after the fallback also failed — its own issue, filed separately).

Requirements

  • Tighten the bail contract so it fires ONLY on mechanical incoherence: an empty/truncated/garbled diff, or file contents contradicting the PR's own file list — never on "the description doesn't match the (valid) diff".
  • A response that bails while citing description/title-vs-diff reasoning should be treated as a reviewable scope observation: proceed with the review (the model already demonstrated it can read the diff) or convert to a finding — never an abstention.
  • Keep the legitimate bail path intact for the case fix(review): short-circuit the retry loop on a deliberate INCOHERENT_DIFF_ASSESSMENT bail #7518 actually fixed (genuinely out-of-sync head).

Deliverables

Test Coverage Requirements

99%+ patch coverage, branch-counted.

Links

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions