ai(fallback): Ollama gets an oversized prompt with no num_ctx, full blocker authority, and votes attributed to Claude

[JSONbored]

Summary

When Claude is unavailable, reviews fall back to local Ollama — which is given a prompt far larger than its context window with no num_ctx set, silently truncates it, and is then held to identical blocker authority as Claude. Nothing in the verdict, the public comment, or the finding says which model actually decided. Reviewer votes are additionally attributed to the primary model even when the fallback produced the review.

1. The fallback is given a prompt it cannot fit

The review call sends up to 120,000 chars of diff plus an aggregate context budget of up to 240,000 chars (~60k+ tokens) — src/services/ai-review.ts:1483-1523, :1178, :1124.

The Ollama chat call (src/selfhost/ai.ts:374-402) sets max_tokens, temperature and response_format but no providerOptions / num_ctx. The only place num_ctx is ever set in the codebase is the vision call (src/queue/processors.ts:8808) — even though AiRunOptions.providerOptions was added (#4327/#4335) for exactly this.

Ollama silently left-truncates input beyond num_ctx (server/model default, typically 4k–8k tokens) and answers confidently over whatever survived — which is the tail of the prompt (context sections), not the diff.

2. The fallback carries full blocker authority

A fallback-produced review is held to the same schema validation and the same confidence semantics against aiReviewCloseConfidence. A llama3.1-class self-reported 0.95 is not the same instrument as Claude's, but the gate cannot tell them apart. Combined with #9476 (Claude timeouts retrying 3×), a Claude outage means every review is decided this way.

Only loopover_ai_review_model_fallback_total, the diagnostics, and parsedReviewModelIds record that a fallback happened — the verdict, the public copy ("AI review flagged a likely critical defect") and the finding itself are silent.

3. Votes are attributed to the wrong model

src/services/ai-review.ts:2935-2936, :3001:

if (a.review) reviewerVotes.push({ reviewer: primary.model, votedFail: … });

runWorkersOpinion iterates [primary, fallback] internally and ReviewerOpinionOutcome carries no model identity, so an Ollama-produced opinion is recorded as a claude-code vote. The doc comment claims "swap-proof attribution … slot↔model is unambiguous by construction" — true for the tie-break order swap, false for in-slot provider fallback.

Those votes become reviewer_vote audit events and feed recordRoutingShadow and scoreJudgmentAgreement. This directly poisons the input data of #8229 (evidence-weighted reviewer routing). parsedReviewModelIds (:569-571) already has the correct value from the diagnostics.

Deliverables

• Pass providerOptions: { num_ctx } for the Ollama review path, sized to the deployed model; fail loudly rather than silently truncate when the prompt does not fit (shrink deliberately, or refuse and hold).
• Tier the fallback's authority: a lone blocker from a fallback-served review should route to hold_for_review rather than a one-shot close. State the chosen policy explicitly in the gate documentation.
• Name the serving provider in the finding, the decision record, and the public copy, so a reader can tell which instrument produced the verdict.
• Return the producing model from runWorkersOpinion (or derive it from the last status: "parsed" diagnostic) and use it for reviewerVotes.
• Backfill consideration: existing reviewer_vote rows are unreliable — decide whether calibration: evidence-weighted reviewer routing (flag-gated, report-first) #8229's calibration should discard votes recorded before this fix.

Tests (must fail against current main)

• Ollama review path asserts num_ctx is set and the prompt fits, or that the call is refused.
• A fallback-produced blocker does not yield a one-shot close under the chosen policy.
• reviewerVotes[].reviewer equals the model that actually produced the opinion when the fallback ran.
• Provider identity appears in the decision record.

Related

• ai: claude_stalled_no_output escapes the no-retry guard — every Claude timeout costs 3 full attempts and parks the queue #9476 — makes fallback far more frequent than it looks.
• calibration: evidence-weighted reviewer routing (flag-gated, report-first) #8229 — consumes the misattributed votes.

[linear-code]

JSO-6542

[JSONbored]

PR #9513 implements the context bound and the vote attribution halves.

Deliberately NOT included: tiering the fallback provider's blocker authority (routing a fallback-served lone blocker to hold_for_review rather than a one-shot close).

That is a policy change to the gate's disposition, not a correctness fix, and the case for it is weaker now than when the issue was filed. The original argument was that a fallback verdict may have been produced from a fraction of the diff — Ollama silently left-truncating past num_ctx and answering over the prompt tail. With an explicit num_ctx now sent, the model actually sees the change, so "the fallback saw less" is no longer the reason to distrust it.

What remains is the narrower claim that a local model's self-reported 0.95 is not the same instrument as Claude's against aiReviewCloseConfidence. That is probably true, but it is a calibration question best answered with data — the reviewer-vote attribution fixed in #9513 is precisely what makes that data trustworthy, since fallback-produced votes were previously recorded under the primary's name and poisoned #8229's routing track records.

Suggested sequence: let #9513 land, let corrected votes accumulate, then decide the authority tier from the observed agreement rate rather than from first principles. Leaving this issue open for that decision.

One related gap also worth noting and not addressed here: nothing in the verdict, the public copy ("AI review flagged a likely critical defect"), or the finding surfaces which provider decided — only the metrics and diagnostics do. Naming the serving provider in the decision record would make the calibration question answerable from the record itself.