ACT Phase 5: Integrate trained critic into Squall review pipeline

[DSado88]

Context

After Phase 4 produces a trained critic model that passes evaluation (>65% preference accuracy), integrate it into Squall's review pipeline as a post-processing scoring pass.

Depends on: #18 (feedback), #19 (structured findings), Phase 4 DPO training (successful eval)

Proposal

Integration point

After the review fan-out returns and before results are serialized, run the critic as a scoring pass:

Models return → Extract findings (#19) → Critic scores each finding → Annotated results → Claude synthesizes

Scoring output

Each finding gets an actionable_score: f64 (0.0-1.0) from the critic:

{
  "finding_id": "abc123",
  "summary": "Race condition in session middleware",
  "severity": "high",
  "actionable_score": 0.87,
  "model_key": "grok"
}

Deployment options

A. Local inference (preferred)

• Serve Qwen3-8B via Ollama or vLLM on Mac Studio
• Squall calls it as a new CLI-backend model
• ~2-5s per finding, parallelizable

B. Squall-native scoring

• Add a critic backend type in dispatch registry
• Dedicated scoring endpoint, not a full review model
• Lower latency but more Rust code

Start with A (reuses existing CLI dispatch). Migrate to B if latency matters.

UX

Invisible to the user. Claude sees the scores in the review results and uses them during synthesis. Higher-scored findings get elevated; lower-scored ones get tagged as "likely noise."

The user's workflow doesn't change:

squall review → findings (now with scores) → claude synthesizes (informed by scores) → "you agree?" → fix via TDD

Feedback loop

The critic's predictions feed back into training:

• Critic said 0.9, finding was acted on → good calibration
• Critic said 0.9, finding was dismissed → overconfident, needs training signal
• Critic said 0.2, finding was acted on → underconfident, valuable correction

This closes the loop: usage improves the critic, improved critic improves usage.

Success criteria

• Critic scores appear in review results JSON
• Claude's synthesis quality measurably improves (fewer false elevations)
• No latency regression >10% on review end-to-end time
• Feedback loop captures critic prediction vs actual outcome

Estimated scope

• Option A: ~100 LoC Rust (config + scoring pass) + model config
• Option B: ~300 LoC Rust (new backend type)