feat(llm): eval-gated demote-down — the router cost lever

[HomenShum]

Roadmap #2 of the LLM router. Adds DEMOTE-DOWN — the Prism cost lever.

A pool opts in with mode: "demote" to default to its quality TARGET (heaviest) and drop to a cheaper candidate only on clearly-light turns, and only to models eval-CLEARED for the task class. For over-provisioned paths (e.g. a persona router pinning Opus every turn).

router (shared/llm/router.ts)

• RouteMode + TaskPool.mode? (default escalate — existing pools UNCHANGED)
• RouteDecision.demoted (additive)
• DEMOTE_THRESHOLD (0.25) — only trivial turns demote
• isDemoteCleared: conservative static DEMOTE_CLEARANCE + pluggable RouteOptions.clearance hook (the seam for the live agentRunJudge feed)
• FAIL-SAFE: nothing cleared → stay on target; quality never dropped un-cleared
• agent_reason is the first demote pool (Opus target, demote→Sonnet). No live caller yet → behavior-preserving today; the cache-sticky agent wiring (Fix agent type mismatches and tool wrappers #3) is the first caller.

Pure + DETERMINISTIC. Additive — the opts param + demoted field don't touch /ask or search callers (tsc clean).

Tests (+8 scenario)

demote trivial→Sonnet; hard→Opus; forceTarget→Opus; fail-safe un-cleared→Opus; live-clearance override both ways; determinism; threshold boundary; escalate pools never demoted.

Verification

tsc --noEmit clean; 20 router tests pass; build clean.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
nodebench-ai	Ready	Preview, Comment	Jun 2, 2026 7:10pm

[augmentcode]

🤖 Augment PR Summary

Summary: This PR adds an opt-in demote-down routing mode to the shared LLM router, enabling cost savings on over-provisioned paths while keeping quality protected by an eval-style clearance gate.

Changes:

• Introduces RouteMode (escalate default, demote opt-in) and TaskPool.mode.
• Adds RouteDecision.demoted, DEMOTE_THRESHOLD, and a conservative static DEMOTE_CLEARANCE allowlist.
• Adds RouteOptions.clearance hook to override static clearance (seam for future live eval feed).
• Implements demote-mode routing: default to the heaviest target, demote only on clearly-light turns and only to cleared cheaper candidates (fail-safe: nothing cleared → stay on target).
• Marks agent_reason as the first demote pool (Opus target, demote→Sonnet) and updates architecture docs/roadmap accordingly.
• Adds scenario tests covering demotion behavior, fail-safe, override hook, determinism, threshold boundary, and ensuring escalate pools are unaffected.

Technical Notes: The demote mechanism is additive to existing callers (default mode remains escalate), and the clearance system is designed to be conservative until a live agentRunJudge/dogfood feed is wired.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 2 suggestions posted.

Comment augment review to trigger a new review at any time.

[augmentcode]

shared/llm/router.ts:L42 — The header comment says routeLLM is a pure function of (taskClass, signals, env), but it now also depends on opts (and especially opts.clearance). Consider updating this invariant (or documenting that clearance must be deterministic for replay safety) so the determinism guarantee matches the new API.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augmentcode]

shared/llm/router.ts:L298 — In demote mode, signals defaults to {}, so computeComplexityScore becomes 0 and a caller that forgets to pass signals will demote on any cleared candidate, which seems to contradict the “uncertainty resolves UP” guarantee. Consider treating missing/empty signal cases as uncertain and staying on the target (heaviest) rather than demoting.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}