feat(task): auto-generate autorouting tiers from declared providers - #3764
feat(task): auto-generate autorouting tiers from declared providers#3764Yeachan-Heo wants to merge 2 commits into
Conversation
Adds a fixed fast/balanced/strong tier vocabulary for Task-tool spawns, gated behind `task.autorouting.enabled` (default false). Tiers come from `task.autorouting.tiers` or one of the locked presets (anthropic, openai-codex, google, xai); explicit tiers win over a preset and an omitted item tier routes as balanced. An autorouting pin overrides the manual model chain, unmatched tiers fall back to manual resolution per item with a bounded reason, and routing evidence propagates to the task result, receipt, renderer, and task-summary prompt. Selectors must be exact provider-qualified `provider/modelId` strings with an optional thinking suffix; globs, bare ids, and `pi/<role>` aliases are rejected by the generated config schema. With autorouting disabled the model-resolution path is unchanged.
Turn sub-agent autorouting from a hand-authored tier map into a declare-your-providers flow: pick the providers you have, in priority order, and gjc materializes deterministic fast/balanced/strong cross-provider fallback chains for you. Curation and generation: - Add a curated per-model tier-label map in packages/coding-agent, seeded 1:1 from the shipped presets, with a skip list and a version fingerprint. Only current models carry labels, so "newest only" is a curation discipline rather than recency-filter code. - Add a pure, clock-free generator over the full model catalog. Given a declaration it emits byte-identical chains: declaration order, then curation rank, then a stable tie-break, deduped. Credentials and disabled providers are routing-time facts and never generation inputs, so adding an API key cannot silently change routing. - A provider that lacks a label for a tier contributes nothing to it. Unfillable tiers stay absent and fall through to the existing manual fallback rather than being padded with a model that was never labeled for that tier. - Add a CI forcing gate, wired into both required root checks, that fails when a new current catalog model is neither labeled nor skip-listed. Settings and TUI: - Add typed task.autorouting.setup and .provenance behind a narrow optional-object setting kind, wired through the schema, validation, reconciliation and JSON-schema generation. Both default to absent, so untouched configs round-trip byte-identically and the disabled path is unchanged. - Add a smart-routing panel reachable from the /model presets landing. It edits provider order and an optional model allowlist, previews the generated chains, and commits tiers + setup + provenance in one atomic batch, so what you previewed is exactly what is written. Refresh regenerates from the recorded declaration, Clear removes the generated keys while preserving preset and enabled, and the toggle writes only enabled. Hand-edited tiers are detected by fingerprint and never overwritten without explicit confirmation. Runtime: - Add an autorouting preflight mode for initial routed tasks: a bounded ledger of at most three unique candidates spanning throwaway probes and the durable attempt. Only typed transient pre-start failures advance; anything unclassified is terminal, and nothing switches once the request fence is crossed, so there is still no mid-run failover. - Publish failed attempts nowhere. Staged sessions suppress breadcrumbs and session-list registration until a single commit point, staged paths are excluded from every discovery reader, and artifacts are attempt-scoped with a serialized reservation and a frozen id map that is applied before the transcript becomes visible. - Record why routing did what it did: bounded skip evidence that distinguishes disabled providers from missing snapshot entries and missing credentials, a phase-tagged attempt ledger, and terminal outcomes that survive even when no candidate ever executes. The declaration is the only priority channel; the optional model list filters eligibility and never reorders. Manual tiers and presets remain as the power-user escape hatch.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f0462a8ba2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| await this.ctx.settings.commitAtomicBatchWithCurrent(() => | ||
| buildAutoroutingSettingsBatch({ | ||
| tiers: preview.tiers, | ||
| setup: preview.setup, | ||
| provenance: preview.provenance, | ||
| }), | ||
| ); |
There was a problem hiding this comment.
When another process edits the generated tiers after the settings snapshot was loaded—or between #assertSmartRoutingNotHandEdited and this write—the guard examines stale in-memory data, while this callback ignores the current on-disk configuration supplied by commitAtomicBatchWithCurrent. Apply therefore silently overwrites the concurrent hand edit without the required confirmation; Refresh repeats the same pattern. Re-evaluate the provenance/tier fingerprint from current inside the locked callback before returning the patches.
Useful? React with 👍 / 👎.
| const attempts: AutoroutingAttempt[] = []; | ||
| const consumed = new Set<string>(); | ||
| const skips = buildBoundedRoutingSkips(options.autoroutingSkips); | ||
| const candidates = options.autoroutingCandidates.map(boundedSelector).filter(selector => selector.length > 0); |
There was a problem hiding this comment.
When a valid custom provider/model selector exceeds 256 characters or contains NFKC-sensitive characters, this applies evidence sanitization to the selector before it is used as modelOverride. Although the original selector already matched the registry snapshot, the truncated or normalized value no longer identifies that model, so preflight can incorrectly exhaust the route. Keep the original candidates for execution and call boundedSelector only when recording routing evidence.
Useful? React with 👍 / 👎.
| if (result.routing) { | ||
| const model = result.routing.effectiveModel ?? "not-executed"; | ||
| lines.push(`${continuePrefix}${theme.fg("dim", `Routing: ${model} ${result.routing.note ?? ""}`)}`); |
There was a problem hiding this comment.
When a custom provider reports a model identifier containing tabs or ANSI/control sequences, effectiveModel is interpolated directly into the Task tool renderer, allowing malformed alignment or terminal escape injection; the new routing note is also rendered without the shared width bound. Apply replaceTabs() and truncateToWidth()/ui.truncate() to the complete routing line before passing it to the theme renderer.
AGENTS.md reference: AGENTS.md:L115-L121
Useful? React with 👍 / 👎.
Computer/task backlog ownership routingExact head Signature: GJC backlog census | PR #3764 | exact-head |
Signed task-autorouting conflict handoffOwner: existing branch/PR mutation owner A three-way
Scope boundary: resolve only this rebase and directly required generated/schema/test fallout for the task-autorouting change. Do not absorb unrelated source cleanup. Reconfirm the branch head before pushing because this handoff is signed only for the SHA pair above. No duplicate mutation owner was started, and no CI workflow was rerun or cancelled. Signature: GJC task-autorouting batch lane | owner |
Owner hostile review —
|
Maintainer triage — LEFT OPEN, blocked (not mergeable in current state)Verified against current
Path to merge: rebase onto current |
1 participant
Turns sub-agent autorouting from a hand-authored tier map into a declare-your-providers flow: pick the providers you have, in priority order, and gjc materializes deterministic
fast/balanced/strongcross-provider fallback chains for you.Builds on the opt-in autorouting feature in
982e6b9.The UX
In the panel you set provider order (the only priority channel) and an optional model allowlist (filters eligibility, never reorders), see a live preview of the generated chains, and hit Apply. That commits
tiers+setup+provenancein one atomic batch — what you previewed is exactly what is written.Then a Task spawn only needs
tier: fast|balanced|strong. Tier-by-tier model selection is gone.What's in it
Curation & generation
packages/coding-agent, seeded 1:1 from the shipped presets, with a skip list and version fingerprint. Only current models carry labels, so "newest only" is a curation discipline rather than recency-filter code.Settings & TUI
task.autorouting.setup/.provenancebehind a narrowoptional-objectsetting kind, wired through schema, validation, reconciliation and JSON-schema generation. Both default to absent, so untouched configs round-trip byte-identically and the disabled path is unchanged./modelpresets landing. Refresh regenerates from the recorded declaration; Clear removes the generated keys while preservingpresetandenabled; the toggle writes onlyenabled. Hand-edited tiers are detected by fingerprint and never overwritten without explicit confirmation.Runtime
Non-goals (unchanged)
No dynamic runtime tier derivation, no credential-based auto-activation, no mid-run failover, no cross-provider scalar quality ranking, no CLI/wizard as the primary surface, no general model-selection presets. Manual
tiersandpresetremain the power-user escape hatch.Verification
packages/aiandschemas/models.schema.jsonare untouched by design.Pre-existing and unrelated (reproduced on a clean worktree at the base commit): 4
FileSessionStorageWriterpath-security failures insession-storage.test.ts, and rootcheck:sdk-closure.Review
Eight review generations across an ai-slop-cleaner lane, an architect lane and an executor QA/red-team lane closed 25 findings, then a terminal critic signed off. Four were genuine defects that would otherwise have shipped:
not_directorybug that made managed autorouted publication impossible;The adversarial suite (
autorouting-boundary-redteam.test.ts, 39 cases) covers forged previews, catalog races, attempt-id traversal, cyclic/substring/nested URI re-keying, bounded skip projection, breadcrumb interleaving, cleanup-failure surfacing, and leaked-id retirement.Accepted limitations