Surface ruflo dual-mode role delegation + agentic-qe agent-type routing in ak's provider UX

[pacphi]

Summary

ak x provider pick already wires ruflo's dual-host env flags (ENABLE_CLAUDE_CODE/ENABLE_CODEX) and agentic-qe's primary/fallback LLM provider (AQE_LLM_PROVIDER + .agentic-qe/llm-config.json). What it doesn't surface is the layer above that: ruflo and agentic-qe both support role/agent-type-scoped model delegation (not just "which providers are turned on"), and none of that is discoverable from ak's UX today. This issue records the source-grounded findings (via RuvNet Brain / search_ruvnet) and proposes a phased UX change to close the gap — scoped strictly to what ruflo and agentic-qe actually support today, not new capability we'd have to build ourselves.

Findings (source-grounded)

1. Two distinct "Claude + Codex together" mechanisms exist — don't conflate them

• ADR-034 "Optional MCP Backends" (ruflo/docs/adr/ADR-034-OPTIONAL-MCP-BACKENDS.md, Accepted) — env toggles (ENABLE_CLAUDE_CODE, ENABLE_CODEX, ENABLE_GEMINI_MCP) that let ruflo's agent loop run both host CLIs as MCP backends, tool-namespaced (claude__Read, codex__execute). This is the axis ak already wires. It has no role/delegation logic — it just turns both hosts on.
• @claude-flow/codex (separate npm package, bin claude-flow-codex; source v3/@claude-flow/codex/src/dual-mode/{orchestrator,cli}.ts) — implements actual role-based delegation: a dual CLI subcommand running a pipeline of WorkerConfig{id, platform:'claude'|'codex', role, prompt, dependsOn} objects over shared memory. ruflo init --dual requires this package as a prerequisite — src/lib/providers.mjs's ensureCodexAdapter()/ensureDualAgents() already installs it correctly, but nothing downstream tells the user it exists or how to use it.

Confirmed built-in role templates in cli.ts:

Template	Pipeline	Claude workers	Codex workers
feature	architect → coder → tester → reviewer	architect, reviewer	coder, tester
security	scanner → analyzer → fixer	analyzer	scanner, fixer
refactor	analyzer → planner → refactorer → validator	analyzer, planner	refactorer, validator

Custom role splits are supported via repeatable --worker "<platform>:<role>:<prompt>" flags (chained sequentially via implicit dependsOn, or --parallel-workers), or a --config path.json exporting {workers, taskContext}.

Today's gap: ak x provider pick enables the hosts and installs the adapter, but never mentions claude-flow-codex dual run --template … or the custom --worker syntax. A user who enables both hosts has no way to discover role delegation exists.

2. Judgment/evaluation quality — two measured findings worth acting on

• Cross-vendor judging is measurably less biased than same-vendor. v3/@claude-flow/cli/assets/model-router/openrouter-alts.json's judge_bias_check_2026_06_15 block: grading the same answers with openai/gpt-4.1 as judge instead of the default anthropic/claude-sonnet-4-6 scored everything 7–11pp harsher across the board but preserved the same relative ranking, and showed no Anthropic-family favoritism (gpt-4.1 was rated above Sonnet's own score). Actionable: same-vendor judges run ~8–10pp inflated but are ordinally trustworthy; use a different vendor when you need calibrated absolute scores.
• A cheap LM judge can score worse than a plain vote. agenticow/examples/multi-persona-consensus.mjs found "a verifier-gated cheap-LM judge picks WORSE than a plain vote — a negative generation–verification gap" and instead uses a deterministic two-part selector (hard-constraint gate + rubric-distance score) across N persona branches, discarding losers for free via COW branching.
• AQE's own qe-quality-gate agent (agentic-qe/.claude/agents/v3/qe-quality-gate.md) encodes the same lesson structurally via ADR-105 evidence tiers: gates block only on EXECUTED/STATIC evidence; INFERRED (model judgment) routes to adversarial verification (ADR-102) rather than being trusted solo; CONJECTURE never gates.

Today's gap: none of this is surfaced anywhere in ak. If a user wires an aqe fallback chain or a dual-mode judge worker, they get no guidance on vendor diversity for judging.

3. agentic-qe already has agent-type-scoped model routing — deeper than what ak exposes

• HybridRouter (ADR-043, agentic-qe/src/shared/llm/router/hybrid-router.ts) — 4 routing modes (manual / rule-based / cost-optimized / performance-optimized), config precedence override → env (AQE_LLM_*, provider keys) → .agentic-qe/llm-config.json → DEFAULT_ROUTER_CONFIG.
• Milestone 8 (agentic-qe/src/shared/llm/router/agent-router-config.ts, routing-rules.ts) — "Smart routing by agent type": a comprehensive agent-to-model mapping across all 59+ QE agent types, capability-based routing (reasoning/tools/cost), agent-category detection, and an override mechanism for specific tasks.
• ak's applyAqeRouter() (src/lib/providers.mjs) currently writes only a flat, global fallbackChain + defaultProvider — it has no notion of per-agent-type overrides, even though AQE's router supports them internally.
• Not yet confirmed: whether the per-agent-type override is exposed via a stable on-disk schema in .agentic-qe/llm-config.json today, or is programmatic-only (createModelRouter({...}) call sites). There's also a separately-confirmed, stable customization surface — .claude/agent-overrides/*.yaml (BMAD-002 overlay schema, agentic-qe/src/agents/overlay-schema.ts) which supports "Runtime configuration overrides" per agent and explicitly survives aqe init reinstalls — that may or may not be the right place to carry a provider override. This needs a source spike before any code is written (see Phase 3 below).

4. Current ak implementation snapshot (for reference)

src/lib/providers.mjs / src/commands/x/provider.mjs today:

• Detects claude/codex CLIs, installs whichever is absent, writes ENABLE_CLAUDE_CODE/ENABLE_CODEX.
• Installs @claude-flow/codex and runs ruflo init --dual --force when codex is opted in (prerequisite-only — never invokes the dual CLI itself).
• Lets the user set AQE_LLM_PROVIDER (primary) and an ordered aqeFallback chain, written to .agentic-qe/llm-config.json via applyAqeRouter().
• Has zero UX connecting "both hosts are enabled" to "here's how to actually delegate roles between them," and zero guidance on judge-vendor diversity.

Proposal: phased UX improvements

Scope is deliberately capped at surfacing and gently defaulting what ruflo/agentic-qe already support — no new routing capability invented inside ak.

Phase 1 — Guidance surfacing (no new state, low risk)

• After a successful pick with both hosts enabled, print a one-line pointer to role delegation: claude-flow-codex dual run --template feature|security|refactor (or custom --worker specs) — Claude designs/reviews, Codex implements/tests by default.
• When both hosts are enabled (in pick and in status), print a one-line judge-bias tip referencing the measured cross-grading result, recommending a different vendor as judge for calibrated scores.
• Acceptance: ak x provider pick --host claude,codex and ak x provider status both print the two tips; existing tests unaffected; new assertions added for the printed guidance.

Phase 2 — Smarter interactive defaults (no new persisted fields)

• In the interactive aqe fallback chain prompt, when codex is among the hosts being enabled and the user leaves the fallback prompt blank, pre-fill/suggest claude-code:<default>; openai:<default> instead of defaulting to empty — since Codex's models are reached via the openai provider type, this is a direct inference from the hosts already chosen in the same session.
• Acceptance: interactive pick with --host implying codex shows the suggested chain as the prompt default (still overridable); non-interactive --aqe-fallback behavior unchanged.

Phase 3 — Spike: per-agent-type provider overrides (build only if confirmed)

• Before writing code: confirm whether AQE's agent-router-config.ts / routing-rules.ts override mechanism has a stable, documented on-disk config surface (vs. programmatic-only), and whether .claude/agent-overrides/*.yaml (BMAD-002) is the intended carrier for a provider/model override field.
• If confirmed: extend ak x provider pick with an optional "route specific agent types to a specific provider" step (e.g. qe-security-scanner → openai, qe-test-architect → claude-code), written to whichever surface AQE actually reads.
• If not confirmed as stable/documented: close this phase as "not exposed yet upstream" rather than reverse-engineering an undocumented internal shape.

Phase 4 — Deferred, opt-in only (explicit sign-off required)

• A persisted kit.json role-split preference (e.g. providers.dualRoleSplit) that ak uses to scaffold package.json convenience scripts (dual:feature, dual:security, dual:refactor) pointing at the right claude-flow-codex dual run invocation. This is a real product opinion (scaffolding files into the user's package.json), not just wiring — held back pending explicit go-ahead rather than bundled with the rest.

Non-goals

• No changes to ruflo's or agentic-qe's own routing logic — this is ak UX only.
• No new LLM-judge implementation inside ak — Phase 1's guidance only points at existing measured findings, it doesn't build a judge.
• Phase 4 is explicitly excluded from this issue's initial scope.

[pacphi]

Amendment: qe-court (agentic-qe v3.13.0) — a third, more granular configuration surface

Re-grounded via RuvNet Brain against a fresh clone (the prior KB snapshot from 2026-07-07 predated this feature entirely and didn't have it at all — worth noting the ingest script's default github.com/ruvnet/<name> guess 404s for agentic-qe; its real upstream is github.com/proffesor-for-testing/agentic-qe). Confirmed installed on this machine: agentic-qe@3.13.0 (global, mise-managed) — not yet aqe init'd into this repo.

What it is

QE-Court (ADR-124, composes ADR-117–123, shipped in v3.13.0, docs/releases/v3.13.0.md / CHANGELOG.md) — an adversarial review protocol: a delivery (diff/PR/test-suite) is prosecuted by independent AI reviewers from different vendors, each with its own probe set, filing blind. A blind-refuter kill round (src/verification/adversarial-verify) discards weak charges; a cross-model jury (the existing two-gate LLM-judge, ADR-119) weighs what survives; a SHIP verdict must then survive an escalating overturn round before it stands (loop-until-dry: K consecutive deeper reviews finding nothing new — the genuinely new mechanic over prior AQE verification). Output is always three-valued — SHIP / REMAND / BLOCK — never bare pass/fail, and a numeric score is only emitted if its rubric clears an ANOVA screen (ADR-122).

Installed by default: qe-court is in the unconditional skill list in src/init/skills-installer.ts:72 and .claude/skills/skills-manifest.json, so any aqe init on agentic-qe ≥3.13.0 ships it — no opt-in flag to detect.

The court roster and its default routing (.claude/skills/qe-court/config.json)

Role	Job	Default provider
Defense	writer model, states case-for-ship only, never grades	cognitum-low
Prosecutor: devils-advocate	gap/assumption hunting	cognitum-mid
Prosecutor: brutal-honesty	rigor lens	claude-code (Sonnet)
Prosecutor: sherlock	deductive/root-cause	cognitum-high
Prosecutor: security-scanner	SAST	cognitum-mid
Prosecutor: mutation	test-adequacy	ollama
Prosecutor: codex-review	cross-vendor GPT lens	codex
Jury	two-gate judge, cross-model	cognitum-high
Deeper reviewer (overturn)	escalation	codex
Judge	rules on the strongest case both ways	you (human)

Invariants enforced in code (src/skills/qe-court/referee.ts, pure/dependency-free, unit-tested, Tier-3/verified — not just documented convention):

• validatePanel() — ≥2 distinct vendors seated (minVendors, default 2), and jury vendor ∉ {writer, defense} (a model may never grade its own or its writer's output).
• resolveVerdict() — the overturn asymmetry: charges surfaced only at depth ≤ overturnDepth count; overturnDepth: 0 disables escalation (regression-tested to prove the mechanic is load-bearing).
• shouldEmitScore() — a numeric score requires both emitScore: true AND the ADR-122 DoE/ANOVA screen having passed.

The concrete gap this surfaces for ak

qe-court's codex routing id traces to a new first-class LLMProviderType (src/shared/llm/interfaces.ts) backed by a real CodexProvider (src/shared/llm/providers/codex.ts, ADR-123 M3.5) — shells out to the Codex CLI on the user's ChatGPT subscription (strips OPENAI_API_KEY/CODEX_API_KEY from the child env so it can't silently revert to metered billing), the direct sibling of the existing claude-code provider (claude -p on the Claude subscription, which ak already knows about via HOSTS[].aqe).

However: codex is registered in provider-manager.ts's createProvider() switch and in config-store.ts's RUNTIME_CONSTRUCTIBLE_PROVIDERS set, but is missing from router/types.ts's ALL_PROVIDER_TYPES array — the array that gates AQE_LLM_PROVIDER=<type> validation. So today, AQE_LLM_PROVIDER=codex is rejected ("not a known provider") even though the provider is fully implemented and working — qe-court only reaches it because it calls ProviderManager.createProvider() directly, bypassing that gate.

Practical upshot for src/lib/providers.mjs's AQE_PROVIDER_TYPES list: it currently excludes 'codex', and that happens to match upstream's ALL_PROVIDER_TYPES today — but for the wrong reason (mirroring an apparent upstream oversight, not a real capability boundary). It will go stale silently the moment upstream fixes ALL_PROVIDER_TYPES to include 'codex'. Action: don't hardcode a workaround for this; just note it and re-check ALL_PROVIDER_TYPES on each aqe version bump (the existing ak x provider status / drift plumbing is the natural place).

More importantly, qe-court's routing map is a third configuration surface, more granular than anything ak manages today:

1. ruflo host env (ENABLE_CLAUDE_CODE/ENABLE_CODEX) — which CLI runs the agent loop
2. aqe global primary + fallback chain (AQE_LLM_PROVIDER, .agentic-qe/llm-config.json) — which LLM aqe's routers use, project-wide
3. NEW — per-court-role routing (.claude/skills/qe-court/config.json's routing map) — which specific provider plays which adversarial role, with a richer vocabulary (tiered cognitum-low/mid/high plus the bare codex id) and hard invariants enforced in code, not convention.

Proposed additional phases (extends the original Phase 1–4 plan; same non-goals apply — no reimplementation of the court itself)

• Phase A — guidance, no new state. Once agentic-qe ≥3.13.0 is detected, tell the user qe-court exists and that enabling both claude and codex hosts (which ak x provider pick already does) is exactly what satisfies its vendor-diversity invariant. This is the most natural cross-sell in the whole thread — dual-host is already ak's bread and butter, and qe-court is the first place that dual-host choice pays off as a hard requirement rather than a convenience.
• Phase B — read-only awareness. Teach ak x provider status to detect .claude/skills/qe-court/config.json once aqe init has created it, and print its current routing table plus a pass/fail against the two invariants. Reimplementing vendorOf()/validatePanel() from referee.ts is ~20 lines of pure, dependency-free logic — no need to shell out to aqe, and it can be tested the same way referee.test.ts tests the original.
• Phase C — opt-in write. When pick has already gathered both hosts + an aqe provider in the same interactive session, offer to patch qe-court's routing block with defaults derived from those choices (e.g. prosecutor.codex-review / deeperReviewer → codex once the codex host is enabled; jury → whichever aqe provider was picked, unless it collides with the writer's vendor, in which case pick a different one and say why). Validate against the validatePanel()-equivalent logic before writing — never silently produce an invalid panel.
• Version gate. All of the above behind an agentic-qe ≥3.13.0 check, reusing the existing installedVersion/drift plumbing in versions.mjs — none of it should fire against an older aqe that doesn't ship the skill.

Explicit non-goals (unchanged from the original issue's spirit)

• No changes to the court protocol itself (overturnDepth, emitScore, the prosecutor roster, DoE gating) — that's aqe's product surface.
• No reimplementation of adversarial-verify or the two-gate judge inside ak.
• ak only ever reads and defaults qe-court/config.json's routing block; it never touches the rest of the file.

[pacphi]

Phase 3 spike results: closing, no stable on-disk surface exists

Per the plan for Phase 3 ("confirm a stable on-disk surface before building anything"), I read the actual source of every candidate directly off the installed agentic-qe@3.13.0 package ($(npm root -g)/agentic-qe/dist/...) rather than relying on the earlier "unconfirmed" note.

1. agent-router-config.ts (Milestone 8, ADR-043) — programmatic only

AgentRoutingOverride has exactly the shape you'd want: condition: {agentTypes?, taskPatterns?, complexity?, requiresTools?, requiresReasoning?} + modelPreference: {provider, model, temperature, maxTokens?, priority}. But the only entry point is buildAgentRouterConfig({ customOverrides: AgentRoutingOverride[] }) — a plain JS function called with in-memory objects. I checked every exported function in agent-router-config.d.ts (getPreferredModelForAgent, createAgentRoutingRule, createAllAgentRoutingRules, etc.) — none of them touch the filesystem. overrides/agents/categoryDefaults are all in-memory Maps built from code constants (DEFAULT_CATEGORY_MODELS, AGENT_CATEGORY_MAP, DEFAULT_ROUTING_OVERRIDES). There is no loadOverridesFromDisk() or equivalent anywhere in this module.

2. .claude/agent-overrides/*.yaml (BMAD-002 overlay) — real surface, wrong schema

This one genuinely is a stable, documented, on-disk surface — its own doc comment says overlays "survive aqe init reinstalls." But AgentOverlay.config only has: minimumFindings, maxParallelAgents, preferredFrameworks, severityThresholds, needsContext. No provider or model field exists in the schema at all. It overrides an agent's content and behavior (description, instructions, capabilities) — never which LLM runs it. This was the "candidate carrier, unconfirmed" from the original amendment; it's now confirmed as a dead end for this purpose.

3. config-store.js — same global chain ak already writes

The code that actually reads .agentic-qe/llm-config.json off disk (mergeRouterConfig(DEFAULT_ROUTER_CONFIG, onDisk)) is exactly the global primary+fallback-chain schema ak's applyAqeRouter() already manages. No per-agent-type key exists in DEFAULT_ROUTER_CONFIG or the merge logic either.

Conclusion

Closing Phase 3 as "not exposed yet upstream," per the issue's own stated fallback. For ak to build this, upstream would need one of:

• agent-router-config.ts gaining a disk-load path that feeds buildAgentRouterConfig()'s customOverrides, or
• the BMAD-002 overlay schema growing a provider/model field that something downstream actually consumes, or
• a new dedicated file/schema for this purpose.

Without one of those, there's nothing for ak to write into — a made-up field on either existing surface would just be silently ignored by aqe's actual routing code path. Re-open (or file upstream against agentic-qe) if that changes.

---

Phases 1, 2, A, B, and C are implemented and tested in #38.

[pacphi]

Filed upstream as proffesor-for-testing/agentic-qe#568, requesting agentic-qe expose Phase 3's missing per-agent-type provider routing surface (extend .agentic-qe/llm-config.json with an agentOverrides map, wired into agent-router-config.ts's existing getPreferredModelForAgent()). If/when that lands, Phase 3 can be reopened here against a real on-disk contract.