[codex] Route hybrid direct-content tasks to reasoning guidance#6
Draft
sambarrowclough wants to merge 1 commit into
Draft
[codex] Route hybrid direct-content tasks to reasoning guidance#6sambarrowclough wants to merge 1 commit into
sambarrowclough wants to merge 1 commit into
Conversation
sambarrowclough
force-pushed
the
codex/qwen-reasoning-routing
branch
from
June 9, 2026 12:02
842c954 to
936a8df
Compare
sambarrowclough
force-pushed
the
codex/qwen-reasoning-routing
branch
from
June 9, 2026 12:03
936a8df to
3a06e8a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a high-signal routing rule to
together-chat-completionsfor direct-content tasks on hybridreasoning models.
Qwen/Qwen3.5-9Bis the observed failing case, not a claim about everyQwen3-family model.
This traces back to the core
voice-agentbenchmark task:That benchmark is intentionally broad: it combines realtime STT, a low-latency chat turn, and
realtime TTS. During investigation, we reduced the broad task to the
qwen-reasoning-contentinvestigative task:
That task identified the failure family: for low-latency/direct-response uses of a hybrid
reasoning model, the agent needs to know when to disable reasoning so usable text lands in
message.content.This PR was then validated with the reduced
qwen-content-low-tokentracer, which removes thevoice-agent framing and tests the smallest repeatable API behavior:
When a task uses a hybrid reasoning model, expects a direct answer in
response.choices[0].message.content, and sets a smallmax_tokensbudget, the agent should readthe reasoning-model guidance first and pass
reasoning={"enabled": False}unless the userexplicitly asked for reasoning output.
Why
Some hybrid reasoning models have reasoning enabled by default. In a low-token direct-response
task, the model can spend the token budget on reasoning and leave
message.contentempty. Theexisting reference docs already contained the correct behavior, but the skill entry point routed
agents through the basic chat path first, so they only found the reasoning guidance after observing
empty output.
Evidence
Eval evidence branch: https://github.com/togethercomputer/together-agent-eval/tree/codex/eval-loop-tracers
Repro/provenance doc: https://github.com/togethercomputer/together-agent-eval/blob/codex/eval-loop-tracers/harness/verify/qwen-reasoning-content-repro.md
qwen-reasoning-contentinvestigative matrix, run on 2026-06-09:Reduced
qwen-content-low-tokentracer before the skill routing fix, using the old skill at tracercommit
bdb86a9:skillsrunsrecovered-after-empty-contentReduced
qwen-content-low-tokentracer after this skill routing fix:skillsrunsfixed-reasoning-disabledThe verifier checks the saved session/code for the target model, a chat completion call,
reasoning={"enabled": False}, no empty-content reproduction, and the expected printedhello.Validation
These validation commands run in this skills repo/PR branch, not in the eval evidence branch: