Skip to content

[codex] Route hybrid direct-content tasks to reasoning guidance#6

Draft
sambarrowclough wants to merge 1 commit into
zainhas:mainfrom
sambarrowclough:codex/qwen-reasoning-routing
Draft

[codex] Route hybrid direct-content tasks to reasoning guidance#6
sambarrowclough wants to merge 1 commit into
zainhas:mainfrom
sambarrowclough:codex/qwen-reasoning-routing

Conversation

@sambarrowclough

@sambarrowclough sambarrowclough commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a high-signal routing rule to together-chat-completions for direct-content tasks on hybrid
reasoning models. Qwen/Qwen3.5-9B is the observed failing case, not a claim about every
Qwen3-family model.

This traces back to the core voice-agent benchmark task:

Write and run a streaming voice agent backend that uses Together's serverless APIs for Whisper
realtime STT, Qwen X and Kokoro realtime TTS for the lowest end-to-end latency. Use Websocket APIs
where available.

That benchmark is intentionally broad: it combines realtime STT, a low-latency chat turn, and
realtime TTS. During investigation, we reduced the broad task to the qwen-reasoning-content
investigative task:

Using the Together Python SDK, make one low-latency voice-agent chat turn with model
`Qwen/Qwen3.5-9B`. Ask it: `Reply with exactly: hello`. Print the spoken response text that
would be sent to TTS. Run the code and verify the printed spoken text is non-empty.

That task identified the failure family: for low-latency/direct-response uses of a hybrid
reasoning model, the agent needs to know when to disable reasoning so usable text lands in
message.content.

This PR was then validated with the reduced qwen-content-low-token tracer, which removes the
voice-agent framing and tests the smallest repeatable API behavior:

Using the Together Python SDK and model `Qwen/Qwen3.5-9B`, ask `Reply with exactly: hello` with
`max_tokens=32`. Print `response.choices[0].message.content`. Run it and make sure the printed
value is non-empty.

When a task uses a hybrid reasoning model, expects a direct answer in
response.choices[0].message.content, and sets a small max_tokens budget, the agent should read
the reasoning-model guidance first and pass reasoning={"enabled": False} unless the user
explicitly asked for reasoning output.

Why

Some hybrid reasoning models have reasoning enabled by default. In a low-token direct-response
task, the model can spend the token budget on reasoning and leave message.content empty. The
existing reference docs already contained the correct behavior, but the skill entry point routed
agents through the basic chat path first, so they only found the reasoning guidance after observing
empty output.

Evidence

Eval evidence branch: https://github.com/togethercomputer/together-agent-eval/tree/codex/eval-loop-tracers

Repro/provenance doc: https://github.com/togethercomputer/together-agent-eval/blob/codex/eval-loop-tracers/harness/verify/qwen-reasoning-content-repro.md

qwen-reasoning-content investigative matrix, run on 2026-06-09:

  • baseline: 2 classified runs, 2 issue reproductions
  • MCP: 2 classified runs, 2 verified fixes
  • skills: 2 classified runs, 2 verified fixes

Reduced qwen-content-low-token tracer before the skill routing fix, using the old skill at tracer
commit bdb86a9:

  • 3 classified skills runs
  • 2/3 verifier-confirmed issue reproductions
  • 2/3 recovered-after-empty-content
  • 0 harness failures

Reduced qwen-content-low-token tracer after this skill routing fix:

  • 3 classified skills runs
  • 3/3 fixed-reasoning-disabled
  • 0/3 issue reproductions
  • 0/3 recoveries after reproducing issue
  • 0 harness failures

The verifier checks the saved session/code for the target model, a chat completion call,
reasoning={"enabled": False}, no empty-content reproduction, and the expected printed hello.

Validation

These validation commands run in this skills repo/PR branch, not in the eval evidence branch:

python scripts/quick_validate.py skills/together-chat-completions
python scripts/quality_check.py

@sambarrowclough
sambarrowclough force-pushed the codex/qwen-reasoning-routing branch from 842c954 to 936a8df Compare June 9, 2026 12:02
@sambarrowclough
sambarrowclough force-pushed the codex/qwen-reasoning-routing branch from 936a8df to 3a06e8a Compare June 9, 2026 12:03
@sambarrowclough sambarrowclough changed the title [codex] Route Qwen direct content tasks to reasoning guidance [codex] Route hybrid direct-content tasks to reasoning guidance Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant