Skip to content

[SSE Completion - Issue 2] Prior-turn reasoning leaks <think> into the visible answer #120

Description

@ashwing

Part of #118.

On a multi-round gateway tool session, raw <think>...</think> markup appears in the client's visible output_text channel (while reasoning also streams correctly on its own reasoning_text channel). Customer sees think-tags inside the final answer.

Root cause (localized with layered vLLM-direct vs gateway tests): the Continue branch in executor/engine.rs calls append_output_items_to_input(&mut ctx.enriched_request.input, &current_output), and OutputItem::to_input_item maps Reasoning → InputItem::Reasoning (types/io/output.rs:390). So each round feeds the previous round's reasoning item back into inference; the qwen3 reasoning parser then re-emits it wrapped in <think> on the visible channel.

vLLM is not at fault — it's clean unless a prior reasoning item is in the input history:

Test (vLLM-direct) <think> in output_text
single turn no
round-2 with function_call + output, no reasoning no
round-2 with round-1 reasoning fed back yes
round-2 reasoning excluded no (fix confirmed)

Fix: don't carry prior-turn reasoning items into subsequent inference rounds — reasoning is ephemeral per-turn. Scope the fix to the loop's Continue carry-forward (filter Reasoning out of the append_output_items_to_input(&current_output) call) rather than changing to_input_item globally — the latter is also used on the client-action / previous_response_id continuation paths, where whether reasoning should round-trip is a separate question worth not disturbing here. The excluded-reasoning test confirms the leak vanishes and the answer stays correct.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions