[Bug]: The end of the Thinking/Reasoning content being returned in the output

[RCS-KBD]

App Version

v0.8.8

Device Information

Samsung Galaxy S23 Ultra

(Optional) API / Local and Model Info

Chutes

Description

Even with the "Remove Think Tags" enabled, in DeepSeek-R1 and ChatML formatting, it's returning the end of the thinking. The model is Chutes.AI's GLM-4.7-TEE, which uses vLLM. It should be returning a separate reasoning_content and content field, so there should be no reason to parse it. Any advice would be appreciated.

[mratsim]

Related to #384, happens to me to with vllm and say GLM-4.6V or GLM-4.5-Air.

I'm so annoyed that I'm building a proxy that will intercept and handle reasoning_content for ChatterUI. mratsim/llm-reasoning-proxy#1

[mratsim]

Also I think vLLM parser for GLM-4.x models has issues. I get nested <think><think>...content... [missing token]</think>end of content</think>

I can reproduce that reliably with GLM-4.5-Air-Fp8.

I don't have the issue at all with SGLang.

[Vali-98]

This has likely been resolved with the new parser.

[mratsim]

Actually I resolved it by avoiding sending stop tokens in my queries.