Skip to content

127.0.0.1:8000 is not exposing "thinking" properly #9

Description

@42e6

While Deepseek-API correctly informs chat.deepseek.com about the selected thinking and search toggles, it doesn't properly expose them on the local OpenAI-compatible API at 127.0.0.1:8000, as it sends the thinking text along the normal text with no distinction. This leaves AI harnesses no choice but to mush thinking text into the normal reply text. E.g. opencode can't wrap them in that little nice ⠟ Thinking … collapsible section.

127.0.0.1:8000 also eats the 1st token from both: the thinking text, and the normal text — when the stream of thinking text ends and the stream of normal reply text starts, the normal reply will have its 1st token eaten.

Here is how 127.0.0.1:8000 speaks OpenAI API locally when sending thinking text:

data: {"id": "chatcmpl-<...>", "object": "chat.completion.chunk", "created": <...>, "model": "deepseek-expert", "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": null}]}

data: {"id": "chatcmpl-<...>", "object": "chat.completion.chunk", "created": <...>, "model": "deepseek-expert", "choices": [{"index": 0, "delta": {"content": " user"}, "finish_reason": null}]}

<...>

Two problems:

  • The 1st delta is empty. It is supposed to contain token "The", but 127.0.0.1:8000 ate it.
  • The thinking delta is sent as normal text "content".

Solution:

  • Un-eat the 1st token.
  • Pass thinking tokens as "reasoning_content" instead of "content" as follows:
data: {
  "id": "chatcmpl-<...>",
  "object": "chat.completion.chunk",
  "created": <...>,
  "model": "deepseek-expert",
  "choices": [
    {
      "index": 0,
      "delta": {
        "reasoning_content": "The"
      },
      "finish_reason": null
    }
  ]
}
  • Switch deltas to "content" when the normal text starts (when thinking is over).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions