Skip to content

kosong: OpenAILegacy emits reasoning_effort: null for thinking "off" — invalid for strict APIs and does not disable reasoning #2465

Description

@0xbentang

Summary

kosong 0.53.0's OpenAILegacy provider puts "reasoning_effort": null on the
wire when thinking effort is off. This is wrong in two ways: null is not a
valid value in the OpenAI chat-completions schema (it must be an enum string, or
the field absent), and it does not actually disable reasoning. Lenient backends
(e.g. Ollama Cloud) accept null but treat it as "reasoning on by default", so
thinking stays enabled; strict validators (e.g. Synthetic.new, TypeBox) reject
it with HTTP 400, which then drives a retry/rate-limit loop.

Root cause

  • chat_provider/openai_common.pythinking_effort_to_reasoning_effort("off")
    returns Python None (line ~120).
  • contrib/chat_provider/openai_legacy.py::generate() sets
    reasoning_effort = self._reasoning_effort (~129) and passes
    reasoning_effort=reasoning_effort to client.chat.completions.create()
    (~148). The OpenAI SDK serializes an explicit None as null; only the
    omit/Omit sentinel is dropped from the payload.
  • The auto-enable guard at ~134 only handles Omit, so a None from
    with_thinking("off") (~167) flows straight through.

Steps to reproduce

  1. provider.with_thinking("off") on an OpenAILegacy provider.
  2. generate(...) against a strict OpenAI-compatible endpoint.
  3. Outgoing body contains "reasoning_effort": null → HTTP 400 on strict
    validators. Wire check:
    curl .../v1/chat/completions -d '{"model":"...","messages":[...],"reasoning_effort":null}'
    returns null is not equal to none/low/medium/high... (TypeBox), while a
    valid string succeeds.

Expected behavior

off should send a valid value that actually disables reasoning, not null.

Suggested fix

Map off to the string "none" in thinking_effort_to_reasoning_effort
instead of None. "none" is a valid enum value, passes strict validators, and
disables thinking on backends that honor it (Ollama Cloud, Synthetic/TypeBox).

Caveat: a few OpenAI-compatible backends reject "none" (e.g. Fireworks wants
"low"), and native OpenAI only added "none" for gpt-5.1+. A provider-aware
mapping or a configurable off-value fallback may be warranted. Can send a PR.

Environment

  • kosong 0.53.0 (confirmed on main, packages/kosong)
  • Endpoints observed: Ollama Cloud (accepts null, reasoning stays on),
    Synthetic.new (400)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions