Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ account を削除しても mapping は保持され、同じ id を再追加す
| `noTemperatureModels?` | `string[]` |発信者指定の`temperature`を拒否するモデル。 |
| `noTopPModels?` | `string[]` |発信者指定の`top_p`を拒否するモデル。 |
| `noPenaltyModels?` | `string[]` |存在/周波数ペナルティを拒否するモデル。 |
| `noStructuredOutputModels?` | `string[]` | `openai-chat` エンドポイントが `response_format` を拒否する正確なモデル ID。要求モデルが項目と完全一致する場合だけフィールドを省略し、その他の `openai-chat` モデルでは structured-output 変換を維持します。 |
| `parallelToolCalls?` | `boolean` |並列ツール呼び出しを切り替えます。 OpenAI Chat はデフォルトでオンになっています。非チャット アダプターは明示的な `true` でのみアドバタイズします。 |
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` |正確なプレースホルダー ID、欠落している端末 ID、および(`repairInvalidIds` で)正規の `msg_`/`rs_` 接頭辞を欠く message/reasoning ID に対するダウンストリーム SSE 修復はデフォルトで無効になっています。関数呼び出し ID は決して書き換えられません。組み込み DeepSeek は最後の 2 つをデフォルトで有効にします。 |
| `responsesSnapshotRepair?` | `boolean` | デフォルトで無効のクライアント向け修復です。SSE と JSON の Responses ライフサイクルで欠落した status、output、ツールメタデータを補完し、raw 検査と永続化は変更しません。 |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ managed map을 활성화하면 privacy-safe selector를 만들고, 이후 계정
| `noTemperatureModels?` | `string[]` | 호출자가 지정한 `temperature`를 거부하는 모델입니다. |
| `noTopPModels?` | `string[]` | 호출자가 지정한 `top_p`를 거부하는 모델입니다. |
| `noPenaltyModels?` | `string[]` | presence/frequency penalty를 허용하지 않는 모델입니다. |
| `noStructuredOutputModels?` | `string[]` | `openai-chat` 엔드포인트가 `response_format`을 거부하는 정확한 모델 ID입니다. 요청 모델이 항목과 정확히 일치할 때만 필드를 생략하며, 그 외 `openai-chat` 모델에서는 structured-output 변환을 유지합니다. |
| `parallelToolCalls?` | `boolean` | 병렬 도구 호출을 켜거나 끕니다. OpenAI Chat은 기본으로 켜져 있고, 비-chat 어댑터는 명시적으로 `true`일 때만 이를 노출합니다. |
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | 기본값이 꺼진 downstream SSE 복구입니다. 정확한 자리표시자 id, 누락된 종료 id, 그리고(`repairInvalidIds`) 정규 `msg_`/`rs_` 접두사가 없는 message/reasoning id를 복구합니다. function-call id는 다시 쓰지 않습니다. 내장 DeepSeek은 마지막 두 가지를 기본으로 켭니다. |
| `responsesSnapshotRepair?` | `boolean` | 기본값이 꺼진 클라이언트용 복구입니다. SSE와 JSON의 Responses 수명 주기에서 누락된 status, output, 도구 메타데이터를 채우며 raw 검사와 영속화는 변경하지 않습니다. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids.
| `noTemperatureModels?` | `string[]` | Models that reject caller-specified `temperature`. |
| `noTopPModels?` | `string[]` | Models that reject caller-specified `top_p`. |
| `noPenaltyModels?` | `string[]` | Models that reject presence/frequency penalties. |
| `noStructuredOutputModels?` | `string[]` | Exact model IDs whose `openai-chat` endpoint rejects `response_format`. Only an exact requested-model match omits the field; structured-output translation stays enabled for every other `openai-chat` model. |
| `parallelToolCalls?` | `boolean` | Toggle parallel tool calls. OpenAI Chat defaults on; non-chat adapters advertise only on explicit `true`. |
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | Disabled-by-default downstream SSE repair for exact placeholder ids, missing terminal ids, and (with `repairInvalidIds`) message/reasoning ids missing the canonical `msg_`/`rs_` prefix. Function-call ids are never rewritten. Built-in DeepSeek enables the last two by default. |
| `responsesSnapshotRepair?` | `boolean` | Disabled-by-default client-facing repair for sparse Responses lifecycle snapshots in SSE and JSON. Fills missing canonical status, output, and tool metadata while raw inspection and persistence remain unchanged. |
Expand Down
7 changes: 4 additions & 3 deletions docs-site/src/content/docs/reference/proxy-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,9 @@ Structured output is part of that translation: `response_format` with `json_obje
`json_schema` is forwarded to routed `openai-chat` models. On `POST /v1/responses` the
equivalent request field is `text.format`: native Responses routes preserve it in the raw
Responses body, and it is translated to `response_format` when the model routes to an
`openai-chat` provider. A backend without structured-output support returns its own error
instead of the proxy rejecting the request locally.
`openai-chat` provider. A model listed in the provider's `noStructuredOutputModels` omits
`response_format` on that chat wire; sibling models keep the translation. Unclassified backends
receive the field and return their own error instead of the proxy guessing their capability.

Non-streaming output has `object: "chat.completion"`. Streaming output uses SSE objects with
`object: "chat.completion.chunk"`, choice deltas, a terminal choice with `finish_reason`, and
Expand Down Expand Up @@ -287,4 +288,4 @@ structurally valid Fernet runs unchanged. If an `agent_message` loses all encryp
that repair, it becomes a normal user message. If a current v2 task remains genuinely encrypted
but the selected routed target cannot read native ChatGPT ciphertext, opencodex fails with
`unreadable_encrypted_agent_task` instead of sending unreadable bytes to that provider. See
[Sub-agent Surface](/guides/sub-agent-surface/) for the client behavior around worker tasks.
[Sub-agent Surface](/guides/sub-agent-surface/) for the client behavior around worker tasks.
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ cross-route credential fallback не существует. Строки API GPT-
| `noTemperatureModels?` | `string[]` | Модели, отвергающие переданный вызывающей стороной `temperature`. |
| `noTopPModels?` | `string[]` | Модели, отвергающие переданный вызывающей стороной `top_p`. |
| `noPenaltyModels?` | `string[]` | Модели, отвергающие penalty presence/frequency. |
| `noStructuredOutputModels?` | `string[]` | Точные идентификаторы моделей, чей endpoint `openai-chat` отклоняет `response_format`. Поле опускается только при точном совпадении запрошенной модели; для остальных моделей `openai-chat` преобразование structured output остаётся включённым. |
| `parallelToolCalls?` | `boolean` | Переключатель parallel tool call'ов. Для OpenAI Chat по умолчанию включено; не-chat adapter'ы рекламируют это только при явном `true`. |
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | По умолчанию выключенная downstream SSE-repair для exact placeholder-id, отсутствующих terminal-id и (с `repairInvalidIds`) message/reasoning id без канонического префикса `msg_`/`rs_`. Function-call id никогда не переписываются. Встроенный DeepSeek включает последние два по умолчанию. |
| `responsesSnapshotRepair?` | `boolean` | По умолчанию выключенная клиентская repair для неполных lifecycle snapshot'ов Responses в SSE и JSON. Добавляет отсутствующие status, output и tool metadata, не меняя raw inspection и persistence. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ selector,而不是分配一个新名称。
| `noTemperatureModels?` | `string[]` | 会拒绝调用方指定 `temperature` 的模型。 |
| `noTopPModels?` | `string[]` | 会拒绝调用方指定 `top_p` 的模型。 |
| `noPenaltyModels?` | `string[]` | 会拒绝 presence/frequency penalty 的模型。 |
| `noStructuredOutputModels?` | `string[]` | `openai-chat` 端点拒绝 `response_format` 的精确模型 ID。仅当请求模型与条目完全匹配时才省略该字段;其他 `openai-chat` 模型仍启用 structured-output 转换。 |
| `parallelToolCalls?` | `boolean` | 切换并行工具调用。OpenAI Chat 默认开启;非 chat 适配器只有显式 `true` 时才会声明支持。 |
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; repairInvalidIds?: boolean }` | 默认关闭的下游 SSE 修复,用于精确占位 id、缺失的终止 id,以及(`repairInvalidIds`)缺少规范 `msg_`/`rs_` 前缀的 message/reasoning id。function-call id 永远不会被重写。内置 DeepSeek 默认启用后两项。 |
| `responsesSnapshotRepair?` | `boolean` | 默认关闭的客户端修复,用于补全 SSE 与 JSON 中稀疏 Responses 生命周期快照缺失的 status、output 和工具元数据;原始检查与持久化保持不变。 |
Expand Down
32 changes: 19 additions & 13 deletions src/adapters/openai-chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -813,19 +813,25 @@ export function createOpenAIChatAdapter(provider: OcxProviderConfig): ProviderAd
if (provider.promptCacheKey && parsed.options.promptCacheKey !== undefined) {
body.prompt_cache_key = parsed.options.promptCacheKey;
}
const textFormat = parsed.options.textFormat;
if (textFormat?.type === "json_object") {
body.response_format = { type: "json_object" };
} else if (textFormat?.type === "json_schema") {
body.response_format = {
type: "json_schema",
json_schema: {
name: textFormat.name ?? "response",
...(textFormat.description !== undefined ? { description: textFormat.description } : {}),
...(textFormat.schema !== undefined ? { schema: textFormat.schema } : {}),
...(textFormat.strict !== undefined ? { strict: textFormat.strict } : {}),
},
};
// Structured-output support varies by the physical upstream model even when one
// gateway exposes a uniform OpenAI-compatible endpoint. Keep the #1137 translation
// as the default, but let an exact model opt out instead of forcing a provider-wide
// rollback that would silently return prose for siblings that support JSON Schema.
if (!provider.noStructuredOutputModels?.includes(parsed.modelId)) {
const textFormat = parsed.options.textFormat;
if (textFormat?.type === "json_object") {
body.response_format = { type: "json_object" };
} else if (textFormat?.type === "json_schema") {
body.response_format = {
type: "json_schema",
json_schema: {
name: textFormat.name ?? "response",
...(textFormat.description !== undefined ? { description: textFormat.description } : {}),
...(textFormat.schema !== undefined ? { schema: textFormat.schema } : {}),
...(textFormat.strict !== undefined ? { strict: textFormat.strict } : {}),
},
};
}
}

if (tools) {
Expand Down
34 changes: 34 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,9 @@ const providerConfigSchema = z.object({
supportsServiceTier: z.boolean().optional(),
preserveResponsesReasoningContent: z.boolean().optional(),
allowPrivateNetwork: z.boolean().optional(),
noStructuredOutputModels: z.array(z.string().min(1))
.transform(normalizeNonBlankStringArray)
.optional(),
retryOn429: retryOn429PolicySchema.optional(),
codexAccountMode: z.enum(["pool", "direct"]).optional(),
responsesItemIdRepair: z.object({
Expand Down Expand Up @@ -820,6 +823,26 @@ export function positiveIntegerConfigError(value: unknown, field: string): strin
return null;
}

export function nonBlankStringArrayConfigError(value: unknown, field: string): string | null {
if (value === undefined) return null;
if (!Array.isArray(value)) return `${field} must be an array`;
for (const [index, entry] of value.entries()) {
if (typeof entry !== "string" || !entry.trim()) {
return `${field}.${index} must be a nonblank model id`;
}
}
return null;
}

/**
* Keep hand-edited config and management writes on one canonical model-id list.
* Validation happens separately so an all-whitespace value is rejected rather than
* normalized into a model id that can never match at runtime.
*/
export function normalizeNonBlankStringArray(value: readonly string[]): string[] {
return [...new Set(value.map(entry => entry.trim()))];
}

export function booleanRecordConfigError(value: unknown, field: string): string | null {
if (value === undefined) return null;
if (!value || typeof value !== "object" || Array.isArray(value)) return `${field} must be a plain object`;
Expand Down Expand Up @@ -1408,6 +1431,17 @@ const configSchema = z.object({
message: maxOutputError,
});
}
const structuredOutputOptOutError = nonBlankStringArrayConfigError(
(provider as { noStructuredOutputModels?: unknown }).noStructuredOutputModels,
"noStructuredOutputModels",
);
if (structuredOutputOptOutError) {
ctx.addIssue({
code: "custom",
path: ["providers", redactSecretString(name), "noStructuredOutputModels"],
message: structuredOutputOptOutError,
});
}
if (Object.hasOwn(provider, "codexAccountMode") && provider.codexAccountMode !== undefined) {
// Persisted account mode is valid ONLY on the canonical built-in `openai` forward provider.
// Old openai-multi rows stay parseable (they never carry a mode) so startup can migrate them.
Expand Down
7 changes: 7 additions & 0 deletions src/server/auth-cors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
modelAdapterRecordConfigError,
modelPreferHostedToolsConfigError,
codexAutoStartEnabled,
nonBlankStringArrayConfigError,
positiveIntegerConfigError,
positiveIntegerRecordConfigError,
providerBaseUrlConfigError,
Expand Down Expand Up @@ -509,6 +510,11 @@ export function providerManagementConfigError(name: unknown, provider: unknown):
if (defaultMaxOutputError) return `provider ${name} ${defaultMaxOutputError}`;
const maxOutputError = positiveIntegerRecordConfigError(raw.modelMaxOutputTokens, "modelMaxOutputTokens");
if (maxOutputError) return `provider ${name} ${maxOutputError}`;
const structuredOutputOptOutError = nonBlankStringArrayConfigError(
raw.noStructuredOutputModels,
"noStructuredOutputModels",
);
if (structuredOutputOptOutError) return `provider ${name} ${structuredOutputOptOutError}`;
const openRouterError = openRouterRoutingConfigError(typed);
if (openRouterError) return `provider ${name} ${openRouterError}`;
if (typed.authMode === "local") {
Expand Down Expand Up @@ -589,6 +595,7 @@ export function safeConfigDTO(config: OcxConfig): unknown {
"noTemperatureModels",
"noTopPModels",
"noPenaltyModels",
"noStructuredOutputModels",
"autoToolChoiceOnlyModels",
"preserveReasoningContentModels",
"requiresReasoningPlaceholderModels",
Expand Down
16 changes: 16 additions & 0 deletions src/server/management/provider-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import {
hasOwnProvider,
isValidProviderName,
multiAgentGuidanceEnabled,
nonBlankStringArrayConfigError,
normalizeNonBlankStringArray,
providerBaseUrlConfigError,
providerHeadersConfigError,
saveConfigPreservingClaudeCode,
Expand Down Expand Up @@ -205,6 +207,19 @@ function applyProviderPatchFields(
}
touched = true;
}
if (Object.hasOwn(rawBody, "noStructuredOutputModels")) {
const value = rawBody.noStructuredOutputModels;
if (value === null) {
delete next.noStructuredOutputModels;
} else {
const error = nonBlankStringArrayConfigError(value, "noStructuredOutputModels");
if (error) return { error };
const models = normalizeNonBlankStringArray(value as string[]);
if (models.length > 0) next.noStructuredOutputModels = models;
else delete next.noStructuredOutputModels;
}
touched = true;
}

// headers is the one object-valued field in the mask. PATCH semantics merge it
// shallowly into the existing block so a single fingerprint header can be added
Expand Down Expand Up @@ -291,6 +306,7 @@ export async function handleProviderRoutes(ctx: ManagementContext): Promise<Resp
models: p.models ?? [],
contextWindow: p.contextWindow,
modelContextWindows: p.modelContextWindows,
noStructuredOutputModels: p.noStructuredOutputModels,
authMode: p.authMode,
apiKeyTransport: p.apiKeyTransport,
disabled: p.disabled === true,
Expand Down
6 changes: 6 additions & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1393,6 +1393,12 @@ export interface OcxProviderConfig {
noTopPModels?: string[];
/** Model ids that reject caller-specified presence/frequency penalty values. */
noPenaltyModels?: string[];
/**
* Model ids whose Chat Completions endpoint rejects `response_format`.
* Structured-output translation remains enabled by default; this is a narrow
* per-model compatibility escape hatch for mixed-capability gateways.
*/
noStructuredOutputModels?: string[];
/**
* Allow multiple tool calls per completion. DEFAULT-ON for openai-chat providers (the
* buffered stream parser assembles interleaved/fragmented multi-call turns safely);
Expand Down
18 changes: 18 additions & 0 deletions structure/04_transports-and-sidecars.md
Original file line number Diff line number Diff line change
Expand Up @@ -528,6 +528,24 @@ adapters advertise the catalog bit only on explicit `true`; cursor keeps its own
Providers with flaky parallel streaming can be opted out individually. Evidence and provider
ledger: `devlog/_fin/260709_parallel_tool_calls/`.

## Chat structured-output compatibility

The `openai-chat` adapter translates Responses `text.format` and Chat Completions
`response_format` through one internal format, then emits `response_format` on the upstream chat
wire. That remains the default because silently returning prose breaks clients that requested a
JSON object or schema. A mixed-capability gateway may list exact native model ids in
`noStructuredOutputModels`; only those models omit the wire field, while siblings keep the normal
translation. The proxy does not infer this from provider names, localhost destinations, or a model
family shared by unrelated upstreams.

[Decision Log]
- 목적과 의도: Recover chat models that reject `response_format` without removing structured output from models that support it.
- 기존 구현 및 제약 조건: The adapter forwarded the field to every routed chat model after #1137, while the same model id may sit behind gateways with different capabilities.
- 검토한 주요 대안: Revert translation globally; blacklist a model id globally; detect a proxy by name or URL; add an explicit provider/model opt-out.
- 선택한 방식: Preserve default translation and omit it only for exact ids in `noStructuredOutputModels`.
- 다른 대안 대신 이 방식을 선택한 이유: Global or heuristic rules regress supported providers and make custom gateway names part of the wire contract.
- 장점, 단점 및 영향: Compatible siblings retain schema enforcement and explicitly incompatible models avoid the upstream 400; operators must classify each unsupported model they route.

## Reasoning display parity (hideThinkingSummary)

`hideThinkingSummary` (request reasoning summary absent/"none" — the routed catalog default) is
Expand Down
Loading
Loading