Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ Adapters can be set at **provider level** (applied to every model under the prov
|---------|-------------|
| `reasoning_content` | Renames `reasoning` / `thinking.content` → `reasoning_content` on outbound assistant messages for providers that use Fireworks/DeepSeek field naming (e.g. Fireworks DeepSeek-R1). Fixes *"Extra inputs are not permitted, field: messages[N].reasoning"* errors. |
| `suppress_developer_role` | Rewrites the `developer` role to `system` on outbound messages for providers that do not support the newer OpenAI `developer` role. |
| `model_override` | Conditionally rewrites the provider model name based on request payload fields. Used for providers that expose reasoning variants as separate model names rather than respecting reasoning-related fields in the request body. See [Model Override Adapter](#model-override-adapter) below. |

**Example — provider-level:**
```json
Expand All @@ -164,6 +165,66 @@ PUT /v0/management/providers/fireworks

Adapters are applied in order on outbound (preDispatch) and in reverse on inbound (postDispatch). Pass-through optimisation is automatically disabled when any adapter is active.

### Model Override Adapter

The `model_override` adapter conditionally rewrites the provider model name based on the values or presence of fields in the request payload. This is useful for providers that expose reasoning variants as **separate model names** (e.g. `model-name` with reasoning, `model-name-fast` without) rather than respecting reasoning-related fields in the request body.

**How it works:**

When the resolved provider model matches a rule's `model` field AND **any** of the rule's conditions are satisfied (OR semantics), the model name is rewritten to `rewriteTo`. Conditions use dotted paths into the request payload.

**Configuration:**

The `model_override` adapter is configured at **model level** only (not provider level). It accepts a `rules` array in its options:

```json
{
"models": {
"zai-org/GLM-5.1-FP8": {
"adapter": [
{
"name": "model_override",
"options": {
"rules": [
{
"model": "zai-org/GLM-5.1-FP8",
"rewriteTo": "glm-5.1-fast",
"conditions": [
{ "field": "enable_thinking", "value": false },
{ "field": "reasoning.enabled", "value": false },
{ "field": "reasoning.effort", "value": "none" },
{ "field": "budget_tokens", "value": 0 }
]
}
]
}
}
]
}
}
}
```

**Rule fields:**

| Field | Description |
|-------|-------------|
| `model` | The provider model name to match against (must match the resolved target model) |
| `rewriteTo` | The model name to send to the provider instead |
| `conditions` | Array of conditions; **any** match triggers the rewrite |

**Condition fields:**

| Field | Description |
|-------|-------------|
| `field` | Dotted path into the request payload (e.g. `reasoning.enabled`, `chat_template_kwargs.enable_thinking`) |
| `value` | Value to match (strict equality). If omitted, the condition matches when the field is present (any value) |

**Notes:**
- The adapter operates on the **transformed provider payload**, so fields must survive API transformation to be matchable. For chat-to-chat requests, all fields from the original request body are preserved (including non-standard fields like `enable_thinking`, `thinking_budget`, etc.).
- Multiple rules are evaluated in order; only the first matching rule applies.
- The rewrite is transparent to the client — billing and usage tracking still reference the original canonical model name.

### Provider Quota Checkers

Quota checkers monitor upstream provider rate limits and prevent routing to exhausted providers.
Expand Down
44 changes: 40 additions & 4 deletions docs/openapi/components/schemas/ProviderConfig.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -109,11 +109,25 @@ properties:
- type: string
- type: array
items:
type: string
oneOf:
- type: string
- type: object
required:
- name
properties:
name:
type: string
description: Adapter name.
options:
type: object
additionalProperties: true
description: Adapter-specific options.
description: >
Adapter name(s) applied to this specific model. Appended after
any provider-level adapters. See the provider-level `adapter`
field for available values.
any provider-level adapters. Accepts bare strings or
`{ name, options }` objects for parameterized adapters
(e.g. `model_override` with rules). See the provider-level
`adapter` field for available values.
description: >
Map of model ID → per-model configuration. Allows per-model pricing
overrides and access aliases.
Expand Down Expand Up @@ -176,13 +190,28 @@ properties:
- type: string
- type: array
items:
type: string
oneOf:
- type: string
- type: object
required:
- name
properties:
name:
type: string
description: Adapter name.
options:
type: object
additionalProperties: true
description: Adapter-specific options.
description: >
Adapter name(s) applied to every model under this provider.
Adapters rewrite outbound request payloads (preDispatch) and inbound
provider responses (postDispatch) to fix provider-specific field-name
incompatibilities. Applied before model-level adapters.

Adapters can be specified as bare strings (backward compatible) or as
objects with `{ name, options }` for adapters that require configuration.

Available adapters:

- `reasoning_content` — Renames `reasoning`/`thinking.content` →
Expand All @@ -192,6 +221,13 @@ properties:
- `suppress_developer_role` — Rewrites the `developer` role to `system`
for providers that do not support the OpenAI `developer` role.

- `model_override` — Conditionally rewrites the provider model name
based on request payload fields. Accepts a `rules` array in options
where each rule specifies a `model` to match, a `rewriteTo` model name,
and an array of `conditions` (OR semantics) using dotted field paths.
Used for providers that expose reasoning variants as separate model
names rather than respecting reasoning-related request fields.

Pass-through optimisation is automatically disabled when any adapter
is active.
timeoutMs:
Expand Down
101 changes: 68 additions & 33 deletions packages/backend/src/__tests__/adapters/adapter-resolver.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ import { resolveAdapters } from '../../services/adapter-resolver';
import type { RouteResult } from '../../services/router';

// Minimal RouteResult factory
function makeRoute(
providerAdapter?: string | string[],
modelAdapter?: string | string[]
): RouteResult {
function makeRoute(providerAdapter?: any[], modelAdapter?: any[]): RouteResult {
return {
provider: 'test-provider',
model: 'test-model',
Expand All @@ -29,48 +26,86 @@ describe('resolveAdapters', () => {
expect(resolveAdapters(route)).toHaveLength(0);
});

it('resolves a provider-level string adapter', () => {
const route = makeRoute('reasoning_content');
const adapters = resolveAdapters(route);
expect(adapters).toHaveLength(1);
expect(adapters[0]!.name).toBe('reasoning_content');
it('resolves a provider-level adapter entry', () => {
const route = makeRoute([{ name: 'reasoning_content', options: {} }]);
const resolved = resolveAdapters(route);
expect(resolved).toHaveLength(1);
expect(resolved[0]!.adapter.name).toBe('reasoning_content');
expect(resolved[0]!.options).toEqual({});
});

it('resolves a provider-level array adapter', () => {
const route = makeRoute(['reasoning_content', 'suppress_developer_role']);
const adapters = resolveAdapters(route);
expect(adapters.map((a) => a.name)).toEqual(['reasoning_content', 'suppress_developer_role']);
it('resolves a model-level adapter entry', () => {
const route = makeRoute(undefined, [{ name: 'suppress_developer_role', options: {} }]);
const resolved = resolveAdapters(route);
expect(resolved).toHaveLength(1);
expect(resolved[0]!.adapter.name).toBe('suppress_developer_role');
expect(resolved[0]!.options).toEqual({});
});

it('resolves a model-level string adapter', () => {
const route = makeRoute(undefined, 'suppress_developer_role');
const adapters = resolveAdapters(route);
expect(adapters).toHaveLength(1);
expect(adapters[0]!.name).toBe('suppress_developer_role');
it('merges provider-level then model-level adapters in order', () => {
const route = makeRoute(
[{ name: 'reasoning_content', options: {} }],
[{ name: 'suppress_developer_role', options: {} }]
);
const resolved = resolveAdapters(route);
expect(resolved.map((r) => r.adapter.name)).toEqual([
'reasoning_content',
'suppress_developer_role',
]);
});

it('merges provider-level then model-level adapters in order', () => {
const route = makeRoute('reasoning_content', 'suppress_developer_role');
const adapters = resolveAdapters(route);
expect(adapters.map((a) => a.name)).toEqual(['reasoning_content', 'suppress_developer_role']);
it('passes options through from config', () => {
const rules = [
{
model: 'deepseek-r1',
rewriteTo: 'deepseek-r1-fast',
conditions: [{ field: 'reasoning.enabled', value: false }],
},
];
const route = makeRoute([{ name: 'model_override', options: { rules } }]);
const resolved = resolveAdapters(route);
expect(resolved).toHaveLength(1);
expect(resolved[0]!.adapter.name).toBe('model_override');
expect(resolved[0]!.options).toEqual({ rules });
});

it('skips and warns on unknown adapter names (does not throw)', () => {
const route = makeRoute('nonexistent_adapter');
// Should not throw; unknown names are skipped
const adapters = resolveAdapters(route);
expect(adapters).toHaveLength(0);
const route = makeRoute([{ name: 'nonexistent_adapter', options: {} }]);
const resolved = resolveAdapters(route);
expect(resolved).toHaveLength(0);
});

it('handles mixed valid and invalid adapter names', () => {
const route = makeRoute(['reasoning_content', 'bogus'], 'suppress_developer_role');
const adapters = resolveAdapters(route);
expect(adapters.map((a) => a.name)).toEqual(['reasoning_content', 'suppress_developer_role']);
const route = makeRoute(
[
{ name: 'reasoning_content', options: {} },
{ name: 'bogus', options: {} },
],
[{ name: 'suppress_developer_role', options: {} }]
);
const resolved = resolveAdapters(route);
expect(resolved.map((r) => r.adapter.name)).toEqual([
'reasoning_content',
'suppress_developer_role',
]);
});

it('handles multiple provider-level adapter entries', () => {
const route = makeRoute([
{ name: 'reasoning_content', options: {} },
{ name: 'suppress_developer_role', options: {} },
]);
const resolved = resolveAdapters(route);
expect(resolved.map((r) => r.adapter.name)).toEqual([
'reasoning_content',
'suppress_developer_role',
]);
});

it('handles model-level array adapters', () => {
const route = makeRoute(undefined, ['reasoning_content', 'suppress_developer_role']);
const adapters = resolveAdapters(route);
expect(adapters.map((a) => a.name)).toEqual(['reasoning_content', 'suppress_developer_role']);
it('resolves model_override adapter', () => {
const route = makeRoute([{ name: 'model_override', options: { rules: [] } }]);
const resolved = resolveAdapters(route);
expect(resolved).toHaveLength(1);
expect(resolved[0]!.adapter.name).toBe('model_override');
});
});
56 changes: 55 additions & 1 deletion packages/backend/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,61 @@ const PricingSchema = z.discriminatedUnion('source', [
}),
]);

const AdapterConfigSchema = z.union([z.string(), z.array(z.string())]).optional();
// ─── Adapter Config ─────────────────────────────────────────────────
// Adapters are configured as an array of { name, options } entries.
// Legacy bare-string forms are normalized at read time in config-repository.

const ModelOverrideConditionSchema = z.object({
/** JSON dotted path into the payload (e.g. "reasoning.enabled", "reasoning.effort"). */
field: z.string().min(1),
/** If omitted, matches when the field is present (any value). If set, matches when value equals this. */
value: z.any().optional(),
});

const ModelOverrideRuleSchema = z.object({
/** The model name in the payload to match against (e.g. "deepseek-r1"). */
model: z.string().min(1),
/** The model name to rewrite to when conditions match (e.g. "deepseek-r1-fast"). */
rewriteTo: z.string().min(1),
/** Conditions — ANY match triggers the rewrite (OR semantics). */
conditions: z.array(ModelOverrideConditionSchema).min(1),
});

const ModelOverrideOptionsSchema = z.object({
rules: z.array(ModelOverrideRuleSchema).min(1),
});

const AdapterEntrySchema = z.object({
name: z.string().min(1),
options: z.record(z.string(), z.any()).default({}),
});

/**
* Accepts both the legacy format (string | string[]) and the new
* uniform format ({ name, options }[]) and normalizes everything
* to AdapterEntry[]. This ensures backward compatibility with
* existing YAML configs while enforcing the canonical shape at
* validation time.
*/
const AdapterConfigSchema = z.preprocess((val) => {
if (val === undefined || val === null) return undefined;
// Already an array (or single entry) — normalize each element
const arr = Array.isArray(val) ? val : [val];
return arr.map((entry: any) => {
if (typeof entry === 'string') {
return { name: entry, options: {} };
}
if (entry && typeof entry === 'object' && 'name' in entry) {
return { name: entry.name, options: entry.options ?? {} };
}
return entry; // Let Zod produce a clear validation error
});
}, z.array(AdapterEntrySchema).optional());

export type ModelOverrideCondition = z.infer<typeof ModelOverrideConditionSchema>;
export type ModelOverrideRule = z.infer<typeof ModelOverrideRuleSchema>;
export type ModelOverrideOptions = z.infer<typeof ModelOverrideOptionsSchema>;
export type AdapterEntry = z.infer<typeof AdapterEntrySchema>;

const ModelProviderConfigSchema = z.object({
pricing: PricingSchema.default({
Expand Down
Loading