Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fix-gateway-caching-type.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@ai-sdk/gateway": patch
---

fix(provider/gateway): add missing `caching` property to GatewayProviderOptions type
8 changes: 8 additions & 0 deletions content/providers/01-ai-sdk-providers/00-ai-gateway.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -851,6 +851,14 @@ The following gateway provider options are available:

For full details, see [Provider Timeouts](https://vercel.com/docs/ai-gateway/models-and-providers/provider-timeouts).

- **caching** _'auto'_

Automatic prompt caching strategy. When set to `'auto'`, AI Gateway handles caching automatically based on the provider. For providers that require explicit cache markers (Anthropic, MiniMax), AI Gateway adds a `cache_control` breakpoint at the end of static content. For providers with implicit caching (OpenAI, Google, DeepSeek), no modification is needed.

When not set, requests pass through without modification.

For full details, see [Automatic Caching](https://vercel.com/docs/ai-gateway/models-and-providers/automatic-caching).

You can combine these options to have fine-grained control over routing and tracking:

```ts
Expand Down
26 changes: 26 additions & 0 deletions examples/ai-functions/src/stream-text/gateway/auto-caching.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import type { GatewayProviderOptions } from '@ai-sdk/gateway';
import { streamText } from 'ai';
import { run } from '../../lib/run';

run(async () => {
const result = streamText({
model: 'anthropic/claude-sonnet-4.6',
system:
'You are a helpful assistant with access to a large knowledge base. ' +
'Answer questions concisely and accurately.',
prompt: 'What is the capital of France?',
providerOptions: {
gateway: {
caching: 'auto',
} satisfies GatewayProviderOptions,
},
});

for await (const textPart of result.textStream) {
process.stdout.write(textPart);
}

console.log();
console.log('Token usage:', await result.usage);
console.log('Finish reason:', await result.finishReason);
});
27 changes: 27 additions & 0 deletions packages/gateway/src/gateway-provider-options.test-d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { describe, it } from 'vitest';
import type { GatewayProviderOptions } from './index';

describe('GatewayProviderOptions', () => {
it('should accept caching: auto', () => {
const options = {
caching: 'auto',
} satisfies GatewayProviderOptions;

options;
});

it('should make caching optional', () => {
const options = {} satisfies GatewayProviderOptions;

options;
});

it('should reject unknown caching values', () => {
const options = {
// @ts-expect-error 'manual' is not a valid caching value
caching: 'manual',
} satisfies GatewayProviderOptions;

options;
});
});
13 changes: 13 additions & 0 deletions packages/gateway/src/gateway-provider-options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,19 @@ const gatewayProviderOptions = lazySchema(() =>
byok: z.record(z.string(), z.number().int().min(1000)).optional(),
})
.optional(),
/**
* Automatic prompt caching strategy.
*
* - `'auto'`: Let AI Gateway handle caching automatically. For providers that
* require explicit cache markers (Anthropic, MiniMax), AI Gateway adds a
* `cache_control` breakpoint at the end of static content. For providers
* with implicit caching (OpenAI, Google, DeepSeek), no modification is needed.
*
* When not set, requests pass through without modification.
*
* See https://vercel.com/docs/ai-gateway/models-and-providers/automatic-caching
*/
caching: z.literal('auto').optional(),
}),
),
);
Expand Down