Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 19 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -521,13 +521,15 @@ For more information on support for the Azure API, see [azure.md](azure.md).

## Amazon Bedrock

To use this library with [Amazon Bedrock's OpenAI-compatible API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html), use the `BedrockOpenAI` class instead of the `OpenAI` class.
To use this library with [Amazon Bedrock's OpenAI-compatible API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html), configure the standard `OpenAI` client with the Bedrock provider:

```ts
import { BedrockOpenAI } from 'openai';
import OpenAI from 'openai';
import { bedrock } from 'openai/providers/bedrock/aws';

// gets the bearer token from AWS_BEARER_TOKEN_BEDROCK and the region from AWS_REGION/AWS_DEFAULT_REGION
const client = new BedrockOpenAI();
const client = new OpenAI({
provider: bedrock({ region: 'us-west-2' }),
});

const response = await client.responses.create({
model: 'openai.gpt-5.4',
Expand All @@ -537,19 +539,24 @@ const response = await client.responses.create({
console.log(response.output_text);
```

`BedrockOpenAI` configures AWS bearer auth and the Bedrock Mantle endpoint, then uses the normal SDK resources. AWS controls which endpoints and features are supported; unsupported calls surface the provider's normal HTTP errors through the SDK.
Use a model that [supports the Responses API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html). A model returned by the Models API may support a different Bedrock inference API instead.

Pass `baseURL` or set `AWS_BEDROCK_BASE_URL` to override the derived `https://bedrock-mantle.<region>.api.aws/openai/v1` endpoint. For long-running apps, pass `bedrockTokenProvider` to refresh the Bedrock bearer token before each request.
This uses the regional `https://bedrock-mantle.<region>.api.aws/openai/v1` endpoint. The region can also come from `AWS_REGION` or `AWS_DEFAULT_REGION`, and `AWS_BEDROCK_BASE_URL` can override the endpoint.

Set `AWS_BEARER_TOKEN_BEDROCK` to an [Amazon Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html). To refresh tokens yourself, pass a provider instead of `apiKey`:
The AWS entrypoint uses the standard AWS credential chain by default. It also accepts a named profile, static credentials, or a custom credential provider. Install its peer dependencies before importing it:

```ts
const client = new BedrockOpenAI({
awsRegion: 'us-west-2',
bedrockTokenProvider: async () => refreshBedrockToken(),
});
```bash
npm install @aws-sdk/credential-provider-node @smithy/hash-node @smithy/signature-v4
```

The AWS entrypoint uses normal static imports so bundlers and serverless packagers can trace these dependencies. If one is missing, importing `openai/providers/bedrock/aws` fails immediately with the runtime's normal module-not-found error, for example:

```text
Cannot find module '@aws-sdk/credential-provider-node'
```

For Bedrock API key authentication, import `bedrock` from `openai/providers/bedrock` instead. That entrypoint has no AWS dependencies and works in browser-compatible runtimes when `dangerouslyAllowBrowser` is enabled. SigV4 authentication is supported in Node.js and compatible server runtimes and requires replayable request bodies. The legacy, bearer-only `BedrockOpenAI` class remains available for compatibility.

For more information on support for Amazon Bedrock, see [bedrock.md](bedrock.md).

## Advanced Usage
Expand Down
146 changes: 137 additions & 9 deletions bedrock.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
# Amazon Bedrock

To use this library with [Amazon Bedrock's OpenAI-compatible API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html), use the `BedrockOpenAI`
class instead of the `OpenAI` class.
To use this library with [Amazon Bedrock's OpenAI-compatible API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html), configure the standard `OpenAI` client with the Bedrock provider:

```ts
import { BedrockOpenAI } from 'openai';
import OpenAI from 'openai';
import { bedrock } from 'openai/providers/bedrock/aws';

// gets the bearer token from AWS_BEARER_TOKEN_BEDROCK and the region from AWS_REGION/AWS_DEFAULT_REGION
const client = new BedrockOpenAI();
const client = new OpenAI({
provider: bedrock({ region: 'us-west-2' }),
});

const response = await client.responses.create({
model: 'openai.gpt-5.4',
Expand All @@ -17,15 +18,142 @@ const response = await client.responses.create({
console.log(response.output_text);
```

`BedrockOpenAI` configures AWS bearer auth and the Bedrock Mantle endpoint, then uses the normal SDK resources. AWS controls which endpoints and features are supported; unsupported calls surface the provider's normal HTTP errors through the SDK.
Use a model that [supports the Responses API](https://docs.aws.amazon.com/bedrock/latest/userguide/models-api-compatibility.html). A model returned by the Models API may support a different Bedrock inference API instead.

The provider uses the regional `https://bedrock-mantle.<region>.api.aws/openai/v1` endpoint and the normal SDK resources. AWS controls which endpoints and features are supported; unsupported calls surface the provider's normal HTTP errors through the SDK.

The region defaults to `AWS_REGION` or `AWS_DEFAULT_REGION`. Pass `baseURL` or set `AWS_BEDROCK_BASE_URL` to override the derived endpoint:

```ts
const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
baseURL: 'https://bedrock.example.com/openai/v1',
}),
});
```

## Authentication

The AWS entrypoint selects authentication in this order:

1. One explicit mode passed to `bedrock(...)`: `apiKey` or `tokenProvider`, static AWS credentials, `profile`, or `credentialProvider`.
2. The [Amazon Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) in `AWS_BEARER_TOKEN_BEDROCK`.
3. The default AWS credential chain.

Explicit bearer and AWS credential modes are mutually exclusive. Similarly, configure only one AWS credential mode at a time.

### Bearer authentication

Pass a Bedrock API key directly, set `AWS_BEARER_TOKEN_BEDROCK`, or use `tokenProvider` to resolve a fresh token before every request attempt:

```ts
const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
apiKey: process.env['BEDROCK_API_KEY'],
}),
});
```

For a refreshable bearer credential:

Pass `baseURL` or set `AWS_BEDROCK_BASE_URL` to override the derived `https://bedrock-mantle.<region>.api.aws/openai/v1` endpoint. For long-running apps, pass `bedrockTokenProvider` to refresh the Bedrock bearer token before each request.
```ts
const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
tokenProvider: async () => refreshBedrockToken(),
}),
});
```

Set `AWS_BEARER_TOKEN_BEDROCK` to an [Amazon Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html). To refresh tokens yourself, pass a provider instead of `apiKey`:
Bearer authentication does not require any additional dependencies when imported from the dependency-free entrypoint:

```ts
import { bedrock } from 'openai/providers/bedrock';

const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
apiKey: process.env['AWS_BEARER_TOKEN_BEDROCK'],
}),
});
```

The dependency-free entrypoint supports only `apiKey`, `tokenProvider`, and `AWS_BEARER_TOKEN_BEDROCK`. Use the AWS entrypoint for SigV4 authentication.

### AWS credentials and SigV4

Install the AWS entrypoint's peer dependencies to sign requests with SigV4:

```sh
npm install @aws-sdk/credential-provider-node @smithy/hash-node @smithy/signature-v4
```

The AWS entrypoint uses normal static imports so Vite, Webpack, and serverless packagers can include these dependencies. If one is missing, importing `openai/providers/bedrock/aws` fails immediately with the runtime's normal module-not-found error, for example:

```text
Cannot find module '@aws-sdk/credential-provider-node'
```

Import the AWS entrypoint, then omit explicit authentication to use the default AWS credential chain or select a shared-config profile:

```ts
import { bedrock } from 'openai/providers/bedrock/aws';

const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
profile: 'my-profile',
}),
});
```

Pass temporary AWS credentials directly, including the session token:

```ts
const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
accessKeyId: process.env['AWS_ACCESS_KEY_ID'],
secretAccessKey: process.env['AWS_SECRET_ACCESS_KEY'],
sessionToken: process.env['AWS_SESSION_TOKEN'],
}),
});
```

For credentials that can change, pass a provider. It is called before every request attempt, including retries:

```ts
const client = new OpenAI({
provider: bedrock({
region: 'us-west-2',
credentialProvider: async () => ({
accessKeyId: process.env['AWS_ACCESS_KEY_ID']!,
secretAccessKey: process.env['AWS_SECRET_ACCESS_KEY']!,
sessionToken: process.env['AWS_SESSION_TOKEN'],
}),
}),
});
```

SigV4 authentication is supported in Node.js and compatible server runtimes. Bearer authentication can be used in other runtimes without loading the AWS packages by importing from `openai/providers/bedrock`.

The SDK's current SigV4 mode requires a replayable, buffered body such as a string, `ArrayBuffer`, or typed-array view. The standard JSON API methods already meet this requirement. Custom `FormData`, readable streams, and other non-replayable request bodies are rejected before sending; response streaming is unaffected. Signed requests also do not automatically follow redirects, because the redirect target would require a new signature.

Bedrock Mantle also supports `UNSIGNED-PAYLOAD` and AWS-chunked request signing, but this SDK does not enable those modes. Mantle waits for the complete request body before authentication and authorization, so streaming a request body does not reduce request latency.

## Legacy `BedrockOpenAI` class

The `BedrockOpenAI` class remains available for existing bearer-authenticated applications. It accepts the `awsRegion` and `bedrockTokenProvider` option names and uses the same `/openai/v1` endpoint as the provider:

```ts
import { BedrockOpenAI } from 'openai';

const client = new BedrockOpenAI({
awsRegion: 'us-west-2',
bedrockTokenProvider: async () => refreshBedrockToken(),
apiKey: process.env['AWS_BEARER_TOKEN_BEDROCK'],
});
```

New applications using AWS credentials should prefer `new OpenAI({ provider: bedrock(...) })` with the `openai/providers/bedrock/aws` entrypoint.
17 changes: 17 additions & 0 deletions ecosystem-tests/ts-browser-webpack/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import OpenAI, { toFile } from 'openai';
import { distance } from 'fastest-levenshtein';
import { bedrock } from 'openai/providers/bedrock';
import { ChatCompletion } from 'openai/resources/chat/completions';

type TestCase = {
Expand Down Expand Up @@ -96,6 +97,22 @@ const params = new URLSearchParams(location.search);

const client = new OpenAI({ apiKey: params.get('apiKey') ?? undefined, dangerouslyAllowBrowser: true });

it('supports Bedrock bearer authentication without AWS dependencies', async function () {
let authorization: string | null = null;
const bedrockClient = new OpenAI({
provider: bedrock({ region: 'us-east-1', apiKey: 'bedrock-token' }),
dangerouslyAllowBrowser: true,
fetch: async (_url, init) => {
authorization = new Headers(init?.headers).get('authorization');
return new Response('{}', { headers: { 'Content-Type': 'application/json' } });
},
});

await bedrockClient.request({ method: 'get', path: '/models' });

expect(authorization).toEqual('Bearer bedrock-token');
});

async function typeTests() {
// @ts-expect-error this should error if the `Uploadable` type was resolved correctly
await client.audio.transcriptions.create({ file: { foo: true }, model: 'whisper-1' });
Expand Down
13 changes: 9 additions & 4 deletions examples/bedrock/responses.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
#!/usr/bin/env -S npm run tsn -- -T

import { BedrockOpenAI } from 'openai';
import OpenAI from 'openai';
import { bedrock } from 'openai/providers/bedrock/aws';

const client = new BedrockOpenAI();
const client = new OpenAI({
provider: bedrock({ region: 'us-west-2' }),
});

// For refreshed Bedrock bearer tokens:
// const client = new BedrockOpenAI({ awsRegion: 'us-west-2', bedrockTokenProvider: getBedrockToken });
// For refreshed Bedrock bearer tokens, import from 'openai/providers/bedrock':
// const client = new OpenAI({
// provider: bedrock({ region: 'us-west-2', tokenProvider: getBedrockToken }),
// });

async function main() {
const response = await client.responses.create({
Expand Down
2 changes: 1 addition & 1 deletion jest.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const config: JestConfigWithTsJest = {
'<rootDir>/deno_tests/',
'<rootDir>/packages/',
],
testPathIgnorePatterns: ['scripts'],
testPathIgnorePatterns: ['scripts', '<rootDir>/tests/live/'],
// prettierPath: require.resolve('prettier-2'),
};

Expand Down
11 changes: 11 additions & 0 deletions jest.live.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import type { JestConfigWithTsJest } from 'ts-jest';

import baseConfig from './jest.config';

const config: JestConfigWithTsJest = {
...baseConfig,
testMatch: ['<rootDir>/tests/live/**/*.live.test.ts'],
testPathIgnorePatterns: [],
};

export default config;
5 changes: 5 additions & 0 deletions jsr.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,15 @@
"version": "6.44.0",
"exports": {
".": "./index.ts",
"./providers/bedrock": "./providers/bedrock.ts",
"./providers/bedrock/aws": "./providers/bedrock/aws.ts",
"./helpers/zod": "./helpers/zod.ts",
"./beta/realtime/websocket": "./beta/realtime/websocket.ts"
},
"imports": {
"@aws-sdk/credential-provider-node": "npm:@aws-sdk/credential-provider-node@^3.972.47",
"@smithy/hash-node": "npm:@smithy/hash-node@^4.3.5",
"@smithy/signature-v4": "npm:@smithy/signature-v4@^5.4.5",
"zod": "npm:zod@3"
},
"publish": {
Expand Down
17 changes: 17 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
},
"scripts": {
"test": "./scripts/test",
"test:live:bedrock": "jest --config jest.live.config.ts --runInBand tests/live/bedrock.live.test.ts",
"build": "./scripts/build",
"prepublishOnly": "echo 'to publish, run pnpm build && (cd dist; npm publish)' && exit 1",
"format": "./scripts/format",
Expand All @@ -28,6 +29,10 @@
},
"devDependencies": {
"@arethetypeswrong/cli": "^0.18.3",
"@aws-sdk/credential-provider-node": "^3.972.47",
"@smithy/hash-node": "^4.3.5",
"@smithy/protocol-http": "^5.4.5",
"@smithy/signature-v4": "^5.4.5",
"@swc/core": "^1.3.102",
"@swc/jest": "^0.2.29",
"@types/jest": "^29.4.0",
Expand Down Expand Up @@ -80,10 +85,22 @@
}
},
"peerDependencies": {
"@aws-sdk/credential-provider-node": ">=3.972.0 <4",
"@smithy/hash-node": ">=4.3.0 <5",
"@smithy/signature-v4": ">=5.4.0 <6",
"ws": "^8.18.0",
"zod": "^3.25 || ^4.0"
},
"peerDependenciesMeta": {
"@aws-sdk/credential-provider-node": {
"optional": true
},
"@smithy/hash-node": {
"optional": true
},
"@smithy/signature-v4": {
"optional": true
},
"ws": {
"optional": true
},
Expand Down
Loading
Loading