Support tracing the OpenAI Codex SDK

[jonahsnider]

OpenAI has an SDK specifically for using Codex programmatically: @openai/codex-sdk for JS and an experimental Python implementation.

It'd be nice if the Braintrust SDK was able to wrap the Codex SDK and provide tracing, the same way it currently supports the OpenAI Agents SDK.

See also: https://developers.openai.com/codex/sdk

[AbhiPrasad]

thanks for the issue Jonah Snider (@jonahsnider)! we'll backlog this, but this is definitely something we'll look to do!

[AbhiPrasad]

Note: I asked gpt5.5 to generate a summary for what we should look to implement. The following is some LLM slop:

I did a pass over the current @openai/codex-sdk TypeScript SDK and our instrumentation layout. This is the shape I think we should target.

Summary

@openai/codex-sdk is an ESM Node 18+ SDK that wraps/spawns the codex CLI and communicates with it via JSONL events over stdin/stdout. The public API is small but agent-shaped, and is closer to our Claude Agent SDK / OpenRouter Agent instrumentation than to wrapOpenAI() chat completions.

Users create a Codex, start or resume a Thread, then run one or more turns:

import { Codex } from "@openai/codex-sdk";

const codex = new Codex();
const thread = codex.startThread({ workingDirectory: process.cwd() });

const turn = await thread.run("Diagnose the test failure and propose a fix");
console.log(turn.finalResponse, turn.usage);

const { events } = await thread.runStreamed("Implement the fix");
for await (const event of events) {
  console.log(event);
}

SDK surfaces to instrument

SDK surface	What it does	Suggested tracing behavior
new Codex(options)	Configures CLI path/env/base URL/API key/global config	Preserve behavior; optionally capture safe option metadata on later spans
codex.startThread(options)	Creates a local Thread handle; thread id is unknown until first thread.started event	Return wrapped Thread whose methods are instrumented
codex.resumeThread(id, options)	Reattaches to a persisted ~/.codex/sessions thread	Same as above, with codex.thread_id known up front
thread.run(input, turnOptions)	Buffers stream until terminal turn; returns { items, finalResponse, usage } or throws on turn.failed	One Braintrust span per turn
thread.runStreamed(input, turnOptions)	Returns { events: AsyncGenerator<ThreadEvent> }	One Braintrust span that stays open until the returned generator is consumed/throws/returns
turnOptions.signal	Cancels a turn	Preserve cancellation; mark span as cancelled/error if stream aborts
turnOptions.outputSchema	Requests structured JSON output	Record that structured output was requested; do not dump huge schemas by default

Input can be either a string or an array of { type: "text", text } and { type: "local_image", path }. For image inputs, trace text plus image path/count metadata, not image bytes.

Events/items to map

The stream currently exposes these top-level events:

• thread.started with thread_id
• turn.started
• turn.completed with usage
• turn.failed with { message }
• item.started / item.updated / item.completed with a ThreadItem
• error with { message }

Item types worth handling explicitly:

• agent_message: final/assistant text
• reasoning: reasoning summary text
• command_execution: command, aggregated output, exit code, status
• file_change: changed paths and add/delete/update kinds, status
• mcp_tool_call: MCP server/tool, arguments, result/error, status
• web_search: query
• todo_list: agent plan / checklist
• error: non-fatal item error

Desired trace shape

Suggested top-level span name: Codex Turn or OpenAI Codex Turn.

Top-level span fields/metadata:

• provider: openai
• sdk: @openai/codex-sdk
• codex.thread_id from resumeThread() or from thread.started
• codex.operation: run or runStreamed
• codex.model, codex.model_reasoning_effort
• codex.working_directory
• codex.sandbox_mode
• codex.approval_policy
• codex.web_search_mode / codex.web_search_enabled
• codex.network_access_enabled
• codex.additional_directories count/list as appropriate
• codex.structured_output: boolean
• codex.image_count

Input/output/metrics:

• input: normalized prompt text; include safe image path metadata separately
• output: finalResponse for run(), or final agent_message.text observed in the stream for runStreamed()
• metrics from turn.completed.usage:
  • input_tokens
  • cached_input_tokens
  • output_tokens
  • reasoning_output_tokens

Child spans/events to consider:

• command_execution child span keyed by item id; start on item.started, update/finish on item.completed; include command, exit code, status, and maybe truncated output
• mcp_tool_call child span keyed by item id; include server, tool, arguments/result/error subject to normal input/output logging controls
• web_search child span/event with query
• file_change event/span with changed path list and patch status
• reasoning, todo_list, and agent_message as logged events or structured metadata on the turn span
• turn.failed / top-level error should mark the turn span as errored and preserve SDK error propagation

For runStreamed(), the wrapper needs to wrap the returned async generator and publish/process events as the user iterates it, rather than closing the span when runStreamed() returns. It should also close/cleanup if the consumer breaks early or the generator throws.

Implementation approach

This likely wants the same architecture as Claude Agent SDK support:

• add a wrapper, e.g. wrapCodexSDK() / wrapOpenAICodexSDK()
• add diagnostics-channel definitions for Codex turn lifecycle/events
• add a plugin under js/src/instrumentation/plugins/ that owns span lifecycle and stream event aggregation
• add an auto-instrumentation config for @openai/codex-sdk
• vendor lightweight Codex SDK types under js/src/vendor-sdk-types/ instead of taking a runtime dependency
• patch/wrap the exported Codex class so startThread() and resumeThread() return instrumented Thread instances, and Thread.run() / Thread.runStreamed() are wrapped

Suggested manual API:

import * as codexSDK from "@openai/codex-sdk";
import { wrapCodexSDK } from "braintrust";

const { Codex } = wrapCodexSDK(codexSDK);
const codex = new Codex();
const thread = codex.startThread();
await thread.run("Summarize this repository");

Auto-instrumentation should also cover direct imports:

import { Codex } from "@openai/codex-sdk";

const codex = new Codex();
await codex.startThread().run("Summarize this repository");

Testing suggestions

• Unit tests with a mocked Codex SDK module for:
  • new Codex().startThread().run() success
  • resumeThread(id).run() preserves thread id
  • runStreamed() consumes events and closes span only after generator completion
  • early break from for await closes span/child spans
  • turn.failed, top-level error, thrown JSON/stream errors, and abort signal behavior
  • command execution, MCP tool call, file change, web search, reasoning, todo list, and final agent message event mapping
  • multiple consecutive turns on one Thread produce separate spans with shared thread_id
• Auto-instrumentation tests for ESM import of @openai/codex-sdk
• E2E scenario with a mocked @openai/codex-sdk package or mocked CLI JSONL stream; real Codex calls are probably unsuitable for CI because the SDK shells out to the Codex CLI and needs credentials/local repo state

References / files inspected

Upstream:

• https://github.com/openai/codex/tree/main/sdk/typescript
• sdk/typescript/README.md
• sdk/typescript/src/codex.ts
• sdk/typescript/src/thread.ts
• sdk/typescript/src/events.ts
• sdk/typescript/src/items.ts
• sdk/typescript/src/threadOptions.ts
• sdk/typescript/src/turnOptions.ts

Braintrust repo currently has no Codex-specific support: no wrapper, plugin/channels, auto-instrumentation config, vendor SDK types, examples, or e2e scenario for @openai/codex-sdk / Codex.