Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ This repository contains Braintrust's Python SDKs and integrations, including:

- The main `braintrust` SDK package in [`./py`](./py)
- Built-in integrations under [`py/src/braintrust/integrations`](py/src/braintrust/integrations) and related compatibility packages under [`./integrations`](./integrations)
- Examples, tests, and local development tooling for Python SDK development
- Self-contained `uv`-project examples for every integration in [`./examples`](./examples)
- Tests and local development tooling for Python SDK development

## Quickstart

Expand Down
8 changes: 8 additions & 0 deletions examples/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
run.sh

# Examples install fresh against current PyPI; locks aren't checked in.
uv.lock
.venv/
__pycache__/
*.pyc
.env
96 changes: 96 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Braintrust Python SDK Examples

Each subdirectory in this folder is a self-contained [`uv`](https://docs.astral.sh/uv/) project demonstrating one Braintrust integration or feature. They are designed to be cloned, copied, or run as-is without affecting the rest of the repository.

## Layout

Every example has the same shape:

```
examples/<name>/
├── pyproject.toml # declares deps + a local `path` source for braintrust
├── README.md # what it shows + how to run it
└── *.py # the example script(s)
```

The `pyproject.toml` in each example pins `braintrust` to the local SDK checkout in `py/` via:

```toml
[tool.uv.sources]
braintrust = { path = "../../py", editable = true }
```

That means `uv sync` inside any example installs the version of the SDK currently on disk — edits to `py/src/braintrust/` are picked up immediately.

## Running an example

From the repo root:

```bash
cd examples/<name>
uv sync
uv run python <script>.py
```

You will need a `BRAINTRUST_API_KEY` in your environment, plus whatever provider key the example uses (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). Each example's README lists its requirements.

## The default pattern: `auto_instrument()`

Most provider/framework examples follow the same two-line setup:

```python
import braintrust

braintrust.auto_instrument()
braintrust.init_logger(project="example-<name>")

# now import and use the library normally
```

`braintrust.auto_instrument()` patches every supported integration that's installed in the current environment — there's no need to call individual `setup_*()` or `wrap_*()` helpers. The exceptions are examples that demonstrate non-tracing concerns (`evals/`, `langsmith/`, `otel/`, `temporal/`).

If you want to see `auto_instrument()` light up multiple providers in one trace, the `openai/`, `anthropic/`, and any other provider example can be combined trivially — just install both packages in the same environment, call `auto_instrument()` once, and use both clients.

## Available examples

Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.

| Directory | What it shows |
| --- | --- |
| `adk/` | Google ADK weather agent with one tool |
| `agentscope/` | AgentScope `ReActAgent` against `gpt-4o-mini` |
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
| `anthropic/` | Sync and async Anthropic clients |
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
| `cohere/` | Cohere `ClientV2` chat call |
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
| `dspy/` | DSPy `ReAct` agent with two tools (LiteLLM token metrics propagate) |
| `evals/` | The `Eval` framework — does **not** use `auto_instrument()` |
| `google_genai/` | Google GenAI `generate_content` against Gemini |
| `langchain/` | LangChain `prompt | model` chain — global handler installed by `auto_instrument()` |
| `langsmith/` | Migration helper for projects coming from LangSmith — uses `setup_langsmith()` |
| `litellm/` | LiteLLM `completion` |
| `llamaindex/` | LlamaIndex LLM completion |
| `mistral/` | Mistral chat completion |
| `openai/` | OpenAI chat completion plus `@braintrust.traced` for the surrounding function |
| `openai_agents/` | OpenAI Agents SDK `Runner` running an agent |
| `openrouter/` | OpenRouter chat completion routed to OpenAI |
| `otel/` | OpenTelemetry interop — `BraintrustSpanProcessor`, filtering, distributed tracing |
| `pydantic_ai/` | Pydantic AI agent run inside a `start_span` for a permalink |
| `strands/` | Strands `Agent` against `gpt-4o-mini` |
| `temporal/` | Distributed Temporal workflow tracing via `BraintrustPlugin` |

## Why no committed `uv.lock`?

Examples are meant to demonstrate the **current** SDK against **current** provider releases, not snapshot a specific point in time. Committing lockfiles would create constant churn (a new lockfile for every transitive bump on PyPI) and would actively work against the demo goal — someone running an example next quarter wants it to work with whatever's on PyPI today, not a 3-month-old pin. Lockfiles are gitignored.

If you want a reproducible snapshot for your own use, run `uv lock` locally; it just won't be checked in.

## Adding a new example

1. Create `examples/<name>/` with the layout above.
2. In `pyproject.toml`, declare deps under `[project] dependencies` and add the local `braintrust` source under `[tool.uv.sources]` (see any existing example).
3. Write a short README explaining required env vars and how to run it.
4. Verify with `uv sync && uv run python <script>.py` from inside the example dir.
5. The `pylint` nox session in `py/noxfile.py` automatically lints `examples/`, so any imports your example needs must be in the `lint` dependency group of `py/pyproject.toml`.
13 changes: 13 additions & 0 deletions examples/adk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Google ADK + Braintrust

Calls `braintrust.auto_instrument()` so Google ADK is patched automatically, then runs a weather agent against `gemini-2.0-flash` with one tool. The trace shows the runner span, the agent span, the LLM span, and the tool call.

## Run

```bash
export BRAINTRUST_API_KEY=...
export GOOGLE_API_KEY=...

uv sync
uv run python example.py
```
10 changes: 2 additions & 8 deletions py/examples/adk/auto.py → examples/adk/example.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,12 @@
"""Auto-instrument Google ADK with Braintrust tracing.

Usage:
export BRAINTRUST_API_KEY="your-api-key"
export GOOGLE_API_KEY="your-google-api-key"
python auto.py
"""
"""Google ADK weather agent traced via braintrust.auto_instrument()."""

import asyncio

import braintrust


# Auto-instrument all supported libraries including Google ADK
braintrust.auto_instrument()
braintrust.init_logger(project="example-adk")

from google.adk import Agent
from google.adk.runners import Runner
Expand Down
13 changes: 13 additions & 0 deletions examples/adk/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[project]
name = "braintrust-adk-example"
version = "0.1.0"
description = "Google ADK agents traced via Braintrust"
requires-python = ">=3.10"
dependencies = [
"braintrust",
"google-adk",
"google-genai",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }
13 changes: 13 additions & 0 deletions examples/agentscope/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# AgentScope + Braintrust

Calls `braintrust.auto_instrument()` to register the AgentScope tracing hooks, then runs a `ReActAgent` against `gpt-4o-mini`. The trace shows the agent's `*.reply` task span and the underlying LLM span.

## Run

```bash
export BRAINTRUST_API_KEY=...
export OPENAI_API_KEY=...

uv sync
uv run python example.py
```
35 changes: 35 additions & 0 deletions examples/agentscope/example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env python
"""AgentScope ReAct agent traced via braintrust.auto_instrument()."""

import asyncio

import braintrust


braintrust.auto_instrument()
braintrust.init_logger(project="example-agentscope")

from agentscope.agent import ReActAgent
from agentscope.formatter import OpenAIChatFormatter
from agentscope.memory import InMemoryMemory
from agentscope.message import Msg
from agentscope.model import OpenAIChatModel
from agentscope.tool import Toolkit


async def main() -> None:
agent = ReActAgent(
name="Friday",
sys_prompt="You are a concise assistant. Answer in one sentence.",
model=OpenAIChatModel(model_name="gpt-4o-mini", stream=False),
formatter=OpenAIChatFormatter(),
toolkit=Toolkit(),
memory=InMemoryMemory(),
)

response = await agent(Msg(name="user", content="Say hello in exactly two words.", role="user"))
print(response.content if response is not None else "(no response)")


if __name__ == "__main__":
asyncio.run(main())
13 changes: 13 additions & 0 deletions examples/agentscope/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[project]
name = "braintrust-agentscope-example"
version = "0.1.0"
description = "AgentScope ReAct agent traced with Braintrust"
requires-python = ">=3.10"
dependencies = [
"braintrust",
"agentscope",
"openai",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }
21 changes: 21 additions & 0 deletions examples/agno/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Agno + Braintrust

Each script calls `braintrust.auto_instrument()` before importing `agno`, so all subsequent agent/team activity is traced. The agents use `YFinanceTools` so they actually exercise tool calls.

| Script | Shape |
| --- | --- |
| `simple_agent.py` | one agent, one question |
| `simple_agent_stream.py` | one agent, streamed |
| `async_simple_agent_stream.py` | one agent, async + streamed |
| `team_agent.py` | research + advisor team, sync |
| `async_team_agent.py` | research + advisor team, async + streamed |

## Run

```bash
export BRAINTRUST_API_KEY=...
export OPENAI_API_KEY=...

uv sync
uv run python simple_agent.py
```
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import asyncio

from braintrust.wrappers.agno import setup_agno
import braintrust


setup_agno(project_name="simple-agent-project")
braintrust.auto_instrument()
braintrust.init_logger(project="simple-agent-project")

from agno.agent import Agent
from agno.models.openai import OpenAIChat
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import asyncio

from braintrust.wrappers.agno import setup_agno
import braintrust


# Set up Braintrust observability
setup_agno(project_name="async-team-agent-project")
braintrust.auto_instrument()
braintrust.init_logger(project="async-team-agent-project")

from agno.agent import Agent
from agno.models.openai import OpenAIChat
Expand All @@ -13,7 +13,6 @@


async def main():
# Create specialized agents for the team
research_agent = Agent(
name="Research Analyst",
model=OpenAIChat(id="gpt-4o-mini"),
Expand All @@ -37,7 +36,6 @@ async def main():
debug_mode=True,
)

# Create a team with both agents
investment_team = Team(
name="Investment Research Team",
model=OpenAIChat(id="gpt-4o-mini"),
Expand Down
14 changes: 14 additions & 0 deletions examples/agno/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[project]
name = "braintrust-agno-example"
version = "0.1.0"
description = "Agno agents and teams traced via Braintrust"
requires-python = ">=3.10"
dependencies = [
"braintrust",
"agno",
"openai",
"yfinance",
]

[tool.uv.sources]
braintrust = { path = "../../py", editable = true }
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
from braintrust.wrappers.agno import setup_agno
import braintrust


setup_agno(project_name="simple-agent-project")
braintrust.auto_instrument()
braintrust.init_logger(project="simple-agent-project")

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.yfinance import YFinanceTools


# Create and configure the agent
agent = Agent(
name="Stock Price Agent",
model=OpenAIChat(id="gpt-4o-mini"),
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
from braintrust.wrappers.agno import setup_agno
import braintrust


setup_agno(project_name="simple-agent-project")
braintrust.auto_instrument()
braintrust.init_logger(project="simple-agent-project")

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.yfinance import YFinanceTools


# Create and configure the agent
agent = Agent(
name="Stock Price Agent",
model=OpenAIChat(id="gpt-4o-mini"),
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
from braintrust.wrappers.agno import setup_agno
import braintrust


# Set up Braintrust observability
setup_agno(project_name="team-agent-project")
braintrust.auto_instrument()
braintrust.init_logger(project="team-agent-project")

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.team import Team
from agno.tools.yfinance import YFinanceTools


# Create specialized agents for the team
research_agent = Agent(
name="Research Analyst",
model=OpenAIChat(id="gpt-4o-mini"),
Expand All @@ -34,7 +33,6 @@
debug_mode=True,
)

# Create a team with both agents
investment_team = Team(
name="Investment Research Team",
model=OpenAIChat(id="gpt-4o-mini"),
Expand Down
14 changes: 14 additions & 0 deletions examples/anthropic/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Anthropic + Braintrust

Calls `braintrust.auto_instrument()` to patch the Anthropic SDK so both sync and async clients are traced automatically. The async script also exercises `messages.stream(...)` and `stream=True` create calls.

## Run

```bash
export BRAINTRUST_API_KEY=...
export ANTHROPIC_API_KEY=...

uv sync
uv run python sync.py
uv run python async_example.py
```
Loading