Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 10 additions & 20 deletions .github/workflows/adk-py-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,40 +15,30 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
- name: Set up mise
uses: jdx/mise-action@v3
with:
python-version: ${{ inputs.python-version }}

- name: Set up uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: |
py/requirements-build.txt
py/requirements-dev.txt
py/setup.py
py/noxfile.py
integrations/adk-py/pyproject.toml
integrations/adk-py/uv.lock
cache: true
experimental: true
install_args: python@${{ inputs.python-version }} uv

- name: Install deprecated package compatibility dependencies
working-directory: integrations/adk-py
run: |
uv sync
mise exec python@${{ inputs.python-version }} -- uv sync

- name: Lint deprecated compatibility package
working-directory: integrations/adk-py
run: |
uv run ruff check src
mise exec python@${{ inputs.python-version }} -- uv run ruff check $(git ls-files '*.py' | grep -v 'examples/')

- name: Run deprecated compatibility tests
working-directory: integrations/adk-py
run: |
uv run pytest src/tests/test_reexports.py
mise exec python@${{ inputs.python-version }} -- uv run pytest src/tests/test_reexports.py

- name: Test deprecated package import
working-directory: integrations/adk-py
run: |
uv run python -c "import braintrust_adk; print('braintrust_adk imported successfully')"
uv run python -c "from braintrust_adk import setup_braintrust; print('setup_braintrust imported successfully')"
mise exec python@${{ inputs.python-version }} -- uv run python -c "import braintrust_adk; print('braintrust_adk imported successfully')"
mise exec python@${{ inputs.python-version }} -- uv run python -c "from braintrust_adk import setup_braintrust; print('setup_braintrust imported successfully')"
22 changes: 10 additions & 12 deletions .github/workflows/langchain-py-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,29 +22,27 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up uv
uses: astral-sh/setup-uv@v3
- name: Set up mise
uses: jdx/mise-action@v3
with:
enable-cache: true
cache-dependency-glob: |
integrations/langchain-py/pyproject.toml
integrations/langchain-py/uv.lock
cache: true
experimental: true
install_args: python@${{ inputs.python-version }} uv

- name: Install dependencies
run: |
uv python install ${{ inputs.python-version }}
uv sync
mise exec python@${{ inputs.python-version }} -- uv sync

- name: Lint with ruff
if: ${{ inputs.os == 'ubuntu-latest' }}
run: |
uv run ruff check $(git ls-files '*.py' | grep -v 'examples/')
mise exec python@${{ inputs.python-version }} -- uv run ruff check $(git ls-files '*.py' | grep -v 'examples/')

- name: Run tests
run: |
uv run pytest src
mise exec python@${{ inputs.python-version }} -- uv run pytest src

- name: Test import
run: |
uv run python -c "import braintrust_langchain; print('braintrust_langchain imported successfully')"
uv run python -c "from braintrust_langchain import BraintrustCallbackHandler; print('BraintrustCallbackHandler imported successfully')"
mise exec python@${{ inputs.python-version }} -- uv run python -c "import braintrust_langchain; print('braintrust_langchain imported successfully')"
mise exec python@${{ inputs.python-version }} -- uv run python -c "from braintrust_langchain import BraintrustCallbackHandler; print('BraintrustCallbackHandler imported successfully')"
12 changes: 8 additions & 4 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,14 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch full history for proper diff
- uses: actions/setup-python@v3
- uses: pre-commit/action@v3.0.0
- name: Set up mise
uses: jdx/mise-action@v3
with:
extra_args: --from-ref origin/${{ github.base_ref || 'main' }} --to-ref HEAD
cache: true
experimental: true
- name: Run pre-commit
run: |
mise exec -- pre-commit run --from-ref origin/${{ github.base_ref || 'main' }} --to-ref HEAD
12 changes: 6 additions & 6 deletions .github/workflows/publish-py-sdk.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for changelog generation
- name: Set up Python
uses: actions/setup-python@v5
- name: Set up mise
uses: jdx/mise-action@v3
with:
python-version: "3.13"
cache: true
experimental: true
install_args: python@3.13
- name: Build and verify
working-directory: py
run: |
make install-dev
make verify-build
mise exec python@3.13 -- make -C py install-dev verify-build
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
Expand Down
26 changes: 15 additions & 11 deletions .github/workflows/py.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,25 +26,27 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
- name: Set up mise
uses: jdx/mise-action@v3
with:
python-version: ${{ matrix.python-version }}
cache: true
experimental: true
install_args: python@${{ matrix.python-version }}
- name: Install dependencies
run: |
cd py && make install-dev
mise exec python@${{ matrix.python-version }} -- make -C py install-dev
- name: Test whether the Python SDK can be installed
run: |
# This is already done by make install-dev, but we're keeping this as a separate step
# to explicitly verify that installation works
python -m uv pip install -e ./py[all]
mise exec python@${{ matrix.python-version }} -- python -m uv pip install -e ./py[all]
- name: Test whether the Python SDK can be imported
run: |
python -c 'import braintrust'
mise exec python@${{ matrix.python-version }} -- python -c 'import braintrust'
- name: Run nox tests (shard ${{ matrix.shard }}/2)
shell: bash
run: |
cd py && ./scripts/nox-matrix.sh ${{ matrix.shard }} 2
mise exec python@${{ matrix.python-version }} -- bash ./py/scripts/nox-matrix.sh ${{ matrix.shard }} 2

adk-py:
uses: ./.github/workflows/adk-py-test.yaml
Expand All @@ -68,13 +70,15 @@ jobs:
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
- name: Set up mise
uses: jdx/mise-action@v3
with:
python-version: "3.13"
cache: true
experimental: true
install_args: python@3.13
- name: Install build dependencies and build wheel
run: |
cd py && make install-build-deps && make build
mise exec python@3.13 -- make -C py install-build-deps build
- name: Upload wheel as artifact
uses: actions/upload-artifact@v4
with:
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/test-publish-py-sdk.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,18 @@ jobs:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.ref }}
- name: Set up Python
uses: actions/setup-python@v5
- name: Set up mise
uses: jdx/mise-action@v3
with:
python-version: "3.13"
cache: true
experimental: true
install_args: python@3.13
- name: Install build dependencies
working-directory: py
run: |
make install-dev
mise exec python@3.13 -- make -C py install-dev
- name: Build and verify
working-directory: py
run: |
make verify-build
mise exec python@3.13 -- make -C py verify-build
- name: Get version from built wheel
id: get_version
run: |
Expand Down
1 change: 1 addition & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
python 3.13.3
pre-commit 4.2.0
uv 0.7.8
131 changes: 131 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Braintrust SDK Agent Guide

Guide for contributing to the Braintrust Python SDK repository.

## Defaults

- Use `mise` as the source of truth for tools and environment.
- Prefer `py/` commands over root `make` targets when working on the SDK itself.
- Keep changes narrow and run the smallest relevant test session first.
- Do not rely on optional provider packages being installed unless the active nox session installs them.

## Repo Map

- `py/`: main Python package, tests, examples, nox sessions, release build
- `integrations/`: separate integration packages
- `internal/golden/`: compatibility and golden projects
- `docs/`: supporting docs

Important code areas in `py/src/braintrust/`:

- core SDK modules: top-level package files
- wrappers/integrations: `wrappers/`
- temporal: `contrib/temporal/`
- CLI/devserver: `cli/`, `devserver/`
- tests: colocated `test_*.py`

## Setup

Preferred repo bootstrap:

```bash
mise install
make develop
```

Package-focused setup:

```bash
cd py
make install-dev
```

Install optional provider dependencies only if needed:

```bash
cd py
make install-optional
```

## Commands

Preferred SDK workflow:

```bash
cd py
make lint
make test-core
nox -l
```

Targeted wrapper/session runs:

```bash
cd py
nox -s "test_openai(latest)"
nox -s "test_openai(latest)" -- -k "test_chat_metrics"
```

Root `Makefile` exists as a convenience wrapper. The authoritative SDK workflow is in `py/Makefile` and `py/noxfile.py`.

## Tests

`py/noxfile.py` is the source of truth for compatibility coverage.

Key facts:

- `test_core` runs without optional vendor packages.
- wrapper coverage is split across dedicated nox sessions by provider/version.
- `pylint` installs the broad dependency surface before checking files.
- `test-wheel` is a wheel sanity check and requires a built wheel first.

When changing behavior, run the narrowest affected session first, then expand only if needed.

## VCR

VCR cassette directories:

- `py/src/braintrust/cassettes/`
- `py/src/braintrust/wrappers/cassettes/`
- `py/src/braintrust/devserver/cassettes/`

Behavior from `py/src/braintrust/conftest.py`:

- local default: `record_mode="once"`
- CI default: `record_mode="none"`
- wheel-mode skips VCR-marked tests
- test fixtures inject dummy API keys and reset global state

Common commands:

```bash
cd py
nox -s "test_openai(latest)"
nox -s "test_openai(latest)" -- --disable-vcr
nox -s "test_openai(latest)" -- --vcr-record=all -k "test_openai_chat_metrics"
```

Only re-record cassettes when the behavior change is intentional. If in doubt, ask the user.

## Build Notes

Build from `py/`:

```bash
cd py
make build
```

Important caveat:

- `py/scripts/template-version.sh` rewrites `py/src/braintrust/version.py` during build.
- `py/Makefile` restores that file afterward with `git checkout`.

Avoid editing `py/src/braintrust/version.py` while also running build commands.

## Editing Guidance

- Keep tests near the code they cover.
- Reuse existing fixtures and cassette patterns.
- If a change affects examples or integrations, update the nearest example or focused test.
- For CLI/devserver changes, consider whether wheel-mode behavior also needs coverage.
27 changes: 0 additions & 27 deletions CLAUDE.md

This file was deleted.

1 change: 1 addition & 0 deletions CLAUDE.md
Loading