Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions .claude/commands/agent-runtime-kit/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ upstream SDK releases are the point of this workflow:
env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE \
uv run python -m examples.sdk_evolution_agent \
--runtime claude-agent-sdk \
--refresh-preview \
--mode report \
--package claude-agent-sdk \
--package openai-codex \
--package openai-codex-cli-bin \
Expand Down Expand Up @@ -159,11 +159,8 @@ BRANCH="sdk-evolution-upgrade-$(date +%Y%m%d-%H%M%S)"
env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE \
uv run python -m examples.sdk_evolution_agent \
--runtime claude-agent-sdk \
--refresh-preview \
--implementation-enabled \
--create-branch \
--mode upgrade-pr \
--branch-name "$BRANCH" \
--draft-pr \
--pr-base main \
--commit-message "Run SDK evolution update" \
--pr-title "Run SDK evolution update across vendor packages" \
Expand All @@ -175,6 +172,11 @@ env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE \

If the user chose another runtime, replace only the `--runtime` value.

Successful upgrade PRs commit the mechanical update artifacts: `uv.lock`,
`pyproject.toml` only when a cap raise was applied, and the tracked
`.sdk-evolution/` baseline. The timestamped `reports/sdk-evolution/` directory
is local and gitignored; its evidence is embedded in the PR body.

## Verification

After implementation, run or verify:
Expand Down
12 changes: 7 additions & 5 deletions .codex/skills/agent-runtime-kit-upgrade/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ freshness cutoffs:
env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE \
uv run python -m examples.sdk_evolution_agent \
--runtime codex-agent-sdk \
--refresh-preview \
--mode report \
--package claude-agent-sdk \
--package openai-codex \
--package openai-codex-cli-bin \
Expand Down Expand Up @@ -132,11 +132,8 @@ BRANCH="sdk-evolution-upgrade-$(date +%Y%m%d-%H%M%S)"
env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE \
uv run python -m examples.sdk_evolution_agent \
--runtime codex-agent-sdk \
--refresh-preview \
--implementation-enabled \
--create-branch \
--mode upgrade-pr \
--branch-name "$BRANCH" \
--draft-pr \
--pr-base main \
--commit-message "Run SDK evolution update" \
--pr-title "Run SDK evolution update across vendor packages" \
Expand All @@ -146,6 +143,11 @@ env -u UV_EXCLUDE_NEWER -u UV_EXCLUDE_NEWER_PACKAGE \
--package google-antigravity
```

Successful upgrade PRs commit the mechanical update artifacts: `uv.lock`,
`pyproject.toml` only when a cap raise was applied, and the tracked
`.sdk-evolution/` baseline. The timestamped `reports/sdk-evolution/` directory
is local and gitignored; its evidence is embedded in the PR body.

## Verification

Run or verify:
Expand Down
8 changes: 8 additions & 0 deletions .sdk-evolution/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# SDK Evolution Baseline

This directory is maintained by `examples.sdk_evolution_agent` after a promoted
SDK evolution run. `baseline.json` records the accepted lockfile hash, package
versions, report artifact hashes, and promoted API snapshot hashes. The
`snapshots/` files hold the current promoted API snapshot per vendor package.

Blocked and report-only runs must not update this directory.
63 changes: 33 additions & 30 deletions docs/sdk-evolution-agent-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,34 +123,38 @@ Step responsibilities:
report with the evidence bundle, analysis, decision, reviewer output,
uncertainty, blocked reasons, and the exact manual review questions. This is a
valid end state, not a failed run.
- **Apply safe implementation**: Apply only the changes allowed by the accepted
architecture decision and deterministic gates. This may include lockfile
updates, adapter changes, tests, docs, examples, compatibility shims, or report
changes. It must not implement changes that were classified as
`manual_design_required`.
- **Apply safe implementation**: Apply only the bounded change vocabulary:
lockfile updates, pyproject upper-bound raises, self-adaptation inside
`examples/sdk_evolution_agent/`, and related SDK-evolution docs/tests.
Adapter source changes remain `manual_design_required`; the example records
the evidence and stops instead of editing adapter modules automatically.
- **Run verification**: Run the verification commands required by the
architecture decision. At minimum, this should cover formatting/linting,
typing, unit tests, lock checks, report generation checks, and any available
live smoke needed for the affected runtime behavior.
- **Promote updated state to current baseline**: After implementation and
verification pass, save the updated lock/package/API/release-note/probe state
as the new current-state baseline for the next run. This promotion should be
explicit, atomic, and tied to the verified commit or workspace state. Failed,
blocked, or manual-design-required runs must not replace the current baseline.
verification pass, save `.sdk-evolution/baseline.json` plus
`.sdk-evolution/snapshots/<package>.json` as the new current-state baseline
for the next run. Promotion refuses snapshots whose observed imported version
differs from the locked version. Failed, blocked, or manual-design-required
runs must not replace the current baseline.
- **Write report and optional draft PR**: Write the final local report with
evidence, decisions, implementation summary, baseline-promotion result, test
results, uncertainty, and manual checklist. If explicitly configured and
authenticated, create or update a draft PR. This step must never auto-merge.
authenticated, create or update a draft PR. Draft PR bodies contain a
machine-readable SDK-evolution marker; matching existing PRs are reused, and
older marked PRs with different deltas are superseded. This step must never
auto-merge.

Every box before direction analysis is deterministic. AI stages may interpret
evidence, but they should not invent evidence that was not collected.

## Operating Modes

The default command should be report-only:
The tested report command is:

```bash
python -m examples.sdk_evolution_agent --runtime fake --refresh-preview
python -m examples.sdk_evolution_agent --mode report --runtime fake
```

This mode collects evidence, writes artifacts, runs the analysis stages through
Expand All @@ -161,9 +165,9 @@ schemas, not the quality of AI reasoning.
A real analysis run should select one configured runtime:

```bash
python -m examples.sdk_evolution_agent --runtime claude-agent-sdk --refresh-preview
python -m examples.sdk_evolution_agent --runtime codex-agent-sdk --refresh-preview
python -m examples.sdk_evolution_agent --runtime antigravity-agent-sdk --refresh-preview
python -m examples.sdk_evolution_agent --mode report --runtime claude-agent-sdk
python -m examples.sdk_evolution_agent --mode report --runtime codex-agent-sdk
python -m examples.sdk_evolution_agent --mode report --runtime antigravity-agent-sdk
```

Before a Codex-backed run, prepare the dedicated SDK evolution auth home:
Expand All @@ -190,7 +194,7 @@ runs should inspect all tracked packages:
```bash
python -m examples.sdk_evolution_agent \
--runtime antigravity-agent-sdk \
--refresh-preview \
--mode report \
--package claude-agent-sdk \
--package openai-codex \
--package openai-codex-cli-bin \
Expand All @@ -204,19 +208,17 @@ installs and imports freshly downloaded upstream code, so it is opt-in and
runs in a credential-scrubbed environment, with explicit `skip` records when
it is off.

Implementation mode should remain explicitly gated:
Implementation mode remains explicitly gated:

```bash
python -m examples.sdk_evolution_agent \
--runtime antigravity-agent-sdk \
--refresh-preview \
--implementation-enabled
python -m examples.sdk_evolution_agent --mode upgrade --runtime antigravity-agent-sdk
```

Even in implementation mode, deterministic gates decide whether edits are
allowed. Draft PR creation is separate and should only happen when the local Git
and GitHub environment is authenticated and explicitly configured with
`--draft-pr`.
`--mode upgrade` expands to refresh evidence, candidate inspection,
implementation, and guarded cap raises. Even in implementation mode,
deterministic gates decide whether edits are allowed. `--mode upgrade-pr` adds
branch creation and a draft PR, refuses to push the default branch, and only
runs after implementation and verification pass.

## Evidence Layers

Expand All @@ -231,12 +233,13 @@ The agent checks:
- `uv.lock` versions.
- Installed distributions in the local environment.
- PyPI metadata and recent releases.
- `uv lock --dry-run -P ...` output with freshness cutoffs removed.
- structured update candidates from a temporary lockfile diff with freshness
cutoff environment variables removed.

`uv lock --dry-run` is the source of truth for update candidates when it is
available. PyPI `latest` metadata is useful context, but it can be misleading
for prerelease packages. For example, a locked prerelease can be newer than the
stable value reported by package metadata.
The temporary lockfile diff is the source of truth for update candidates. PyPI
`latest` metadata is useful context, but it can be misleading for prerelease
packages. Human-facing uv stdout is stored only as auxiliary evidence and is not
parsed for pass/fail decisions.

### 2. API Shape Evidence

Expand Down
66 changes: 34 additions & 32 deletions docs/sdk-evolution-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,16 @@ changelog strategy, caveats, and alternatives, see
Run it from the repository:

```bash
python -m examples.sdk_evolution_agent --runtime fake
python -m examples.sdk_evolution_agent --mode report --runtime fake
```

The `fake` runtime is deterministic and useful for checking the local pipeline
without credentials. For real AI reasoning, select a configured runtime:

```bash
python -m examples.sdk_evolution_agent --runtime claude-agent-sdk
python -m examples.sdk_evolution_agent --runtime codex-agent-sdk
python -m examples.sdk_evolution_agent --runtime antigravity-agent-sdk
python -m examples.sdk_evolution_agent --mode report --runtime claude-agent-sdk
python -m examples.sdk_evolution_agent --mode report --runtime codex-agent-sdk
python -m examples.sdk_evolution_agent --mode report --runtime antigravity-agent-sdk
```

Every AI-backed stage is dispatched as an `AgentTask` through a runtime resolved
Expand Down Expand Up @@ -105,29 +105,30 @@ installed distributions, then compares it with upstream package metadata for:
- `openai-codex-cli-bin`
- `google-antigravity`

When `--refresh-preview` is used, the targeted `uv lock --dry-run -P ...`
preview runs with freshness cutoff environment variables removed, including
`UV_EXCLUDE_NEWER`. This workflow needs fresh upstream SDK information, so local
cutoff variables must not hide candidate releases.
`--mode report` enables the refresh path. The resolver copies `pyproject.toml`,
`uv.lock`, and the minimal README metadata into a temporary directory, runs
targeted `uv lock -P ...` there with freshness cutoff environment variables
removed, then diffs the temporary lockfile against the real one. The real
workspace is not touched during candidate detection. The older dry-run stdout
preview may still be stored as auxiliary evidence, but it is not parsed for
candidate decisions.

## Candidate API Inspection

The command treats `uv.lock` as the current baseline. If the active `.venv`
contains a different installed version, the agent inspects the locked baseline
in a temporary isolated virtualenv instead of trusting the drifted environment.
When a refresh preview is available, package update candidates come from the
resolver's `uv lock --dry-run -P ...` output, not only from PyPI's `latest`
metadata. With `--inspect-candidates`, the agent installs each resolver update
candidate in a temporary isolated virtualenv — with a credential-scrubbed
environment (throwaway `HOME`, `PATH` only) — and writes an API snapshot plus
`api_diffs.json` entry, and runs the behavior probes against the candidate the
same way. This avoids false downgrade diffs for packages whose locked
prerelease is newer than PyPI's stable latest field. Candidate inspection is
opt-in because it executes freshly downloaded upstream code; without the flag,
candidates are recorded as explicit `skip` entries rather than silently
missing evidence.

If `uv lock --dry-run -P ...` reports an SDK update but the run cannot produce a
When refresh evidence is available, package update candidates come from the
temporary lockfile diff, not PyPI's `latest` metadata and not human-facing uv
stdout. With `--inspect-candidates`, the agent installs each resolver update
candidate and beyond-cap candidate in a temporary isolated virtualenv — with a
credential-scrubbed environment (throwaway `HOME`, `PATH` only) — then writes an
API snapshot plus `api_diffs.json` entry and runs behavior probes against the
candidate the same way. Candidate inspection is opt-in because it executes
freshly downloaded upstream code; without the flag, candidates are recorded as
explicit `skip` entries rather than silently missing evidence.

If the structured resolver reports an SDK update but the run cannot produce a
candidate-version API diff for that package, implementation is blocked and the
architecture decision is marked `manual_design_required`. An empty added /
removed / changed diff is valid; a missing diff object is not.
Expand All @@ -143,9 +144,13 @@ and API diffs, but only breaking adapter-contract diffs block implementation.
Report-only mode is the default. To allow the implementation stage, pass:

```bash
python -m examples.sdk_evolution_agent --runtime claude-agent-sdk --implementation-enabled
python -m examples.sdk_evolution_agent --mode upgrade --runtime claude-agent-sdk
```

`--mode upgrade` expands to report + candidate inspection + implementation +
guarded cap raises. It refuses to run from a dirty worktree unless `--allow-dirty`
is set for local development.

Implementation is still blocked when:

- the architecture decision sets `manual_design_required`,
Expand All @@ -169,18 +174,15 @@ design review.
Draft PR creation is opt-in:

```bash
python -m examples.sdk_evolution_agent \
--runtime claude-agent-sdk \
--implementation-enabled \
--create-branch \
--branch-name sdk-evolution-update \
--pr-base main \
--draft-pr
python -m examples.sdk_evolution_agent --mode upgrade-pr --runtime claude-agent-sdk
```

When `--draft-pr` is set, the agent stages `uv.lock` and the run report
directory, commits them with `--commit-message`, pushes the branch, and opens a
draft PR with `gh`. It never auto-merges.
`--mode upgrade-pr` creates a `sdk-evolution/<run_id>` branch when one is not
supplied, refuses to push the default branch, stages `uv.lock`,
`pyproject.toml` when a cap raise was applied, and the tracked `.sdk-evolution/`
baseline. The report directory remains local and gitignored; the draft PR body
embeds report evidence plus a machine-readable SDK-evolution marker. The PR
flow runs only after implementation is applied and verification passes.

The command uses local Git and `gh` authentication. It never auto-merges,
auto-publishes, or scrapes unsupported credentials.
9 changes: 7 additions & 2 deletions examples/sdk_evolution_agent/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,25 @@

import sys
from pathlib import Path
from typing import Any
from typing import TYPE_CHECKING, Any

_REPO_SRC = Path(__file__).resolve().parents[2] / "src"
if _REPO_SRC.exists() and str(_REPO_SRC) not in sys.path:
sys.path.insert(0, str(_REPO_SRC))

__all__ = ["RunOptions", "run_agent"]

if TYPE_CHECKING:
from examples.sdk_evolution_agent.cli import run_agent
from examples.sdk_evolution_agent.models import RunOptions


def __getattr__(name: str) -> Any:
"""Lazily expose CLI helpers without importing every submodule."""

if name in __all__:
from examples.sdk_evolution_agent.cli import RunOptions, run_agent
from examples.sdk_evolution_agent.cli import run_agent
from examples.sdk_evolution_agent.models import RunOptions

exports = {"RunOptions": RunOptions, "run_agent": run_agent}
return exports[name]
Expand Down
Loading