Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,69 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
because its SDK process is conversation-scoped.
- The SDK evolution agent now enables vendor process reuse for multi-stage
SDK-backed runs and closes internally owned runtimes when the run exits.
- `FinishReason` enum of the canonical `finish_reason` values, plus first-class
`AgentTask.model` and `AgentTask.reasoning_effort` fields (keyword-only, so
the positional layout that predates them is unchanged; the
`metadata["model"]`/`metadata["reasoning_effort"]` aliases keep working).
`model` is honored by all three adapters. `reasoning_effort` maps to the
Claude and Codex `effort` options; Antigravity has no reasoning-effort
control and rejects the field with a typed error instead of silently
ignoring it.
- Third-party runtime kinds: `AgentRuntimeKind.coerce` and the registry accept
namespaced strings (e.g. `"x-myorg-agent"`), and `runtime_kind_value()`
returns the wire form of either shape.
- Tool observability parity: the Codex adapter emits `agent.tool.requested` /
`agent.tool.completed` events, Claude tool audits carry `result_preview`, and
Antigravity audits are recorded per call rather than collapsed by tool name.
- Claude reports the effective vendor permission mode in
`AgentResult.metadata["permission_mode"]`, and the Codex/Antigravity adapters
record SDK kwargs dropped for compatibility in
`AgentResult.metadata["dropped_options"]`.
- `docs/api-stability.md` documents the public surface, the 0.x compatibility
policy, and the vendor SDK pinning policy.

### Changed

- BREAKING: the `AgentRuntime` protocol now requires `aclose()` and async
context-manager support, and `kind` is typed `AgentRuntimeKind | str` so
third-party runtimes can conform without forking the enum.
- BREAKING: mapping fields on `AgentTask`, `AgentResult`, and related models are
copied at construction and read-only at the top level afterwards (in-place
mutation of the mapping itself raises `TypeError`; the freeze is shallow, so
nested containers are not copied or frozen); `dataclasses.asdict`,
`copy.deepcopy`, `pickle`, and JSON serialization keep working.
- Claude: `CAUTIOUS` now maps to the vendor `default` permission mode instead of
`acceptEdits` (it was looser than `DEFAULT`), and a `READ_ONLY` filesystem
forces `plan` mode.
- Claude: an unsatisfied `output_schema` or an empty completion now returns
`finish_reason="failed"` instead of silent success.
- Codex: `Usage` reports the executed turn rather than cumulative
across-the-thread totals when resuming sessions.
- All adapters: `AgentResult.session_id` falls back to the task-supplied session
handle when the SDK response omits one.
- Antigravity: explicit `vertex=True` takes precedence over ambient API keys,
combining a `READ_ONLY` filesystem or `STRICT` mode with non-read-only
`allowed_tools` is rejected instead of silently granted, and vendor stop
reasons map to `max_tokens`/`failed` finish reasons.
- BREAKING: Antigravity deny-lists are now subtractive in every mode.
`disabled_tools` means "enable everything else", which re-enabled write and
destructive tools past the mode's baseline; now only `PERMISSIVE` (whose
baseline is every tool) still takes that route, while `READ_ONLY`/`STRICT`
get the read-only toolset minus the denied tools and `DEFAULT`/`CAUTIOUS`
the nondestructive toolset minus the denied tools.
- Vendor SDK dependencies now carry pre-1.0 upper bounds (`claude-agent-sdk<0.3`,
`openai-codex<0.2`, `google-antigravity<0.2`) so a breaking upstream minor
cannot reach fresh installs before adapters are revalidated.
- BREAKING: permission-critical SDK options fail closed under vendor drift. If
the installed SDK cannot accept Claude's `permission_mode` (or a requested
tool allow/deny list), Codex's `sandbox`/`approval_mode`, or Antigravity's
`capabilities`/`policies`/workspace scoping, `run()` raises
`UnsupportedTaskInputError` instead of silently running under the SDK's
default (more permissive) posture. A requested `budget_usd` fails closed the
same way if the installed Claude SDK stops accepting `max_budget_usd`, so a
spend cap can never silently vanish. Non-security drift is still tolerated
and recorded in `AgentResult.metadata["dropped_options"]`.

- Installation docs now lead with `agent-runtime-kit[all]` for the easiest
full-provider setup and explain provider extras as dependency isolation, not a
separate API.
Expand All @@ -36,6 +96,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Removed

- Removed the stale internal publish checklist from public documentation.
- BREAKING: removed the dead `AgentCapabilities.sdk_turn_limit` field (no
adapter ever read it).

### Fixed

- Reused vendor SDK clients are evicted when a run is interrupted or cancelled
mid-flight, and `aclose()` no longer races an in-flight run on the same
runtime instance.
- Codex: a turn ending in the SDK's non-terminal `inProgress` status — or any
future unknown status — now fails closed as `finish_reason="failed"` instead
of reading as success with partial output.
- Codex: `Usage.input_tokens` now excludes cached input tokens, which are
reported separately in `cache_read_tokens` (matching the documented `Usage`
contract and the Antigravity adapter) instead of being double-counted across
both fields.
- Event redaction now covers camelCase secret keys (e.g. `accessToken`) and
separator-less ones (e.g. `accesstoken`, `SESSIONTOKEN`) while keeping plural
usage counters (`inputTokens`, `totaltokens`) visible, and the event
sanitizer bounds recursion depth and detects reference cycles so pathological
metadata cannot abort a run.
- SDK evolution example: inspecting candidate SDK versions (which pip-installs
and imports freshly downloaded upstream code) is now opt-in via
`--inspect-candidates` and runs in a credential-scrubbed environment, and
`--draft-pr` no longer fails when the report directory is gitignored.

## 0.2.0 - 2026-06-23

Expand Down
22 changes: 14 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,20 +97,26 @@ asyncio.run(main())

## Runtime Fields

`AgentTask` supports goal, system prompt, working directory, permission profile,
MCP stdio servers, session/resume handles, output schema, budget, metadata, and
an async event sink. Where a runtime cannot honor a field (for example only
Claude maps `budget_usd`; Codex and Antigravity reject it with a typed
`UnsupportedTaskInputError`) the adapter raises rather than silently dropping it.

`AgentResult` returns output, finish reason, parsed structured output, usage,
cost, session id, artifacts, tool-call audits, and provider metadata.
`AgentTask` supports goal, system prompt, model, reasoning effort, working
directory, permission profile, MCP stdio servers, session/resume handles, output
schema, budget, metadata, and an async event sink. (`model` and
`reasoning_effort` are first-class fields; the legacy `metadata["model"]` /
`metadata["reasoning_effort"]` aliases still work.) Where a runtime cannot honor
a field (for example only Claude maps `budget_usd`; Codex and Antigravity reject
it with a typed `UnsupportedTaskInputError`) the adapter raises rather than
silently dropping it.

`AgentResult` returns output, finish reason (see `FinishReason`), parsed
structured output, usage, cost, session id, tool-call audits, and provider
metadata. `artifacts` is a reserved field: no built-in runtime populates it yet,
so it is always an empty tuple today.

## Docs

- [Quickstart](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/quickstart.md)
- [Provider diagnostics](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/providers.md)
- [Capability matrix](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/capability-matrix.md)
- [API stability](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/api-stability.md)
- [Live smoke tests](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/live-smoke.md)
- [Mestre migration notes](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/mestre-migration.md)
- [SDK evolution agent](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/sdk-evolution-agent.md)
56 changes: 56 additions & 0 deletions docs/api-stability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# API stability & versioning

`agent-runtime-kit` is pre-1.0 (`0.x`) and follows semantic versioning with the
usual pre-1.0 caveat: **breaking changes may land in a minor release** (`0.N`)
while the API is being shaped toward 1.0. Patch releases (`0.N.P`) are additive
or bug-fix only.

## Public API

The supported surface is exactly what `agent_runtime_kit.__all__` exports (and
the vendor adapters under `agent_runtime_kit.adapters`). Anything whose module or
name begins with an underscore (`agent_runtime_kit._types`,
`agent_runtime_kit._runtime`, etc.) is internal and may change without notice —
import the names from the top-level package instead.

`agent_runtime_kit.testing` is public and intended for downstream test suites
(fake runtimes and event sinks).

## Compatibility guarantees within a 0.x line

- **Runtime kinds are open.** `AgentRuntimeKind.coerce` accepts namespaced
strings (e.g. `"x-myorg-agent"`), and the registry stores them, so a third
party can ship an adapter for a new runtime without forking the enum.
- **Every runtime exposes the async lifecycle** (`aclose`, `async with`) declared
by the `AgentRuntime` protocol. Stateless runtimes implement it as a no-op.
- **`finish_reason` values** come from `FinishReason`. The field is typed `str`
for forward-compatibility, so new reasons can be added without a type break;
compare against `FinishReason` members rather than bare literals.
- **Result/task mappings are read-only at the top level.** `AgentTask`/`AgentResult`
copy `Mapping` fields into read-only dicts at construction; in-place writes to
the mapping itself raise `TypeError`. The freeze is shallow — nested containers
are not copied or frozen — so treat the whole structure as immutable by
convention and compare by value. The wrappers remain plain `dict` subclasses,
so `dataclasses.asdict`, `pickle`, `copy.deepcopy`, and JSON serialization
keep working.
- **Unsupported inputs raise, they are not dropped.** An adapter that cannot honor
a task field raises `UnsupportedTaskInputError`; the one exception is
vendor-option drift, which is recorded in
`AgentResult.metadata["dropped_options"]` instead.

## Vendor SDK version policy

The vendor SDK extras are pinned with cautious upper bounds (e.g.
`claude-agent-sdk>=0.2.87,<0.3`) because those SDKs are themselves pre-1.0 and
have shipped breaking changes within a minor series. The bounds are raised
deliberately — after the contract tests and the SDK-evolution agent verify a new
version — rather than left unbounded. A weekly CI lane installs the latest vendor
SDKs *within the declared caps*, so drift inside an allowed range surfaces before
it reaches installed users; a release above a cap is by design invisible to that
lane until the cap is raised. A separate lane installs every direct dependency at
its declared floor so a stale minimum cannot sit undetected in the metadata.

## Deprecation

When a public name is slated for removal it will be kept working for at least one
minor release with a `DeprecationWarning` before it is dropped.
22 changes: 14 additions & 8 deletions docs/capability-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@
| Structured output | Native `output_format` when available | Native output schema / JSON parse fallback | Native response schema / JSON parse fallback |
| MCP stdio servers | Yes | No per-task MCP config | Yes, without per-server env |
| Permission mapping | `permission_mode` | approval mode + sandbox | capabilities + policies |
| Streaming output events | Yes — incremental output/tool events while the SDK runs | Not enabled in v1 adapter | Yes — from response chunks |
| Tool audit events | Yes — from streamed message blocks | Yes — parsed from `TurnResult` items | Yes — from tool chunks |
| Streaming output events | Yes — incremental `output.delta` while the SDK runs | No — a single `output.delta` at the end (non-streaming SDK) | Yes — from response chunks |
| Tool audit events | Yes — streamed from message blocks | Yes — emitted from parsed `TurnResult` items after the turn | Yes — from tool chunks |
| `vendor.turn` events | No | No | Yes — from thought/unknown chunks |
| Missing package diagnostics | Yes (`AgentRuntimeUnavailableError`) | Yes (`AgentRuntimeUnavailableError`) | Yes (`AgentRuntimeUnavailableError`) |
| Missing credential diagnostics | Provider-owned/local auth, including API key, Bedrock, Vertex, AWS, and Azure modes | Provider-owned/local auth, including ChatGPT/API key/custom providers/Bedrock | API key or Google ADC / Vertex config |
| Missing credential diagnostics | No — `availability()` reports available with an `auth_source` label; auth failures surface at `run()` | No — same as Claude (`auth_source` label; deferred) | Yes — `availability()` returns `MISSING_CREDENTIALS` when no API key / ADC-Vertex project is configured |
| Live smoke test | Opt-in | Opt-in | Opt-in |

The matrix is intentionally not a lowest-common-denominator contract. Adapters
Expand Down Expand Up @@ -45,9 +46,13 @@ mode: `READ_ONLY` → `read_only`, `WORKSPACE_WRITE` → `workspace_write`,

Antigravity toolset notes: the table above describes the default posture when
`allowed_tools` is empty. A `READ_ONLY` filesystem forces the read-only toolset
regardless of mode. User-supplied `allowed_tools`/`disallowed_tools` override the
defaults and are validated against the `BuiltinTools` enum (for example
`"view_file"`, not `"Read"`); an unknown name raises `UnsupportedTaskInputError`.
regardless of mode. User-supplied `allowed_tools`/`disallowed_tools` are
validated against the `BuiltinTools` enum (for example `"view_file"`, not
`"Read"`); an unknown name raises `UnsupportedTaskInputError`. An allow-list
naming a non-read-only tool under a `READ_ONLY` filesystem or `STRICT` mode is
rejected, and a deny-list subtracts from the mode's baseline toolset rather
than re-enabling everything else (only `PERMISSIVE`, whose baseline is every
tool, uses the SDK's `disabled_tools` route).

## Rejected inputs

Expand All @@ -56,9 +61,10 @@ surface to honor, rather than dropping them silently.

| Field | Claude | Codex | Antigravity |
|-------|--------|-------|-------------|
| `budget_usd` | Mapped (`max_budget_usd`) | Rejected | Rejected |
| `budget_usd` | Mapped (`max_budget_usd`, fails closed under SDK drift) | Rejected | Rejected |
| `reasoning_effort` | Mapped (`effort`) | Mapped (`effort`) | Rejected (no SDK surface) |
| `permissions.network` | Rejected | Rejected | Rejected |
| `allowed_tools` / `disallowed_tools` | Mapped | Rejected | Mapped (`disallowed_tools` → `disabled_tools`); allow-list and deny-list are mutually exclusive and rejected if combined |
| `allowed_tools` / `disallowed_tools` | Mapped | Rejected | Mapped; deny-lists subtract from the mode's baseline (only `PERMISSIVE` uses `disabled_tools`), and allow-list plus deny-list together is rejected |
| `mcp_servers` | Mapped | Rejected (no per-task MCP) | Mapped, without per-server `env` |

Two task fields are informational only and not enforced by any built-in adapter:
Expand Down
39 changes: 31 additions & 8 deletions docs/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,28 @@ to the exact missing extra.

## Runtime Notes

All three adapters map the task's system prompt (Claude `system_prompt`, Codex
`developer_instructions`, Antigravity `system_instructions`) and the `model`
field (falling back to the `metadata` aliases of the same names).
`reasoning_effort` maps to the Claude and Codex `effort` options; Antigravity
has no reasoning-effort control and rejects the first-class field with a typed
error (its legacy `metadata["reasoning_effort"]` alias stays ignored, as it
always has been). Effort values are passed through to the vendor SDK rather
than validated by this library — each vendor defines its own accepted
vocabulary (for example `claude-agent-sdk` 0.2.x accepts
`low`/`medium`/`high`/`xhigh`/`max`), and an SDK too old to accept `effort` at
all records the drop in `AgentResult.metadata["dropped_options"]`.

Claude uses the `claude-agent-sdk` package and maps working directory,
permissions, MCP servers, sessions, structured output, tool allow/deny lists,
runtime environment, and budget where supported by the installed SDK. It
streams incremental output and tool events while the SDK runs, and sets
permissions, filesystem access (a `READ_ONLY` filesystem forces `plan` mode),
MCP servers, sessions, structured output, tool allow/deny lists, runtime
environment, and budget (a requested `budget_usd` fails closed with a typed
error if the installed SDK stops accepting `max_budget_usd`, so a spend cap
can never silently vanish). It streams
incremental output and tool events while the SDK runs, and sets
`finish_reason="max_turns"` when a turn is truncated by the max-turns limit.
The effective vendor `permission_mode` is reported in
`AgentResult.metadata["permission_mode"]`.
`permissions.network` has no SDK surface and is rejected with a typed error.
Claude auth is provider-owned: use Anthropic API key auth, or configure
third-party provider modes through Claude Code environment/settings such as
Expand Down Expand Up @@ -111,11 +128,17 @@ names are validated against the
not accept per-server env values. The default tool posture with no
`allowed_tools` is:

| `PermissionMode` (or `READ_ONLY` filesystem) | Toolset | Policy |
|----------------------------------------------|---------|--------|
| `STRICT`, or any `READ_ONLY` filesystem | read-only | none (no `allow_all`) |
| `CAUTIOUS`, `DEFAULT` | nondestructive (no `run_command`) | `allow_all` |
| `PERMISSIVE` | all tools | `allow_all` |
| `PermissionMode` / filesystem | Toolset | Policy |
|-------------------------------|---------|--------|
| `STRICT` (any filesystem) | read-only | none (no `allow_all`) |
| any mode with `READ_ONLY` filesystem | read-only | `allow_all` (policy follows the mode; only `STRICT` drops it) |
| `CAUTIOUS`, `DEFAULT` (writable filesystem) | nondestructive (no `run_command`) | `allow_all` |
| `PERMISSIVE` (writable filesystem) | all tools | `allow_all` |

A `READ_ONLY` filesystem forces the read-only toolset; the `allow_all` policy is
dropped only for `STRICT`. When an explicit `allowed_tools` list is combined with
a `READ_ONLY` filesystem, any non-read-only tool in it is rejected rather than
silently granted.

A deny-list under a `READ_ONLY` filesystem (or `STRICT`) subtracts from the
read-only toolset, and under `DEFAULT`/`CAUTIOUS` from the nondestructive
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ include = [
"CHANGELOG.md",
"LICENSE",
"pyproject.toml",
"docs/api-stability.md",
"docs/capability-matrix.md",
"docs/live-smoke.md",
"docs/mestre-migration.md",
Expand Down
Loading