diff --git a/CHANGELOG.md b/CHANGELOG.md index 1330c38..130f2e2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,9 +21,69 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 because its SDK process is conversation-scoped. - The SDK evolution agent now enables vendor process reuse for multi-stage SDK-backed runs and closes internally owned runtimes when the run exits. +- `FinishReason` enum of the canonical `finish_reason` values, plus first-class + `AgentTask.model` and `AgentTask.reasoning_effort` fields (keyword-only, so + the positional layout that predates them is unchanged; the + `metadata["model"]`/`metadata["reasoning_effort"]` aliases keep working). + `model` is honored by all three adapters. `reasoning_effort` maps to the + Claude and Codex `effort` options; Antigravity has no reasoning-effort + control and rejects the field with a typed error instead of silently + ignoring it. +- Third-party runtime kinds: `AgentRuntimeKind.coerce` and the registry accept + namespaced strings (e.g. `"x-myorg-agent"`), and `runtime_kind_value()` + returns the wire form of either shape. +- Tool observability parity: the Codex adapter emits `agent.tool.requested` / + `agent.tool.completed` events, Claude tool audits carry `result_preview`, and + Antigravity audits are recorded per call rather than collapsed by tool name. +- Claude reports the effective vendor permission mode in + `AgentResult.metadata["permission_mode"]`, and the Codex/Antigravity adapters + record SDK kwargs dropped for compatibility in + `AgentResult.metadata["dropped_options"]`. +- `docs/api-stability.md` documents the public surface, the 0.x compatibility + policy, and the vendor SDK pinning policy. ### Changed +- BREAKING: the `AgentRuntime` protocol now requires `aclose()` and async + context-manager support, and `kind` is typed `AgentRuntimeKind | str` so + third-party runtimes can conform without forking the enum. +- BREAKING: mapping fields on `AgentTask`, `AgentResult`, and related models are + copied at construction and read-only at the top level afterwards (in-place + mutation of the mapping itself raises `TypeError`; the freeze is shallow, so + nested containers are not copied or frozen); `dataclasses.asdict`, + `copy.deepcopy`, `pickle`, and JSON serialization keep working. +- Claude: `CAUTIOUS` now maps to the vendor `default` permission mode instead of + `acceptEdits` (it was looser than `DEFAULT`), and a `READ_ONLY` filesystem + forces `plan` mode. +- Claude: an unsatisfied `output_schema` or an empty completion now returns + `finish_reason="failed"` instead of silent success. +- Codex: `Usage` reports the executed turn rather than cumulative + across-the-thread totals when resuming sessions. +- All adapters: `AgentResult.session_id` falls back to the task-supplied session + handle when the SDK response omits one. +- Antigravity: explicit `vertex=True` takes precedence over ambient API keys, + combining a `READ_ONLY` filesystem or `STRICT` mode with non-read-only + `allowed_tools` is rejected instead of silently granted, and vendor stop + reasons map to `max_tokens`/`failed` finish reasons. +- BREAKING: Antigravity deny-lists are now subtractive in every mode. + `disabled_tools` means "enable everything else", which re-enabled write and + destructive tools past the mode's baseline; now only `PERMISSIVE` (whose + baseline is every tool) still takes that route, while `READ_ONLY`/`STRICT` + get the read-only toolset minus the denied tools and `DEFAULT`/`CAUTIOUS` + the nondestructive toolset minus the denied tools. +- Vendor SDK dependencies now carry pre-1.0 upper bounds (`claude-agent-sdk<0.3`, + `openai-codex<0.2`, `google-antigravity<0.2`) so a breaking upstream minor + cannot reach fresh installs before adapters are revalidated. +- BREAKING: permission-critical SDK options fail closed under vendor drift. If + the installed SDK cannot accept Claude's `permission_mode` (or a requested + tool allow/deny list), Codex's `sandbox`/`approval_mode`, or Antigravity's + `capabilities`/`policies`/workspace scoping, `run()` raises + `UnsupportedTaskInputError` instead of silently running under the SDK's + default (more permissive) posture. A requested `budget_usd` fails closed the + same way if the installed Claude SDK stops accepting `max_budget_usd`, so a + spend cap can never silently vanish. Non-security drift is still tolerated + and recorded in `AgentResult.metadata["dropped_options"]`. + - Installation docs now lead with `agent-runtime-kit[all]` for the easiest full-provider setup and explain provider extras as dependency isolation, not a separate API. @@ -36,6 +96,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Removed - Removed the stale internal publish checklist from public documentation. +- BREAKING: removed the dead `AgentCapabilities.sdk_turn_limit` field (no + adapter ever read it). + +### Fixed + +- Reused vendor SDK clients are evicted when a run is interrupted or cancelled + mid-flight, and `aclose()` no longer races an in-flight run on the same + runtime instance. +- Codex: a turn ending in the SDK's non-terminal `inProgress` status — or any + future unknown status — now fails closed as `finish_reason="failed"` instead + of reading as success with partial output. +- Codex: `Usage.input_tokens` now excludes cached input tokens, which are + reported separately in `cache_read_tokens` (matching the documented `Usage` + contract and the Antigravity adapter) instead of being double-counted across + both fields. +- Event redaction now covers camelCase secret keys (e.g. `accessToken`) and + separator-less ones (e.g. `accesstoken`, `SESSIONTOKEN`) while keeping plural + usage counters (`inputTokens`, `totaltokens`) visible, and the event + sanitizer bounds recursion depth and detects reference cycles so pathological + metadata cannot abort a run. +- SDK evolution example: inspecting candidate SDK versions (which pip-installs + and imports freshly downloaded upstream code) is now opt-in via + `--inspect-candidates` and runs in a credential-scrubbed environment, and + `--draft-pr` no longer fails when the report directory is gitignored. ## 0.2.0 - 2026-06-23 diff --git a/README.md b/README.md index 0430a68..e173ff4 100644 --- a/README.md +++ b/README.md @@ -97,20 +97,26 @@ asyncio.run(main()) ## Runtime Fields -`AgentTask` supports goal, system prompt, working directory, permission profile, -MCP stdio servers, session/resume handles, output schema, budget, metadata, and -an async event sink. Where a runtime cannot honor a field (for example only -Claude maps `budget_usd`; Codex and Antigravity reject it with a typed -`UnsupportedTaskInputError`) the adapter raises rather than silently dropping it. - -`AgentResult` returns output, finish reason, parsed structured output, usage, -cost, session id, artifacts, tool-call audits, and provider metadata. +`AgentTask` supports goal, system prompt, model, reasoning effort, working +directory, permission profile, MCP stdio servers, session/resume handles, output +schema, budget, metadata, and an async event sink. (`model` and +`reasoning_effort` are first-class fields; the legacy `metadata["model"]` / +`metadata["reasoning_effort"]` aliases still work.) Where a runtime cannot honor +a field (for example only Claude maps `budget_usd`; Codex and Antigravity reject +it with a typed `UnsupportedTaskInputError`) the adapter raises rather than +silently dropping it. + +`AgentResult` returns output, finish reason (see `FinishReason`), parsed +structured output, usage, cost, session id, tool-call audits, and provider +metadata. `artifacts` is a reserved field: no built-in runtime populates it yet, +so it is always an empty tuple today. ## Docs - [Quickstart](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/quickstart.md) - [Provider diagnostics](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/providers.md) - [Capability matrix](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/capability-matrix.md) +- [API stability](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/api-stability.md) - [Live smoke tests](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/live-smoke.md) - [Mestre migration notes](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/mestre-migration.md) - [SDK evolution agent](https://github.com/ebarti/agent-runtime-kit/blob/main/docs/sdk-evolution-agent.md) diff --git a/docs/api-stability.md b/docs/api-stability.md new file mode 100644 index 0000000..8bc85aa --- /dev/null +++ b/docs/api-stability.md @@ -0,0 +1,56 @@ +# API stability & versioning + +`agent-runtime-kit` is pre-1.0 (`0.x`) and follows semantic versioning with the +usual pre-1.0 caveat: **breaking changes may land in a minor release** (`0.N`) +while the API is being shaped toward 1.0. Patch releases (`0.N.P`) are additive +or bug-fix only. + +## Public API + +The supported surface is exactly what `agent_runtime_kit.__all__` exports (and +the vendor adapters under `agent_runtime_kit.adapters`). Anything whose module or +name begins with an underscore (`agent_runtime_kit._types`, +`agent_runtime_kit._runtime`, etc.) is internal and may change without notice — +import the names from the top-level package instead. + +`agent_runtime_kit.testing` is public and intended for downstream test suites +(fake runtimes and event sinks). + +## Compatibility guarantees within a 0.x line + +- **Runtime kinds are open.** `AgentRuntimeKind.coerce` accepts namespaced + strings (e.g. `"x-myorg-agent"`), and the registry stores them, so a third + party can ship an adapter for a new runtime without forking the enum. +- **Every runtime exposes the async lifecycle** (`aclose`, `async with`) declared + by the `AgentRuntime` protocol. Stateless runtimes implement it as a no-op. +- **`finish_reason` values** come from `FinishReason`. The field is typed `str` + for forward-compatibility, so new reasons can be added without a type break; + compare against `FinishReason` members rather than bare literals. +- **Result/task mappings are read-only at the top level.** `AgentTask`/`AgentResult` + copy `Mapping` fields into read-only dicts at construction; in-place writes to + the mapping itself raise `TypeError`. The freeze is shallow — nested containers + are not copied or frozen — so treat the whole structure as immutable by + convention and compare by value. The wrappers remain plain `dict` subclasses, + so `dataclasses.asdict`, `pickle`, `copy.deepcopy`, and JSON serialization + keep working. +- **Unsupported inputs raise, they are not dropped.** An adapter that cannot honor + a task field raises `UnsupportedTaskInputError`; the one exception is + vendor-option drift, which is recorded in + `AgentResult.metadata["dropped_options"]` instead. + +## Vendor SDK version policy + +The vendor SDK extras are pinned with cautious upper bounds (e.g. +`claude-agent-sdk>=0.2.87,<0.3`) because those SDKs are themselves pre-1.0 and +have shipped breaking changes within a minor series. The bounds are raised +deliberately — after the contract tests and the SDK-evolution agent verify a new +version — rather than left unbounded. A weekly CI lane installs the latest vendor +SDKs *within the declared caps*, so drift inside an allowed range surfaces before +it reaches installed users; a release above a cap is by design invisible to that +lane until the cap is raised. A separate lane installs every direct dependency at +its declared floor so a stale minimum cannot sit undetected in the metadata. + +## Deprecation + +When a public name is slated for removal it will be kept working for at least one +minor release with a `DeprecationWarning` before it is dropped. diff --git a/docs/capability-matrix.md b/docs/capability-matrix.md index 8c9ef41..13744f5 100644 --- a/docs/capability-matrix.md +++ b/docs/capability-matrix.md @@ -9,10 +9,11 @@ | Structured output | Native `output_format` when available | Native output schema / JSON parse fallback | Native response schema / JSON parse fallback | | MCP stdio servers | Yes | No per-task MCP config | Yes, without per-server env | | Permission mapping | `permission_mode` | approval mode + sandbox | capabilities + policies | -| Streaming output events | Yes — incremental output/tool events while the SDK runs | Not enabled in v1 adapter | Yes — from response chunks | -| Tool audit events | Yes — from streamed message blocks | Yes — parsed from `TurnResult` items | Yes — from tool chunks | +| Streaming output events | Yes — incremental `output.delta` while the SDK runs | No — a single `output.delta` at the end (non-streaming SDK) | Yes — from response chunks | +| Tool audit events | Yes — streamed from message blocks | Yes — emitted from parsed `TurnResult` items after the turn | Yes — from tool chunks | +| `vendor.turn` events | No | No | Yes — from thought/unknown chunks | | Missing package diagnostics | Yes (`AgentRuntimeUnavailableError`) | Yes (`AgentRuntimeUnavailableError`) | Yes (`AgentRuntimeUnavailableError`) | -| Missing credential diagnostics | Provider-owned/local auth, including API key, Bedrock, Vertex, AWS, and Azure modes | Provider-owned/local auth, including ChatGPT/API key/custom providers/Bedrock | API key or Google ADC / Vertex config | +| Missing credential diagnostics | No — `availability()` reports available with an `auth_source` label; auth failures surface at `run()` | No — same as Claude (`auth_source` label; deferred) | Yes — `availability()` returns `MISSING_CREDENTIALS` when no API key / ADC-Vertex project is configured | | Live smoke test | Opt-in | Opt-in | Opt-in | The matrix is intentionally not a lowest-common-denominator contract. Adapters @@ -45,9 +46,13 @@ mode: `READ_ONLY` → `read_only`, `WORKSPACE_WRITE` → `workspace_write`, Antigravity toolset notes: the table above describes the default posture when `allowed_tools` is empty. A `READ_ONLY` filesystem forces the read-only toolset -regardless of mode. User-supplied `allowed_tools`/`disallowed_tools` override the -defaults and are validated against the `BuiltinTools` enum (for example -`"view_file"`, not `"Read"`); an unknown name raises `UnsupportedTaskInputError`. +regardless of mode. User-supplied `allowed_tools`/`disallowed_tools` are +validated against the `BuiltinTools` enum (for example `"view_file"`, not +`"Read"`); an unknown name raises `UnsupportedTaskInputError`. An allow-list +naming a non-read-only tool under a `READ_ONLY` filesystem or `STRICT` mode is +rejected, and a deny-list subtracts from the mode's baseline toolset rather +than re-enabling everything else (only `PERMISSIVE`, whose baseline is every +tool, uses the SDK's `disabled_tools` route). ## Rejected inputs @@ -56,9 +61,10 @@ surface to honor, rather than dropping them silently. | Field | Claude | Codex | Antigravity | |-------|--------|-------|-------------| -| `budget_usd` | Mapped (`max_budget_usd`) | Rejected | Rejected | +| `budget_usd` | Mapped (`max_budget_usd`, fails closed under SDK drift) | Rejected | Rejected | +| `reasoning_effort` | Mapped (`effort`) | Mapped (`effort`) | Rejected (no SDK surface) | | `permissions.network` | Rejected | Rejected | Rejected | -| `allowed_tools` / `disallowed_tools` | Mapped | Rejected | Mapped (`disallowed_tools` → `disabled_tools`); allow-list and deny-list are mutually exclusive and rejected if combined | +| `allowed_tools` / `disallowed_tools` | Mapped | Rejected | Mapped; deny-lists subtract from the mode's baseline (only `PERMISSIVE` uses `disabled_tools`), and allow-list plus deny-list together is rejected | | `mcp_servers` | Mapped | Rejected (no per-task MCP) | Mapped, without per-server `env` | Two task fields are informational only and not enforced by any built-in adapter: diff --git a/docs/providers.md b/docs/providers.md index e5a4d96..3f92837 100644 --- a/docs/providers.md +++ b/docs/providers.md @@ -25,11 +25,28 @@ to the exact missing extra. ## Runtime Notes +All three adapters map the task's system prompt (Claude `system_prompt`, Codex +`developer_instructions`, Antigravity `system_instructions`) and the `model` +field (falling back to the `metadata` aliases of the same names). +`reasoning_effort` maps to the Claude and Codex `effort` options; Antigravity +has no reasoning-effort control and rejects the first-class field with a typed +error (its legacy `metadata["reasoning_effort"]` alias stays ignored, as it +always has been). Effort values are passed through to the vendor SDK rather +than validated by this library — each vendor defines its own accepted +vocabulary (for example `claude-agent-sdk` 0.2.x accepts +`low`/`medium`/`high`/`xhigh`/`max`), and an SDK too old to accept `effort` at +all records the drop in `AgentResult.metadata["dropped_options"]`. + Claude uses the `claude-agent-sdk` package and maps working directory, -permissions, MCP servers, sessions, structured output, tool allow/deny lists, -runtime environment, and budget where supported by the installed SDK. It -streams incremental output and tool events while the SDK runs, and sets +permissions, filesystem access (a `READ_ONLY` filesystem forces `plan` mode), +MCP servers, sessions, structured output, tool allow/deny lists, runtime +environment, and budget (a requested `budget_usd` fails closed with a typed +error if the installed SDK stops accepting `max_budget_usd`, so a spend cap +can never silently vanish). It streams +incremental output and tool events while the SDK runs, and sets `finish_reason="max_turns"` when a turn is truncated by the max-turns limit. +The effective vendor `permission_mode` is reported in +`AgentResult.metadata["permission_mode"]`. `permissions.network` has no SDK surface and is rejected with a typed error. Claude auth is provider-owned: use Anthropic API key auth, or configure third-party provider modes through Claude Code environment/settings such as @@ -111,11 +128,17 @@ names are validated against the not accept per-server env values. The default tool posture with no `allowed_tools` is: -| `PermissionMode` (or `READ_ONLY` filesystem) | Toolset | Policy | -|----------------------------------------------|---------|--------| -| `STRICT`, or any `READ_ONLY` filesystem | read-only | none (no `allow_all`) | -| `CAUTIOUS`, `DEFAULT` | nondestructive (no `run_command`) | `allow_all` | -| `PERMISSIVE` | all tools | `allow_all` | +| `PermissionMode` / filesystem | Toolset | Policy | +|-------------------------------|---------|--------| +| `STRICT` (any filesystem) | read-only | none (no `allow_all`) | +| any mode with `READ_ONLY` filesystem | read-only | `allow_all` (policy follows the mode; only `STRICT` drops it) | +| `CAUTIOUS`, `DEFAULT` (writable filesystem) | nondestructive (no `run_command`) | `allow_all` | +| `PERMISSIVE` (writable filesystem) | all tools | `allow_all` | + +A `READ_ONLY` filesystem forces the read-only toolset; the `allow_all` policy is +dropped only for `STRICT`. When an explicit `allowed_tools` list is combined with +a `READ_ONLY` filesystem, any non-read-only tool in it is rejected rather than +silently granted. A deny-list under a `READ_ONLY` filesystem (or `STRICT`) subtracts from the read-only toolset, and under `DEFAULT`/`CAUTIOUS` from the nondestructive diff --git a/pyproject.toml b/pyproject.toml index f1fb95f..1651099 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -101,6 +101,7 @@ include = [ "CHANGELOG.md", "LICENSE", "pyproject.toml", + "docs/api-stability.md", "docs/capability-matrix.md", "docs/live-smoke.md", "docs/mestre-migration.md",