From 0051e03acc5c347a2c55e6a77e990f332bf21c3f Mon Sep 17 00:00:00 2001 From: MohammadHaroonAbuomar <40180927+MohammadHaroonAbuomar@users.noreply.github.com> Date: Fri, 7 Aug 2026 06:19:43 +0000 Subject: [PATCH] feat(conformance): CTK vectors for the section 12.1 incremental exception Deliver the vectors PR #64 left as future work, making the buffered_output: false posture CTK-testable (streaming comparison study, agent-control-spec analysis/streaming-acs-vs-maf.md, R7). New streaming/incremental vector part, one vector per exception condition: - AH-CTK-110: a fully permitted stream is released segment by segment under covering verdicts (item 1). - AH-CTK-111: a terminating deny withholds the unreleased remainder, the harmless undelivered tail included, while the released prefix stays released (items 1-2). - AH-CTK-112: residue no emission evaluated fails closed at end of stream with host_error:streaming_unsupported (item 3), via the new stream_truncated scenario form. - AH-CTK-113: durable incorporation is gated by the same discipline as release; withheld and unevaluated content never becomes durable (item 4, section 6.1). The vector grammar grows respond.stream / respond.stream_truncated (chunked mock stream; chunk boundaries pin the evaluated segments) and expect.released_output / expect.persisted_must_not_contain (the egress and durability observables). The part is gated on the new incremental_output capability: buffering hosts (buffered_output: true, the default) do not declare it and skip, so the vectors are additive and no existing declared surface changes. Reference-harness skip manifests in all five SDKs pin the new skips; the Python Capability enum and vendored vector copies are updated. Spec section 12.1's future-work note now points at the part; section 13.1/13.3, CLAIMS.md, HARNESS.md (incremental drive conventions and coverage boundaries), PRODUCTION.md row 8, and THREAT-MODEL.md TM-11 are updated to match. Spec version unchanged (agent-hooks/0.1, 0.1.0-alpha). Verified: ajv vector validation green; Rust CTK reference self-test green with the pinned skip set; agent-control-spec's conformance suite run against this corpus reports 46 passed, 0 failed, 5 skipped of 51 (the four new skips are the capability gate working). Signed-off-by: MohammadHaroonAbuomar <40180927+MohammadHaroonAbuomar@users.noreply.github.com> --- CHANGELOG.md | 17 ++++ conformance/CLAIMS.md | 11 ++- conformance/HARNESS.md | 76 ++++++++++++++-- conformance/vectors.schema.json | 28 +++++- ...-110-incremental-release-within-bound.json | 91 +++++++++++++++++++ ...-incremental-deny-withholds-remainder.json | 91 +++++++++++++++++++ ...-112-incremental-residue-fails-closed.json | 77 ++++++++++++++++ ...-CTK-113-incremental-durability-gated.json | 88 ++++++++++++++++++ docs/PRODUCTION.md | 2 +- docs/THREAT-MODEL.md | 4 +- .../AgentHooks.Tests/CtkReferenceTests.cs | 19 +++- sdk/go/conformance/reference_test.go | 14 ++- sdk/python/python/agent_hooks/ctk/harness.py | 5 + ...-110-incremental-release-within-bound.json | 91 +++++++++++++++++++ ...-incremental-deny-withholds-remainder.json | 91 +++++++++++++++++++ ...-112-incremental-residue-fails-closed.json | 77 ++++++++++++++++ ...-CTK-113-incremental-durability-gated.json | 88 ++++++++++++++++++ sdk/python/tests/test_ctk_reference.py | 11 ++- sdk/rust/core/tests/ctk_reference.rs | 19 +++- sdk/typescript/test/ctk_reference.test.mjs | 15 ++- spec/AGENT-HOOKS-0.1.md | 25 +++-- 21 files changed, 898 insertions(+), 42 deletions(-) create mode 100644 conformance/vectors/AH-CTK-110-incremental-release-within-bound.json create mode 100644 conformance/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json create mode 100644 conformance/vectors/AH-CTK-112-incremental-residue-fails-closed.json create mode 100644 conformance/vectors/AH-CTK-113-incremental-durability-gated.json create mode 100644 sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-110-incremental-release-within-bound.json create mode 100644 sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json create mode 100644 sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-112-incremental-residue-fails-closed.json create mode 100644 sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-113-incremental-durability-gated.json diff --git a/CHANGELOG.md b/CHANGELOG.md index a7e6f3b..4187633 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,23 @@ User-visible changes to the spec and SDKs. Versioning rules: [VERSIONING.md](VERSIONING.md). +## Unreleased + +- **The §12.1 incremental exception is CTK-testable.** The vectors + the alpha.5 entry below left as future work exist: a + `streaming/incremental` part (`AH-CTK-110`–`AH-CTK-113`) exercises + the exception's four conditions against a chunked mock stream — + release under covering verdicts, a terminating deny that withholds + the unreleased remainder, uncleared residue failing closed with + `host_error:streaming_unsupported`, and §6.1-gated durability. The + vector grammar grows `respond.stream`/`respond.stream_truncated` and + `expect.released_output`/`expect.persisted_must_not_contain`; the + part is gated on the new `incremental_output` capability, so every + buffering host (`buffered_output: true`, the default) skips it — + the vectors are additive and no existing declared surface changes. + Reference-harness skip manifests across the five SDKs pin the new + skips. Spec version unchanged (`agent-hooks/0.1`, 0.1.0-alpha). + ## 0.1.0-alpha.5 — tag `v0.1.0-alpha.5` - **Python: `Verdict.allow()` constructor sugar,** completing the diff --git a/conformance/CLAIMS.md b/conformance/CLAIMS.md index 25cf143..f2619fb 100644 --- a/conformance/CLAIMS.md +++ b/conformance/CLAIMS.md @@ -25,7 +25,10 @@ property (§10.1). A claim with `buffered_output: false` MUST state that a `deny` at `output` cannot retract already-streamed content (§12.1a); one whose host mediates incrementally under the §12.1 exception MUST also state the exposure -bound its accounting discipline enforces. +bound its accounting discipline enforces and MUST declare +`incremental_output`, so the `streaming/incremental` vectors +(`AH-CTK-110`–`AH-CTK-113`) run against that discipline instead of +being skipped. | Framework | Adapter version | Spec | Capabilities | Profiles | Identity provider | SDK | Report | Notes | | --- | --- | --- | --- | --- | --- | --- | --- | --- | @@ -50,8 +53,10 @@ artefacts, in the PR: the claim states records/approvals are identity-unbound; custom provider → content-derived or not; `buffered_output: false` → the claim states a deny at `output` cannot retract streamed - content, plus the exposure bound when the host mediates - incrementally under the §12.1 exception. + content, plus the exposure bound and the `incremental_output` + declaration (its report then covers the `streaming/incremental` + part) when the host mediates incrementally under the §12.1 + exception. Acceptance is by CODEOWNERS review (`conformance/` owner). The reviewer checks: the report matches the declared surface tuple; the diff --git a/conformance/HARNESS.md b/conformance/HARNESS.md index 151dc80..e1fc81e 100644 --- a/conformance/HARNESS.md +++ b/conformance/HARNESS.md @@ -106,18 +106,69 @@ coercion class the core's raw-text scan (§10.2) exists to reject; see `true` (the host buffers caller-bound output until the `output` combined verdict permits), and a host that streams to its caller without buffering declares `buffered_output: false` in its surface and -claim. A declaring host may also mediate `post_model_call` -incrementally under the §12.1 exception; its claim then states the -exposure bound (§12.1a). The CTK drives hosts with mocked I/O and -cannot exercise streaming egress, so no vector carries this capability -— the declaration exists to make the retraction limitation visible -(§13.3). Vectors exercising the §12.1 accounting discipline against a -mocked stream are future work. +claim. The CTK drives hosts with mocked I/O and cannot exercise +streaming egress, so no vector carries this capability — the +declaration exists to make the retraction limitation visible (§13.3). + +`incremental_output` gates the `streaming/incremental` vector part +(`AH-CTK-110`–`AH-CTK-113`), which exercises the §12.1 exception's +accounting discipline against a mocked stream (see "Incremental +mediation" below). Declare it only when your host declares +`buffered_output: false` **and** mediates `post_model_call` +incrementally with watermark-gated release (declared exposure bound: +none — the posture ACS §18.1 calls `blocking`); the vectors pin that +posture's deterministic release points, so a host with a looser bound +does not declare the capability and skips the part. A buffering host +never declares it. Non-finite floats (NaN/Infinity) and lone surrogates cannot be expressed in a JSON vector at all — those §4.4 marshalling guards are pinned by per-SDK unit tests, not vectors. +## Incremental mediation + +Vectors in the `streaming/incremental` part carry a chunked mock +stream: `respond.stream` is an ordered list of chunks whose +concatenation equals `respond.content`. A harness declaring +`incremental_output` MUST drive them as follows: + +- The mock model delivers the chunks in order. Each chunk boundary + closes one evaluated segment, and the host emits one + `post_model_call` per segment over the **assembled prefix** through + that chunk: `response.content` is the prefix, `response.finish_reason` + is `"incremental"` for a non-final segment and the scripted + `finish_reason` for the final one. Each emission is an ordinary + `post_model_call` (§12.1); the scripted interceptor answers each. +- Release is watermark-gated: a permitted segment's text egresses on + its verdict; a `deny` terminates the stream, withholds everything + not yet released, and stops chunk consumption. +- `respond.stream_truncated: true` means the stream dies abnormally + after the listed chunks: the final chunk is a partial segment no + emission covers, and the scripted `finish_reason` never arrives. The + host MUST fail closed per §12.1 exception item 3 — a + `post_model_call` over the full delivered assembly with + `response.finish_reason: "stream_incomplete"` and a deny + self-verdict `host_error:streaming_unsupported` — withholding and + not persisting the residue. The vectors assert the record, not + whether interceptors observe that emission (host-defined, as with + provider faults). +- The `RunRecord` grows two fields for this part: `released_output` — + the caller-visible content the host actually egressed, in order + (distinct from `final_output`, which stays null on a blocked run) — + and `persisted` — a serialization of every durable incorporation the + host made for the session (conversation history, session stores). + `expect.released_output` is compared exactly; + `expect.persisted_must_not_contain` asserts substrings that must not + appear in `persisted`. A host that persists nothing reports an empty + value and satisfies the durability assertions vacuously, which is + the always-safe §6.1 posture. + +No in-tree reference harness declares `incremental_output` yet (every +reference harness buffers), so runner-side assertion support for +`released_output`/`persisted_must_not_contain` lands with the first +declaring host; the part is capability-gated precisely so it stays +inert for every buffered surface until then. + ## Running ```bash @@ -151,9 +202,14 @@ is pinned by per-SDK unit tests instead: - **NaN/Infinity marshalling guards (§4.4)** — not representable in a JSON vector file. -- **§12.1 streaming assembly** — the scenario grammar has no partial- - stream form; the fail-closed `host_error:streaming_unsupported` path - is a host obligation the mocked model cannot exercise. +- **§12.1 streaming assembly (buffered path)** — for a buffering host + the scenario grammar has no partial-stream form, so the + assemble-before-`post_model_call` rule and its fail-closed + `host_error:streaming_unsupported` path remain host obligations the + mocked model cannot exercise. The *incremental* path is different: + the `streaming/incremental` part drives it through `respond.stream` + for hosts declaring `incremental_output` (see "Incremental + mediation"), including the residue fail-closed shape (`AH-CTK-112`). - **§12.2 concurrent emissions** — vectors run single-threaded; sequence-uniqueness under concurrency is a per-SDK unit test. - **Multi-turn sessions (§3.1)** — the scenario grammar carries one diff --git a/conformance/vectors.schema.json b/conformance/vectors.schema.json index b653ea2..6b18c20 100644 --- a/conformance/vectors.schema.json +++ b/conformance/vectors.schema.json @@ -45,9 +45,10 @@ "multi_turn", "int64_json", "bigint_json", - "buffered_output" + "buffered_output", + "incremental_output" ], - "description": "int64_json marks vectors whose JSON carries integers beyond 2^53 (within i64/u64): a JavaScript harness cannot even load such a file without silent rounding (JSON.parse), so it omits the capability and skips. bigint_json marks vectors carrying integers beyond u64/i64, which additionally exceed serde_json's integer range — only harness languages whose JSON layer preserves raw numeric tokens (Python int, Go json.Number, .NET JsonNode) can claim it. buffered_output is declaration-only (§12.1a): it appears in a host's declared surface and claim, never as a vector requirement — the CTK's mocked I/O cannot exercise streaming egress." + "description": "int64_json marks vectors whose JSON carries integers beyond 2^53 (within i64/u64): a JavaScript harness cannot even load such a file without silent rounding (JSON.parse), so it omits the capability and skips. bigint_json marks vectors carrying integers beyond u64/i64, which additionally exceed serde_json's integer range — only harness languages whose JSON layer preserves raw numeric tokens (Python int, Go json.Number, .NET JsonNode) can claim it. buffered_output is declaration-only (§12.1a): it appears in a host's declared surface and claim, never as a vector requirement — the CTK's mocked I/O cannot exercise streaming egress. incremental_output marks the streaming/incremental vector part (§12.1 exception): the host declares buffered_output: false, mediates post_model_call incrementally with watermark-gated release (declared exposure bound: none), and its harness can drive a chunked mock stream (scenario respond.stream) and report released_output and persisted content. A host that buffers (buffered_output: true, the default) does not declare it and skips these vectors." }, "default": [] }, @@ -221,6 +222,19 @@ "finish_reason" ], "properties": { + "stream": { + "type": "array", + "items": { + "type": "string" + }, + "minItems": 1, + "description": "Chunked delivery of this response for an incremental_output host (§12.1 exception): the mock model delivers these chunks in order, each chunk boundary closes one evaluated segment, and the host emits one post_model_call per segment over the assembled prefix through that chunk (response.content is the prefix; response.finish_reason is \"incremental\" for a non-final segment and this respond's finish_reason for the final one). The chunk concatenation MUST equal content. Ignored by buffered hosts, which never run a vector carrying incremental_output." + }, + "stream_truncated": { + "type": "boolean", + "default": false, + "description": "The mock stream terminates abnormally after delivering the listed chunks: the final chunk is a partial segment no emission covers and the scripted finish_reason never arrives. The host has residue no emission evaluated and MUST fail closed per §12.1 exception item 3 — a post_model_call over the full delivered assembly with response.finish_reason \"stream_incomplete\" and a deny self-verdict host_error:streaming_unsupported — withholding and not persisting the residue. Only meaningful with stream." + }, "content": true, "tool_calls": { "type": "array", @@ -419,6 +433,16 @@ } }, "final_output": true, + "released_output": { + "description": "Caller-visible content the host actually egressed, in order, by the end of the run (streaming/incremental part only). Asserted against RunRecord.released_output, which the harness of an incremental_output host MUST report. Distinct from final_output: a stream denied mid-way returns no final output, yet the prefix released under covering verdicts had already reached the caller and is pinned here." + }, + "persisted_must_not_contain": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Substrings that MUST NOT appear anywhere in the serialization of RunRecord.persisted — the harness-reported durable incorporations the host made for this session (conversation history, session stores; §6.1, §12.1 exception item 4). A host that persists nothing reports an empty value and passes vacuously, which is the always-safe §6.1 posture; the assertion pins that withheld or unevaluated stream content never became durable." + }, "run_outcome": { "type": "string", "enum": [ diff --git a/conformance/vectors/AH-CTK-110-incremental-release-within-bound.json b/conformance/vectors/AH-CTK-110-incremental-release-within-bound.json new file mode 100644 index 0000000..99635dd --- /dev/null +++ b/conformance/vectors/AH-CTK-110-incremental-release-within-bound.json @@ -0,0 +1,91 @@ +{ + "id": "AH-CTK-110", + "title": "incremental mediation: a fully permitted stream is released segment by segment, each release under a covering verdict (§12.1 exception item 1; the covering-verdict bite on a denied stream is AH-CTK-111)", + "spec_clause": "12.1, 12.1a", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "the quick brown fox", + "stream": [ + "the quick ", + "brown fox" + ], + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": true, + "interceptions": [ + { + "interception_point": "agent_startup" + }, + { + "interception_point": "input" + }, + { + "interception_point": "pre_model_call" + }, + { + "interception_point": "post_model_call", + "context": { + "response.content": "the quick ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "post_model_call", + "context": { + "response.content": "the quick brown fox", + "response.finish_reason": "stop" + } + }, + { + "interception_point": "output", + "context": { + "output.content": "the quick brown fox" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + } + ], + "released_output": "the quick brown fox", + "final_output": "the quick brown fox", + "run_outcome": "completed" + } +} diff --git a/conformance/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json b/conformance/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json new file mode 100644 index 0000000..97c3358 --- /dev/null +++ b/conformance/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json @@ -0,0 +1,91 @@ +{ + "id": "AH-CTK-111", + "title": "incremental mediation: a terminating deny withholds the unreleased remainder — the denied segment and every chunk behind it, harmless or not — while the prefix released under covering verdicts stays released (§12.1 exception items 1–2; with the watermark-gated bound a permitted segment is released on its verdict, so the withheld-but-permitted clause of item 2 is structurally empty here and is exercised on the durable side by AH-CTK-113)", + "spec_clause": "12.1, 12.1a", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "public part, SECRET-PART then a harmless tail", + "stream": [ + "public part, ", + "SECRET-PART", + " then a harmless tail" + ], + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "match": { + "response.content": "public part, SECRET-PART" + }, + "return": { + "decision": "deny", + "reason": "ctk:streamed-secret" + } + }, + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": false, + "interceptions": [ + { + "interception_point": "post_model_call", + "context": { + "response.content": "public part, ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "post_model_call", + "context": { + "response.content": "public part, SECRET-PART", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "interceptions_absent": [ + "output" + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "deny", + "verdict.reason": "ctk:streamed-secret" + } + } + ], + "released_output": "public part, ", + "final_output": null, + "run_outcome": "blocked" + } +} diff --git a/conformance/vectors/AH-CTK-112-incremental-residue-fails-closed.json b/conformance/vectors/AH-CTK-112-incremental-residue-fails-closed.json new file mode 100644 index 0000000..6b6c201 --- /dev/null +++ b/conformance/vectors/AH-CTK-112-incremental-residue-fails-closed.json @@ -0,0 +1,77 @@ +{ + "id": "AH-CTK-112", + "title": "incremental mediation: residue no emission evaluated fails closed at end of stream — the truncated tail is self-denied host_error:streaming_unsupported, withheld, and never settles clean (§12.1 exception item 3, the same failure mode as the non-assembling host)", + "spec_clause": "12.1", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "clean segment, partial tail", + "stream": [ + "clean segment, ", + "partial tail" + ], + "stream_truncated": true, + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": false, + "interceptions": [ + { + "interception_point": "post_model_call", + "context": { + "response.content": "clean segment, ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "interceptions_absent": [ + "output" + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "deny", + "verdict.reason": "host_error:streaming_unsupported" + } + } + ], + "released_output": "clean segment, ", + "persisted_must_not_contain": [ + "partial tail" + ], + "final_output": null, + "run_outcome": "blocked" + } +} diff --git a/conformance/vectors/AH-CTK-113-incremental-durability-gated.json b/conformance/vectors/AH-CTK-113-incremental-durability-gated.json new file mode 100644 index 0000000..8fdd22e --- /dev/null +++ b/conformance/vectors/AH-CTK-113-incremental-durability-gated.json @@ -0,0 +1,88 @@ +{ + "id": "AH-CTK-113", + "title": "incremental mediation: durable incorporation is gated by the same discipline as release — content withheld at the terminating deny, the withheld-but-permitted included, and content no emission evaluated never becomes durable; the released prefix MAY be persisted and is not pinned (§12.1 exception item 4, §6.1 restated at segment granularity)", + "spec_clause": "12.1, 6.1", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "keep this, DROP-THIS-SECRET and this undelivered tail", + "stream": [ + "keep this, ", + "DROP-THIS-SECRET", + " and this undelivered tail" + ], + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "match": { + "response.content": "keep this, DROP-THIS-SECRET" + }, + "return": { + "decision": "deny", + "reason": "ctk:streamed-secret" + } + }, + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": false, + "interceptions": [ + { + "interception_point": "post_model_call", + "context": { + "response.content": "keep this, ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "interceptions_absent": [ + "output" + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "deny", + "verdict.reason": "ctk:streamed-secret" + } + } + ], + "released_output": "keep this, ", + "persisted_must_not_contain": [ + "DROP-THIS-SECRET", + "undelivered tail" + ], + "final_output": null, + "run_outcome": "blocked" + } +} diff --git a/docs/PRODUCTION.md b/docs/PRODUCTION.md index 20183a9..2077d11 100644 --- a/docs/PRODUCTION.md +++ b/docs/PRODUCTION.md @@ -16,7 +16,7 @@ the [operations runbook](OPERATIONS.md) covers day-2 concerns | 5 | **Record persistence** | `InterceptionRecord`s are the audit trail and are payload-free by construction (§10.3). Configure a record sink; the in-memory buffer is drop-oldest with a `records_dropped` counter — alert on it (see OPERATIONS). Persist `result_labels` with produced data and resurface them per §5.4. | | 6 | **Approval-channel redaction** | The `ApprovalRequest` carries the context as presented to the resolver and MAY be redacted (§9). Use the emitter's approval-redactor seam; the request identity is computed over the *redacted* context, so the binding covers exactly what the approver saw. Document redaction in `extensions..redacted`. | | 7 | **Payload bounds** | Contexts are canonicalized, hashed, and deep-copied per interceptor per emission. Apply the RECOMMENDED bounds (5 MiB serialized, depth 128, §12.3); whatever limit you enforce, breach MUST yield `deny host_error:context_invalid`, never a crash or truncation. | -| 8 | **Output streaming** | If you stream output to the caller, buffer until the `output` combined verdict permits (§12.1a) — or declare `buffered_output: false` and state in your claim that a deny at `output` cannot retract streamed content. A declaring host may also evaluate `post_model_call` incrementally under the §12.1 bounded-exposure exception; state the exposure bound in the claim. | +| 8 | **Output streaming** | If you stream output to the caller, buffer until the `output` combined verdict permits (§12.1a) — or declare `buffered_output: false` and state in your claim that a deny at `output` cannot retract streamed content. A declaring host may also evaluate `post_model_call` incrementally under the §12.1 bounded-exposure exception; state the exposure bound in the claim and declare `incremental_output` so the `streaming/incremental` CTK vectors run against your accounting discipline. | | 9 | **Value domain** | String-encode 64-bit identifiers at the adapter boundary (§4.4). JavaScript hosts cannot observe big integers at all (`JSON.parse` rounds first); Go hosts must decode with `json.Number`. | | 10 | **Zero-interceptor state** | An `enforce`-mode emission with nothing registered fails closed (`host_error:no_interceptor`, §7). A deliberate passthrough is an explicit allow-all interceptor — deploy one consciously or not at all. | diff --git a/docs/THREAT-MODEL.md b/docs/THREAT-MODEL.md index ad4d3ce..8245857 100644 --- a/docs/THREAT-MODEL.md +++ b/docs/THREAT-MODEL.md @@ -53,7 +53,7 @@ SDKs; **GAP** = no automated verification exists. | TM-08 | Identity collision via canonicalization divergence: two SDKs (or two values) canonicalize differently, breaking approval binding and audit correlation | S, R | In | §10.2 RFC 8785 via single Rust core; closed required+conditional preimage; all bindings delegate. Non-I-JSON values fail closed: in-memory check + raw-text scan for integer literals serde-class parsers coerce (beyond u64/i64) | `golden` (11 fixtures asserted in Rust/Python/TS/.NET/Go); `sdk/rust/core/src/canonical.rs` JCS + scan tests; AH-CTK-090 (beyond 2⁵³), AH-CTK-091 (beyond u64, `bigint_json` harnesses) | | TM-09 | Audit-record payload leakage: records or failure messages exfiltrate context data into audit storage | I | In | §10.3 payload-free verdict projection (`transform.value` dropped, messages truncated); failure verdicts carry exception *type* only (including Go panic recovery); host-synthesized remediation details are value-free by rule (§6.3/§14) — the I-JSON and envelope rejections name the path and constraint, never the value | Record shape: `spec/schema/interception-record.schema.json`; projection tests per SDK; `canonical.rs` value-free-detail tests (`envelope_details_are_value_free`, raw-scan not-contains assertions); Go panic tests. Residual: an interceptor can still deliberately place payload in its own bounded `reason` | | TM-10 | Exfiltration/SSRF via `evidence.verification_pointers`: attacker-supplied URIs dereferenced by host or audit tooling | I | In (host obligation) | §5.3/§14: host MUST NOT dereference; propagate opaque | **GAP** — prose only; no test, no scheme allow-list guidance | -| TM-11 | Streaming egress before interception: partial model output reaches the caller before `output` (or `post_model_call`) is evaluated | E, I | In | §12.1 covers model→host streaming (assemble before `post_model_call`, else fail closed `host_error:streaming_unsupported`; a `buffered_output: false` host MAY instead mediate incrementally under the §12.1 bounded-exposure exception, whose claim states the exposure bound). §12.1a covers host→caller egress: buffer until the `output` combined verdict permits (MUST), or declare `buffered_output: false` — conformant, but the claim MUST state that a deny at `output` cannot retract streamed content (§13.3) | §12.1 negative path: **GAP** (no vector). §12.1a is declaration-only by construction — mocked CTK I/O cannot exercise egress; visibility is via the claim, not a vector | +| TM-11 | Streaming egress before interception: partial model output reaches the caller before `output` (or `post_model_call`) is evaluated | E, I | In | §12.1 covers model→host streaming (assemble before `post_model_call`, else fail closed `host_error:streaming_unsupported`; a `buffered_output: false` host MAY instead mediate incrementally under the §12.1 bounded-exposure exception, whose claim states the exposure bound). §12.1a covers host→caller egress: buffer until the `output` combined verdict permits (MUST), or declare `buffered_output: false` — conformant, but the claim MUST state that a deny at `output` cannot retract streamed content (§13.3) | §12.1 negative path on the buffered surface: **GAP** (no partial-stream form for a buffering host). The incremental surface is vector-backed: `streaming/incremental` (`AH-CTK-110`–`113`, gated on `incremental_output`) covers verdict-covered release, deny-withholds-remainder, the residue fail-closed path (`AH-CTK-112`), and gated durability. §12.1a is declaration-only by construction — mocked CTK I/O cannot exercise egress; visibility is via the claim, not a vector | | TM-12 | Resource exhaustion: unbounded `target`/`messages` canonicalized, hashed, and deep-copied per interceptor per emission — multiplied by parallel profiles' per-interceptor snapshots | D | In | §12.3 RECOMMENDED bounds (5 MiB / depth 128) with a normative failure mode: breach of whatever limit the host or core enforces MUST yield `deny host_error:context_invalid` and MUST NOT crash or truncate; the identity path enforces the depth default fail-closed. The record itself is bounded: §10.3 payload-free projection (transform.value dropped, messages truncated) plus the §5.3 10240-byte evidence cap | Depth: `sdk/rust/core/src/canonical.rs` depth tests. Evidence cap: AH-CTK-092 + per-SDK gate tests. Projection: AH-CTK-093. Serialized-size default: still host-side (RECOMMENDED, not emitter-enforced) | | TM-13 | Label-flow loss: `result_labels` from non-winning permit verdicts discarded, or §5.4 persistence/resurfacing not honoured | I | In | §7.3 unions: the combined verdict carries the first-seen-ordered label union across every permit verdict in the emission (all profiles, including approval substitutions) | Union half: AH-CTK-086 + per-SDK union tests. §5.4 persistence/`source_labels` resurfacing across emissions: **GAP** — no vector | | TM-14 | Supply-chain compromise of the artefacts: squatted names, mutable CI actions, unpinned deps | T, S | In | Distribution `agent-hooks-sdk` published on PyPI/crates.io (squatted `agent-hooks` avoided); GitHub Actions pinned by commit SHA; `Cargo.lock` committed; CodeQL enabled | Name claims live (registry state); pins in `.github/workflows/*.yml`. Lockfile enforcement (`--locked` CI builds), dependency automation (Dependabot across all ecosystems), and SBOM + provenance attestation in the release pipeline are in place. Published a1 alphas implement a superseded draft (see SECURITY.md) | @@ -94,7 +94,7 @@ Every **GAP** above, with its current status: | No adversarial *mutate* fault in the vector grammar (interceptor mutates its copy in place) | TM-05 | open (fault-grammar extension) | | No systematic test that every failure path keeps verdict messages payload-free | TM-09 | partially closed (core value-free-detail tests); per-SDK sweep remains | | `verification_pointers` no-dereference is prose-only; no test or scheme guidance | TM-10 | open (SSRF guidance) | -| §12.1 streaming fail-closed path untested (no vector) | TM-11 | vector backlog — the §12.1a egress rule itself is decided and normative | +| §12.1 streaming fail-closed path untested on the buffered surface (no partial-stream form for a buffering host) | TM-11 | narrowed — the incremental surface is closed by `streaming/incremental` (`AH-CTK-112` pins the `host_error:streaming_unsupported` residue path); the buffered negative path stays prose + per-SDK unit tests, and the §12.1a egress rule itself is decided and normative | | No serialized-size bound enforced by any emitter (depth is enforced on the identity path; failure mode on breach is now normative §12.3) | TM-12 | RECOMMENDED default stays host-tunable by decision (2026-07-11) | | §5.4 `result_labels` persistence/`source_labels` resurfacing across emissions untested | TM-13 | open (§5.4 vector) | | Lockfile enforcement, dependency automation, SBOM/signing/provenance | TM-14 | closed (`--locked` CI, Dependabot, SBOM/provenance in the release pipeline) | diff --git a/sdk/dotnet/test/AgentHooks.Tests/CtkReferenceTests.cs b/sdk/dotnet/test/AgentHooks.Tests/CtkReferenceTests.cs index a760a4d..cda3eed 100644 --- a/sdk/dotnet/test/AgentHooks.Tests/CtkReferenceTests.cs +++ b/sdk/dotnet/test/AgentHooks.Tests/CtkReferenceTests.cs @@ -23,11 +23,20 @@ public static IEnumerable Vectors() => Runner.LoadVectors(VectorsDir()) .Select(v => new object[] { (string)v["id"]!, v }); - // Pinned skip set: JsonNode preserves raw numeric tokens, - // so the .NET reference harness declares every value-domain - // capability — nothing may skip. An unexpected skip fails; a stale - // manifest (expected-but-not-skipped) fails the aggregate test. - private static readonly IReadOnlySet ExpectedSkips = new HashSet(); + // Pinned skip set: JsonNode preserves raw numeric tokens, so the + // .NET reference harness declares every value-domain capability + // and no value-domain vector may skip. The streaming/incremental + // part (§12.1 exception) skips because the reference harness + // buffers caller-bound output and does not declare + // incremental_output. An unexpected skip fails; a stale manifest + // (expected-but-not-skipped) fails the aggregate test. + private static readonly IReadOnlySet ExpectedSkips = new HashSet + { + "AH-CTK-110", + "AH-CTK-111", + "AH-CTK-112", + "AH-CTK-113", + }; [Theory] [MemberData(nameof(Vectors))] diff --git a/sdk/go/conformance/reference_test.go b/sdk/go/conformance/reference_test.go index 4cf1ea3..ddceb92 100644 --- a/sdk/go/conformance/reference_test.go +++ b/sdk/go/conformance/reference_test.go @@ -16,9 +16,17 @@ import ( // expectedSkips pins the skip set: Go decodes vector JSON via // json.Number, so the reference harness declares every value-domain -// capability — nothing may skip. An unexpected skip fails its subtest; -// a stale manifest (expected-but-not-skipped) fails the aggregate. -var expectedSkips = map[string]struct{}{} +// capability and no value-domain vector may skip. The +// streaming/incremental part (§12.1 exception) skips because the +// reference harness buffers caller-bound output and does not declare +// incremental_output. An unexpected skip fails its subtest; a stale +// manifest (expected-but-not-skipped) fails the aggregate. +var expectedSkips = map[string]struct{}{ + "AH-CTK-110": {}, + "AH-CTK-111": {}, + "AH-CTK-112": {}, + "AH-CTK-113": {}, +} func TestReferenceHarnessConformance(t *testing.T) { dir := filepath.Join("..", "..", "..", "conformance", "vectors") diff --git a/sdk/python/python/agent_hooks/ctk/harness.py b/sdk/python/python/agent_hooks/ctk/harness.py index d03c17f..cd733d3 100644 --- a/sdk/python/python/agent_hooks/ctk/harness.py +++ b/sdk/python/python/agent_hooks/ctk/harness.py @@ -26,6 +26,11 @@ class Capability(str, Enum): #: losslessly (§4.4). JavaScript harnesses omit this. INT64_JSON = "int64_json" BIGINT_JSON = "bigint_json" + #: The host declares buffered_output: false and mediates + #: post_model_call incrementally under the §12.1 exception with + #: watermark-gated release; gates the streaming/incremental vector + #: part. Buffered hosts (the default) omit this and skip it. + INCREMENTAL_OUTPUT = "incremental_output" class RunOutcome(str, Enum): diff --git a/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-110-incremental-release-within-bound.json b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-110-incremental-release-within-bound.json new file mode 100644 index 0000000..99635dd --- /dev/null +++ b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-110-incremental-release-within-bound.json @@ -0,0 +1,91 @@ +{ + "id": "AH-CTK-110", + "title": "incremental mediation: a fully permitted stream is released segment by segment, each release under a covering verdict (§12.1 exception item 1; the covering-verdict bite on a denied stream is AH-CTK-111)", + "spec_clause": "12.1, 12.1a", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "the quick brown fox", + "stream": [ + "the quick ", + "brown fox" + ], + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": true, + "interceptions": [ + { + "interception_point": "agent_startup" + }, + { + "interception_point": "input" + }, + { + "interception_point": "pre_model_call" + }, + { + "interception_point": "post_model_call", + "context": { + "response.content": "the quick ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "post_model_call", + "context": { + "response.content": "the quick brown fox", + "response.finish_reason": "stop" + } + }, + { + "interception_point": "output", + "context": { + "output.content": "the quick brown fox" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + } + ], + "released_output": "the quick brown fox", + "final_output": "the quick brown fox", + "run_outcome": "completed" + } +} diff --git a/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json new file mode 100644 index 0000000..97c3358 --- /dev/null +++ b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-111-incremental-deny-withholds-remainder.json @@ -0,0 +1,91 @@ +{ + "id": "AH-CTK-111", + "title": "incremental mediation: a terminating deny withholds the unreleased remainder — the denied segment and every chunk behind it, harmless or not — while the prefix released under covering verdicts stays released (§12.1 exception items 1–2; with the watermark-gated bound a permitted segment is released on its verdict, so the withheld-but-permitted clause of item 2 is structurally empty here and is exercised on the durable side by AH-CTK-113)", + "spec_clause": "12.1, 12.1a", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "public part, SECRET-PART then a harmless tail", + "stream": [ + "public part, ", + "SECRET-PART", + " then a harmless tail" + ], + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "match": { + "response.content": "public part, SECRET-PART" + }, + "return": { + "decision": "deny", + "reason": "ctk:streamed-secret" + } + }, + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": false, + "interceptions": [ + { + "interception_point": "post_model_call", + "context": { + "response.content": "public part, ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "post_model_call", + "context": { + "response.content": "public part, SECRET-PART", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "interceptions_absent": [ + "output" + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "deny", + "verdict.reason": "ctk:streamed-secret" + } + } + ], + "released_output": "public part, ", + "final_output": null, + "run_outcome": "blocked" + } +} diff --git a/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-112-incremental-residue-fails-closed.json b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-112-incremental-residue-fails-closed.json new file mode 100644 index 0000000..6b6c201 --- /dev/null +++ b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-112-incremental-residue-fails-closed.json @@ -0,0 +1,77 @@ +{ + "id": "AH-CTK-112", + "title": "incremental mediation: residue no emission evaluated fails closed at end of stream — the truncated tail is self-denied host_error:streaming_unsupported, withheld, and never settles clean (§12.1 exception item 3, the same failure mode as the non-assembling host)", + "spec_clause": "12.1", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "clean segment, partial tail", + "stream": [ + "clean segment, ", + "partial tail" + ], + "stream_truncated": true, + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": false, + "interceptions": [ + { + "interception_point": "post_model_call", + "context": { + "response.content": "clean segment, ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "interceptions_absent": [ + "output" + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "deny", + "verdict.reason": "host_error:streaming_unsupported" + } + } + ], + "released_output": "clean segment, ", + "persisted_must_not_contain": [ + "partial tail" + ], + "final_output": null, + "run_outcome": "blocked" + } +} diff --git a/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-113-incremental-durability-gated.json b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-113-incremental-durability-gated.json new file mode 100644 index 0000000..8fdd22e --- /dev/null +++ b/sdk/python/python/agent_hooks/ctk/vectors/AH-CTK-113-incremental-durability-gated.json @@ -0,0 +1,88 @@ +{ + "id": "AH-CTK-113", + "title": "incremental mediation: durable incorporation is gated by the same discipline as release — content withheld at the terminating deny, the withheld-but-permitted included, and content no emission evaluated never becomes durable; the released prefix MAY be persisted and is not pinned (§12.1 exception item 4, §6.1 restated at segment granularity)", + "spec_clause": "12.1, 6.1", + "part": "streaming/incremental", + "capabilities": [ + "model_calls", + "incremental_output" + ], + "scenario": { + "input": { + "content": "stream a reply", + "role": "user" + }, + "model_script": [ + { + "respond": { + "content": "keep this, DROP-THIS-SECRET and this undelivered tail", + "stream": [ + "keep this, ", + "DROP-THIS-SECRET", + " and this undelivered tail" + ], + "tool_calls": [], + "finish_reason": "stop" + } + } + ] + }, + "interceptor_script": [ + { + "at": "post_model_call", + "match": { + "response.content": "keep this, DROP-THIS-SECRET" + }, + "return": { + "decision": "deny", + "reason": "ctk:streamed-secret" + } + }, + { + "at": "post_model_call", + "return": { + "decision": "allow" + } + } + ], + "expect": { + "sequence_strict": false, + "interceptions": [ + { + "interception_point": "post_model_call", + "context": { + "response.content": "keep this, ", + "response.finish_reason": "incremental" + } + }, + { + "interception_point": "agent_shutdown" + } + ], + "interceptions_absent": [ + "output" + ], + "records": [ + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "allow" + } + }, + { + "interception_point": "post_model_call", + "assert": { + "verdict.decision": "deny", + "verdict.reason": "ctk:streamed-secret" + } + } + ], + "released_output": "keep this, ", + "persisted_must_not_contain": [ + "DROP-THIS-SECRET", + "undelivered tail" + ], + "final_output": null, + "run_outcome": "blocked" + } +} diff --git a/sdk/python/tests/test_ctk_reference.py b/sdk/python/tests/test_ctk_reference.py index 61df960..0176673 100644 --- a/sdk/python/tests/test_ctk_reference.py +++ b/sdk/python/tests/test_ctk_reference.py @@ -14,10 +14,13 @@ _VECTORS = pathlib.Path(__file__).resolve().parents[3] / "conformance" / "vectors" # Pinned skip set: Python ints are arbitrary precision, so the -# reference harness declares every value-domain capability — nothing may -# skip. A skip here means a capability regressed or a vector was -# quietly excluded; both must fail the suite. -EXPECTED_SKIPS: frozenset[str] = frozenset() +# reference harness declares every value-domain capability and no +# value-domain vector may skip. The streaming/incremental part (§12.1 +# exception) skips because the reference harness buffers caller-bound +# output and does not declare incremental_output. Any other skip means +# a capability regressed or a vector was quietly excluded; both must +# fail the suite. +EXPECTED_SKIPS: frozenset[str] = frozenset({"AH-CTK-110", "AH-CTK-111", "AH-CTK-112", "AH-CTK-113"}) @pytest.mark.parametrize( diff --git a/sdk/rust/core/tests/ctk_reference.rs b/sdk/rust/core/tests/ctk_reference.rs index 2ce0811..e8180a6 100644 --- a/sdk/rust/core/tests/ctk_reference.rs +++ b/sdk/rust/core/tests/ctk_reference.rs @@ -30,8 +30,21 @@ async fn ctk_reference_all_vectors() { assert!(unexpected.is_empty(), "{unexpected:#?}"); // Pinned skip manifest: Rust holds i64 (int64_json) but // serde_json coerces beyond-u64 vector literals at load (no - // bigint_json). Exact IDs, not a count: the parity gate must fail - // when the skip set drifts in either direction. + // bigint_json), and the reference harness buffers caller-bound + // output, so it does not declare incremental_output and skips the + // streaming/incremental part (§12.1 exception). Exact IDs, not a + // count: the parity gate must fail when the skip set drifts in + // either direction. skipped.sort(); - assert_eq!(skipped, vec!["AH-CTK-091".to_owned()], "skip set drifted"); + assert_eq!( + skipped, + vec![ + "AH-CTK-091".to_owned(), + "AH-CTK-110".to_owned(), + "AH-CTK-111".to_owned(), + "AH-CTK-112".to_owned(), + "AH-CTK-113".to_owned(), + ], + "skip set drifted" + ); } diff --git a/sdk/typescript/test/ctk_reference.test.mjs b/sdk/typescript/test/ctk_reference.test.mjs index 7e24211..e22e978 100644 --- a/sdk/typescript/test/ctk_reference.test.mjs +++ b/sdk/typescript/test/ctk_reference.test.mjs @@ -16,8 +16,19 @@ const vectorsDir = resolve(here, "../../../conformance/vectors"); // suite — the parity gate must not silently degrade to green when a // capability regresses or a new vector is quietly skipped. // JSON.parse rounds beyond-2^53 integers before any guard can see -// them, so TS declares neither int64_json nor bigint_json. -const EXPECTED_SKIPS = new Set(["AH-CTK-090", "AH-CTK-091", "AH-CTK-095"]); +// them, so TS declares neither int64_json nor bigint_json. The +// streaming/incremental part (§12.1 exception) skips because the +// reference harness buffers caller-bound output and does not declare +// incremental_output. +const EXPECTED_SKIPS = new Set([ + "AH-CTK-090", + "AH-CTK-091", + "AH-CTK-095", + "AH-CTK-110", + "AH-CTK-111", + "AH-CTK-112", + "AH-CTK-113", +]); const skipped = new Set(); const vectors = loadVectors(vectorsDir); diff --git a/spec/AGENT-HOOKS-0.1.md b/spec/AGENT-HOOKS-0.1.md index e47b602..3e69589 100644 --- a/spec/AGENT-HOOKS-0.1.md +++ b/spec/AGENT-HOOKS-0.1.md @@ -1165,11 +1165,17 @@ Each such emission is an ordinary `post_model_call` under §4–§7; the discipline governs what the host does with the verdicts, not the emission contract. ACS §18.1 ("Incremental stream mediation", [agent-control-spec](https://github.com/responsibleai/agent-control-spec)) -is one implementation of such a discipline. This exception is -capability-shaped, not vector-backed: conformance vectors exercising -the accounting discipline are future work, and until they exist the -surface is visible only through the declaration and claim (§12.1a, -§13.3). +is one implementation of such a discipline. The CTK carries a vector +part for this exception, `streaming/incremental` +(`AH-CTK-110`–`AH-CTK-113`), exercising the four items above against a +chunked mock stream (`conformance/HARNESS.md`, "Incremental +mediation"). The part is gated on the `incremental_output` capability: +a host that mediates incrementally declares it and runs the part; a +buffering host (`buffered_output: true`, the default) does not declare +it and skips, so the vectors are additive to every existing surface. +The declaration and claim remain the visible statement of the posture +(§12.1a, §13.3); the CTK still cannot exercise real streaming egress, +only the accounting discipline over mocked I/O. ### 12.1a Streaming to the caller @@ -1257,7 +1263,10 @@ There are no conformance tiers, levels, or baseline profiles. A host the CTK defines (e.g. `int64_json`: the language can observe a JSON integer beyond ±(2⁵³−1) without rounding — JavaScript hosts cannot) and the egress capability `buffered_output` (§12.1a; defaults to - `true`, and `false` MUST be declared explicitly), + `true`, and `false` MUST be declared explicitly) with its companion + `incremental_output` (declared by a host that mediates incrementally + under the §12.1 exception; gates the `streaming/incremental` + vectors), - the composition profiles and knob values it supports (§7.2), - its identity provider (§10.1). @@ -1298,7 +1307,9 @@ identity-unbound. A claim with `buffered_output: false` MUST state that a `deny` at `output` cannot retract already-streamed content (§12.1a), and, when the host mediates incrementally under the §12.1 exception, MUST state the exposure bound its accounting discipline -enforces (§12.1a). +enforces (§12.1a) and MUST declare `incremental_output` so the +`streaming/incremental` vectors run against that discipline rather +than being skipped. ---