feat(conformance): CTK vectors for the §12.1 incremental exception - #67
Merged
MohammadHaroonAbuomar merged 1 commit intoAug 7, 2026
Merged
Conversation
MohammadHaroonAbuomar
requested a review
from Caitie McCaffrey (CaitieM20)
as a code owner
August 7, 2026 06:20
MohammadHaroonAbuomar
force-pushed
the
mhabuomar/incremental-conformance-vectors
branch
from
August 7, 2026 06:23
1f4482c to
6c94017
Compare
…tion Deliver the vectors PR #64 left as future work, making the buffered_output: false posture CTK-testable (streaming comparison study, agent-control-spec analysis/streaming-acs-vs-maf.md, R7). New streaming/incremental vector part, one vector per exception condition: - AH-CTK-110: a fully permitted stream is released segment by segment under covering verdicts (item 1). - AH-CTK-111: a terminating deny withholds the unreleased remainder, the harmless undelivered tail included, while the released prefix stays released (items 1-2). - AH-CTK-112: residue no emission evaluated fails closed at end of stream with host_error:streaming_unsupported (item 3), via the new stream_truncated scenario form. - AH-CTK-113: durable incorporation is gated by the same discipline as release; withheld and unevaluated content never becomes durable (item 4, section 6.1). The vector grammar grows respond.stream / respond.stream_truncated (chunked mock stream; chunk boundaries pin the evaluated segments) and expect.released_output / expect.persisted_must_not_contain (the egress and durability observables). The part is gated on the new incremental_output capability: buffering hosts (buffered_output: true, the default) do not declare it and skip, so the vectors are additive and no existing declared surface changes. Reference-harness skip manifests in all five SDKs pin the new skips; the Python Capability enum and vendored vector copies are updated. Spec section 12.1's future-work note now points at the part; section 13.1/13.3, CLAIMS.md, HARNESS.md (incremental drive conventions and coverage boundaries), PRODUCTION.md row 8, and THREAT-MODEL.md TM-11 are updated to match. Spec version unchanged (agent-hooks/0.1, 0.1.0-alpha). Verified: ajv vector validation green; Rust CTK reference self-test green with the pinned skip set; agent-control-spec's conformance suite run against this corpus reports 46 passed, 0 failed, 5 skipped of 51 (the four new skips are the capability gate working). Signed-off-by: MohammadHaroonAbuomar <40180927+MohammadHaroonAbuomar@users.noreply.github.com>
MohammadHaroonAbuomar
force-pushed
the
mhabuomar/incremental-conformance-vectors
branch
from
August 7, 2026 06:24
6c94017 to
0051e03
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements R7 from the streaming comparison study (agent-control-spec
analysis/streaming-acs-vs-maf.md): the conformance vectors #64 promised as future work, makingbuffered_output: falsea CTK-testable surface instead of declaration-only. Additive; spec version staysagent-hooks/0.1(0.1.0-alpha line).The vector part
New
streaming/incrementalpart, one vector per §12.1 exception condition:host_error:streaming_unsupported(item 3)Grammar and gating
respond.stream/respond.stream_truncated: a chunked mock stream; chunk boundaries pin the evaluated segments (prefix emissions,finish_reason: "incremental"for non-final segments), and truncation pins the residue shape.expect.released_output/expect.persisted_must_not_contain: the egress and durability observables, backed by two RunRecord fields the declaring harness reports.incremental_outputcapability gates the part via the existing subset-skip mechanism: a host declaringbuffered_output: true(the default) never declares it and skips. The vectors pin the watermark-gated posture (declared exposure bound: none — ACS §18.1blocking); a looser-bound host also skips, documented in HARNESS.md.Honest limits, stated in the artefacts
released_output/persisted_must_not_containlands with the first declaring host (HARNESS.md says so); today the part is inert-by-gating for every surface, so nothing dead ships in the engines.Docs updated (the #64 future-work notes)
Spec §12.1 (future-work note → pointer to the part), §13.1, §13.3,
conformance/CLAIMS.md,conformance/HARNESS.md(drive conventions + coverage boundaries),docs/PRODUCTION.mdrow 8,docs/THREAT-MODEL.mdTM-11 (gap narrowed to the buffered surface), CHANGELOG.Verification
ajv validateover the corpus with the amended schema: all 51 valid; schema compiles.main) run with the vendored corpus swapped for this branch's: 46 passed, 0 failed, 5 skipped of 51 (previously 46/47 with 1 skip). The four new skips are the capability gate doing its job; the 46-pass surface is intact.