docs(spec): streaming host obligations for durability, observers, and retries - #30
Open
MohammadHaroonAbuomar wants to merge 1 commit into
Open
docs(spec): streaming host obligations for durability, observers, and retries#30MohammadHaroonAbuomar wants to merge 1 commit into
MohammadHaroonAbuomar wants to merge 1 commit into
Conversation
… retries Section 18.1 constrains egress precisely but says nothing about the other ways a withheld rune can escape the accounting. This amends the section with host obligations drawn from the streaming comparison study (analysis/streaming-acs-vs-maf.md), which audited the profile against the enforcement lessons of the MAF agent-hooks integration review: - Verdict before durability: durable incorporation of stream text is gated by the same watermark as emission; a terminal deny or failing settlement forbids persisting withheld or uncleared runes; the released prefix may stay durable alongside the refusal. Restates AGENT-HOOKS-0.1 section 6.1 at this profile's granularity, and is mirrored as StreamSession module doc obligation 5. - Observer channels: any consumer outside the enforcement boundary is a caller, observers, callbacks, and preview channels included, and withheld runes must not be delivered to one. - Retry boundary coverage: a track resuming above zero retains the last L - 1 runes the earlier attempt delivered and includes them in values evaluated near the boundary, since a banned term can straddle the attempts. A mediation test pins the straddling term, with the host that dropped the tail as the negative control. - Released text identity: the runes released must be rune identical to the runes the outcomes were evaluated against; a rewrite after clearance invalidates it. Module doc obligation 4. - Mandatory settlement: every opened session settles, an abandoned one included, so the trail keeps the cleared extent and the residue. - Output point interaction: a deny at the assembled output evaluation cannot recall released runes; record and close per AGENT-HOOKS-0.1 section 6.1a, never settle clean. Spec text and tests only; no accounting mechanism changes and no version bump. Signed-off-by: MohammadHaroonAbuomar <40180927+MohammadHaroonAbuomar@users.noreply.github.com>
This was referenced Aug 7, 2026
MohammadHaroonAbuomar
added a commit
that referenced
this pull request
Aug 7, 2026
Three shapes from the MAF agent-hooks review rounds, pinned against the section 18.1 accounting (analysis/streaming-acs-vs-maf.md, R9): - Retry re-invocation across sessions: an abandoned attempt settles failed on its uncleared residue while the clean resumed retry settles clean, and a retry that replays the abandoned attempt's clearance frontier fails closed rather than releasing a gap nothing evaluated. - Success-then-discard: a fully cleared, never emitted stream settles Complete, because clearance is permission and not a delivery record; the companion test pins that the accounting cannot distinguish an emitting host from a discarding one, which names the host obligation boundary instead of pretending to check it. - Streaming versus whole-snapshot asymmetry probe: deny-bearing content reaches the same terminal outcome through one section 18 whole snapshot and through section 18.1 segments, with the incremental path differing only in the bounded exposure of the cleared prefix; a clean stream is the negative control and lands clean both ways. Complements the resume-boundary straddle pair in PR #30; tests live in a new file so the two branches do not contend. Signed-off-by: MohammadHaroonAbuomar <40180927+MohammadHaroonAbuomar@users.noreply.github.com>
liamcrumm
reviewed
Aug 7, 2026
| decision; the stated obligation is the whole of the protection. | ||
| - Section 18.1 states that the attempt boundary is not a clearance boundary. | ||
| A track resuming at an offset above zero retains the last `L - 1` runes | ||
| the earlier attempt delivered and includes them in the value it evaluates |
Collaborator
There was a problem hiding this comment.
Maybe this should say "delivered so far, across all attempts"? 419 has it right and this doesn't. An attempt that delivers fewer than L - 1 runes leaves the next one short.
forbidden at [1,10) of zforbiddenzzz. Attempt 2 delivers one rune, attempt 3 retains e instead of forbidde, clears, and the caller assembles the term.
Same wording in the PR body and mediation.rs:418.
| The profile holds no stream text. A host receives the payloads, so it already accumulates them, and a host that cuts that accumulation into evaluation units already decides where those cuts fall. An implementation MUST NOT impose a second segmentation on such a host, because two accounts of what was evaluated over the same runes cannot both be authoritative. A host therefore declares the rune range it evaluated together with the outcome it obtained, and the accounting tracks what that clears. | ||
|
|
||
| This places three obligations on the host that an implementation cannot verify for it. First, the value the host evaluates for a span MUST contain at least that span's own text. The unit a policy reasons about crosses segment boundaries, so evaluating less is unsound: a policy that refuses a banned span sees the halves of that span separately, permits each, and the caller receives the span. A host that evaluates the whole accumulated prefix satisfies this obligation exactly. | ||
| The caller this profile withholds from is any consumer outside the enforcement boundary, not only the far end of the connection. A host registered observer, a completion callback, a preview or typing channel, a sub agent or logging sink fed from the raw accumulation: each of these is a caller, and withheld runes MUST NOT be delivered to one. The profile holds no text, so nothing structural separates the host's accumulation from a channel wired ahead of the release decision, and an implementation cannot detect one. A consumer inside the enforcement boundary, such as the segmenter or the evaluation itself, necessarily reads unreleased text, which is what the boundary means. |
Collaborator
There was a problem hiding this comment.
Should this cite AH §12.1a, the way §6.1 and §9 are cited? This and obligation 4 at 417 are close to verbatim from it.
It might also help to make the enforcement boundary decidable. Outside means must not receive withheld text, inside means it reads unreleased text, so I can't tell which side a classifier that's an LLM call falls on. A sub agent is listed outside and the evaluation inside.
| Sizing the bound merely above `L` is unsound, and satisfying the coverage obligation for every span does not rescue it. With `L` of 9, `S` of 4, and `N` of 10, a term at runes 4 through 13 is contained in no evaluated window: the window for the span at runes 12 through 16 covers runes 6 through 16 and holds only the term's tail. Every span clears and the term reaches the caller. The same stream is refused at `N` of 12. Second, the rune counts the host reports MUST match the text it accumulated. Third, the profile is an enforcement path, since its whole purpose is to decide which runes reach the caller, so a host MUST NOT feed an `evaluate_only` result into the accounting. Section 20 forbids presenting one as enforcement and a cleared span is enforcement. An implementation cannot check this, because the mode is not carried on a verdict. | ||
| Sizing the bound merely above `L` is unsound, and satisfying the coverage obligation for every span does not rescue it. With `L` of 9, `S` of 4, and `N` of 10, a term at runes 4 through 13 is contained in no evaluated window: the window for the span at runes 12 through 16 covers runes 6 through 16 and holds only the term's tail. Every span clears and the term reaches the caller. The same stream is refused at `N` of 12. Second, the rune counts the host reports MUST match the text it accumulated. Third, the profile is an enforcement path, since its whole purpose is to decide which runes reach the caller, so a host MUST NOT feed an `evaluate_only` result into the accounting. Section 20 forbids presenting one as enforcement and a cleared span is enforcement. An implementation cannot check this, because the mode is not carried on a verdict. Fourth, the runes the host releases MUST be rune identical to the runes the recorded outcomes were evaluated against. A clearance vouches for the text the task saw, so a host side rewrite after clearance, the host's own post processing included, invalidates the clearance it would ride on. Text the host must alter belongs on the whole snapshot path of section 18, or in a new session over the altered value, where what is evaluated and what is emitted are the same runes again. An implementation holding no text cannot compare the two, so this too is stated rather than checked. | ||
|
|
||
| The clamp at the start of the track reads differently for a resumed session. A session resuming a partially delivered stream starts its track at the resume offset, but the text the caller sees started at zero, and a term the policy must detect can straddle the attempt boundary, beginning inside the tail the earlier attempt released and ending inside the first spans of the new one. The attempt boundary is not a clearance boundary. For a track resuming at an offset above zero the host MUST retain the last `L - 1` runes of the text already delivered and include them in the value it evaluates for any span beginning less than `L - 1` runes past the resume offset. A host that no longer holds that tail cannot satisfy the coverage obligation and MUST NOT resume the track under this profile; it assembles the remainder on the whole snapshot path of section 18 instead. |
Collaborator
There was a problem hiding this comment.
Might be worth scoping this to attempts that cleared everything they delivered. The retained tail only covers what the earlier attempt evaluated, which isn't true under deferred or after a settlement with residue. Attempt 1 delivering [0,100) and clearing [0,20) leaves [20,92) that no task saw.
413 would need the same qualifier, since a resumed track's whole accumulated prefix is attempt 2's text.
|
|
||
| A `deny` denies and is terminal for the session. The host MUST stop the stream and MUST NOT release any withheld text, including runes a task had already cleared but the host had not yet emitted. An implementation MUST NOT offer a release point for a track once the session has ended, so that a host polling for one stops on its own rather than on the strength of a value it was still handed. The offset the track reached is unaffected and remains readable, since an audit record needs it. A `deny` carrying an `approval` block is liftable through the host's approval seam (AGENT-HOOKS-0.1 section 9), and resolving it is a host obligation that MUST happen before the outcome reaches the accounting, because a session cannot hold its connection open across an out of band approval. An implementation that receives one unresolved MUST take it at its word and deny, since the decision it carries is `deny` and honoring the seam is the obligation of the host that owns the connection. Withholding the text is the conservative reading, and a host that intended to lift the deny records the lifted outcome instead. | ||
|
|
||
| Release is not the only way a rune leaves the host's hands, and the watermark gates the other way too. Durable incorporation of stream text, into conversation history, a session store, or any record a later evaluation or a later run can read, follows the same rule as emission: a rune becomes eligible for a durable write when the watermark for its track covers it, and a host MUST NOT persist a rune the watermark has not reached, under every safety level. A `deferred` host emits on arrival because latency is its product, but a durable write has no latency to save and waits for the watermark like any withheld emission. On a terminal `deny` or a failing settlement the host MUST NOT persist the withheld or uncleared runes. AGENT-HOOKS-0.1 section 6.1 already obliges a host to discard a refused response rather than incorporate it into subsequent agent state, and this clause is that rule restated at the granularity this profile evaluates, so that a partially cleared stream cannot enter a store whole at payload arrival time. The released prefix is different: it is already part of the caller visible record, so a host MAY persist it even when the remainder is later refused, and a record that carries the refusal alongside that prefix says what happened, where one that persists the prefix and omits the refusal does not. |
Collaborator
There was a problem hiding this comment.
Under deferred a rune can be emitted and not yet watermark covered, so [40,100) with the watermark at 40 falls under the first sentence's MUST NOT and the last sentence's MAY. It might be clearer to state persistable as the intersection, watermark covered and already emitted.
And maybe §12.1 item 4 is the better reference than §6.1? §6.1 forbids persisting a refused result, this forbids persisting before any verdict arrives, and §12.1 item 4 names 18.1 as its reference implementation.
|
|
||
| Settlement MUST NOT advance a watermark on the failing path. A settlement that fails is precisely when the host must emit nothing further, so raising the release point as a side effect of failing would invite the opposite. When the session settles, any rune that no task cleared is a fail closed condition regardless of safety level. The host MUST NOT emit that residue and MUST record the reserved reason `host_error:streaming_unsupported`. That reason also covers a span that would confirm a gap nothing evaluated, payload arriving after the host closed the stream, an outcome on a settled session, an unrecognized source type, an unrecognized safety level, payload on a track the session does not mediate, a configuration mediating neither track, a session that exceeds the offset ceiling of its transport, an outcome naming a task the track was not configured with, an outcome naming an offset past the text the session was told about, an outcome whose span covers no runes, and a `transform` the session cannot honor. A malformed verdict is reported as `host_error:verdict_invalid` instead, since the fault is the shape of the verdict and not the stream. | ||
|
|
||
| Every session that is opened settles. A host MUST settle every session, including one it abandons: a caller disconnect, a cancellation, and a retry that replaces the session with a resumed one each end the attempt and not the obligation. An abandoned session settles like any other, so residue no task cleared fails it closed and is recorded, while a session whose every rune was cleared settles complete, which records that everything was evaluated and releasable and claims nothing about delivery. A session that is simply dropped leaves no settlement outcome, and a trail with no settlement outcome silently loses both the offset the tasks had reached, which an audit record needs, and the fact that residue existed at all. |
Collaborator
There was a problem hiding this comment.
Might be worth #[must_use] on StreamSession. let end = session.observe(...)?; leaves an unsettled session and compiles clean, so nothing in the type supports this MUST.
What should a host whose process died do here? 419 gives an alternative for the tail a host can't hold, and AH §6.1a handles the equivalent case by emitting agent_shutdown with summary.reason of error.
|
|
||
| Every session that is opened settles. A host MUST settle every session, including one it abandons: a caller disconnect, a cancellation, and a retry that replaces the session with a resumed one each end the attempt and not the obligation. An abandoned session settles like any other, so residue no task cleared fails it closed and is recorded, while a session whose every rune was cleared settles complete, which records that everything was evaluated and releasable and claims nothing about delivery. A session that is simply dropped leaves no settlement outcome, and a trail with no settlement outcome silently loses both the offset the tasks had reached, which an audit record needs, and the fact that residue existed at all. | ||
|
|
||
| This profile evaluates `input` and `post_model_call`. Section 18 keeps `output` on the whole snapshot path in every case, so a host that adopts this profile for caller facing egress and also binds `output` receives that verdict only after runes have reached the caller, whatever safety level the session declared. A `deny` at `output` cannot recall a released rune. The host MUST record it and MUST NOT present the stream as having settled clean over it; the shape is the record and close obligation of AGENT-HOOKS-0.1 section 6.1a, a refusal preserved in the trail with nothing left to prevent, and not a completion. |
Collaborator
There was a problem hiding this comment.
Could this cite §12.1a instead of §6.1a? §6.1a covers agent_startup and agent_shutdown, and §12.1a is the streaming to caller rule. It also requires declaring buffered_output: false and stating an exposure bound, neither of which appears in 18.1.
Might be worth noting too that the accounting can't carry this. The session settles before output runs, so is_clean() is already true and finish() has no amend path.
| // tail the host must retain happens to be the whole of what attempt 1 | ||
| // delivered. | ||
| let runtime = runtime("forbidden"); | ||
| let delivered = "xxxxforb"; |
Collaborator
There was a problem hiding this comment.
The fixture puts the term 4 runes below the boundary, so this denies with a 4 rune tail and never exercises the mandated 8. If you want it to pin the boundary, delivered = "forbidde" with payloads ["n", "yyy"] and released == "" leaks at 7 and denies at 8.
The control below might be sharper with a 7 rune tail than an empty one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The streaming comparison study (
analysis/streaming-acs-vs-maf.md) audited section 18.1 against the enforcement lessons of the MAF agent-hooks integration and found the profile airtight on egress accounting but silent on every other way a withheld rune can escape: nothing forbids persisting accumulated stream text to a history or session store before any task cleared it (verdict-before-durability), host-internal observers, callbacks, and preview channels sit outside the stated caller obligation and can be wired ahead of the release decision, a resumed session's window-coverage clamp lets a banned term straddling the attempt boundary through undetected, and an abandoned session leaves no settlement outcome, so the audit trail silently loses both the cleared extent and the residue. This PR closes those gaps as spec text and tests only — the accounting mechanism and the runtime are untouched, and there is no version bump. These encode enforcement lessons from the agent-hooks integration review in microsoft/agent-framework#7515.What each obligation prevents
StreamSessionmodule doc obligation 5.L - 1runes; a host that no longer holds that tail must not resume under the profile. Pinned by a new mediation test with the tail-dropping host as the negative control (--features streaming).output-point interaction (R8): a post-factooutputdeny being either misread as recall or settled over cleanly. It cannot recall released runes; the host records it and never presents the stream as settled clean (AGENT-HOOKS-0.1 §6.1a record-and-close shape).Test plan
cargo test --workspace --all-features --locked— all green (the two straddling-term tests run under--features streaming; without the feature the file compiles to zero tests, as before).cargo fmt --all --check,cargo clippy --workspace --all-features --locked -- -D warnings— clean.