fix(bedrock): preserve stream event type and drop invocation-metrics trailer#1682
Merged
Conversation
…trailer The Bedrock stream decoder previously wrapped every chunk as ServerSentEvent(event="completion"), discarding the payload's actual event type. Chunks without a "type" field — notably the amazon-bedrock-invocationMetrics trailer that Bedrock appends to every stream — then routed through the completion branch in _streaming.py, which does not backfill data["type"]. construct_type fell back to the first member of the RawMessageStreamEvent union and yielded RawMessageStartEvent(message=None), violating the type contract and crashing any consumer that read event.message. The decoder now reads "type" from the chunk payload and uses it as the SSE event name, drops the invocation-metrics trailer (it is not a model stream event), and only falls back to "completion" for legacy untyped payloads. Fixes #1647
craigie-ant
reviewed
Jun 17, 2026
Comment on lines
+82
to
+86
| # Bedrock appends a trailing chunk that only carries invocation | ||
| # metrics; it is not a model stream event, so drop it rather than | ||
| # let it be mis-constructed against the stream-event union. | ||
| if "amazon-bedrock-invocationMetrics" in payload and "completion" not in payload: | ||
| return None |
Contributor
There was a problem hiding this comment.
this is something we should fix but it is a breaking change, as a user may be using this value, so I'm removing the logic here for now
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Bedrock stream decoder wraps every chunk as
ServerSentEvent(event="completion"), discarding the payload's actual event type. Downstream, the"completion"branch in_streaming.py::__stream__does not backfilldata["type"](unlike the typed-event branch), so chunks without atypefield reachconstruct_typeagainst theRawMessageStreamEventunion with no discriminator. That falls back to the first union member and yieldsRawMessageStartEvent(message=None)— an object that violates the SDK's own type contract.The
amazon-bedrock-invocationMetricstrailer that Bedrock appends to streams has notypefield, so it hits exactly this. Any consumer readingevent.message.usagecrashes withAttributeError: 'NoneType' object has no attribute 'usage'.Fixes #1647.
Fix
In
lib/bedrock/_stream_decoder.py(hand-maintained, not Stainless-generated):typefield as the SSE event name, so typed Messages events route through the branch in__stream__that backfillsdata["type"].type, nocompletionkey) instead of forwarding it to the stream-event union.event="completion"for legacy untyped payloads (and any non-JSON payload), preserving the existing Completions path. A legacy completion chunk that also carriesamazon-bedrock-invocationMetricsis kept, not dropped.Relation to #1651
#1651 preserves the event type but still forwards the metrics trailer (its
.get("type", "completion")returns"completion"for the type-less trailer), so the reported crash still occurs. This PR adds the missing drop and a regression test for it.Behavior notes
typeis not in__stream__'s recognized list is dropped rather than yielded as a mis-constructed object — this matches non-Bedrock behavior.{"type": "error", ...}chunk inside a 200 stream now raises (same as direct SSE) instead of being yielded as data.Tests
Four unit tests on
_chunk_bytes_to_sse: typed event preserved, metrics-only trailer dropped, legacy completion kept, legacy completion with merged metrics kept.