implement schema-serde event streams by lucix-aws · Pull Request #641 · aws/smithy-go

lucix-aws · 2026-03-19T14:50:20Z

implements schema-serde event stream support for aws-framed protocols (i.e. all of them)

https://smithy.io/2.0/aws/amazon-eventstream.html#amazon-eventstream

This is fundamentally the exact same thing that we had before, with all of the pieces of event stream wireup split out to allow for schema-serde. If you open up an existing generated event stream body, e.g. https://github.com/aws/aws-sdk-go-v2/blob/main/service/transcribestreaming/eventstream.go, you will see how things sort of map out in the new runtime. I made a point of preserving the structure of the old code as much as possible.

at a glance:

ClientProtocol is extended with new event stream APIs. in practice since all the protocols use the same one, our implementations can just embed the new eventstream.Codec to get support
the "generated" part of event streams is now just a type marshaler that figures out what variant schema to pass to the underlying event stream writer which is now in the runtime. pasted an example of that below
as you saw in a previous PR i migrated the aws eventstream/ module over to smithy-go. in this patch i've also flattened it a bit by dropping the eventstreamapi/ sub-package, which had middleware stuff in it which caused an import cycle. that got moved over to transport/http and the constants got promoted up into smithy-go/eventstream/
i ended up just adapting the SDK's sigv4 for event streams. I extended the auth resolution stuff to where smithyhttp.Signer can optionally implement a message signer, and the SDK provides support for that.

example of generated type adapter:

type audioStreamWriter struct {
    // smithyhttp.EventStreamWriter is new, previously basically its entire body of logic was generated inline per-op
    writer *smithyhttp.EventStreamWriter
}

var _ AudioStreamWriter = (*audioStreamWriter)(nil)

func (w *audioStreamWriter) Send(ctx context.Context, event types.AudioStream) error {
    var variant *smithy.Schema
    switch event.(type) {
    case *types.AudioStreamMemberAudioEvent:
        variant = schemas.AudioStream_AudioEvent
    case *types.AudioStreamMemberConfigurationEvent:
        variant = schemas.AudioStream_ConfigurationEvent
    default:
        return fmt.Errorf("unknown event type: %T", event)
    }   
    sv, ok := event.(smithy.Serializable)
    if !ok {
        return fmt.Errorf("event %T is not serializable", event)
    }   
   // pass it along to the new runtime, which handles delegating to the protocol
    return w.writer.Send(ctx, variant, sv) 
}

I did manual e2e tests w/ transcribestreaming,bedrockruntime, and cloudwatchlogs.

lucix-aws · 2026-03-19T15:11:23Z

relates #458

wty-Bryant · 2026-03-20T18:33:17Z

eventstream/codec.go

+	inner := c.Codec.Serializer()
+	ss := NewShapeSerializer(&msg, inner)
+
+	v.Serialize(ss)


Some dummy question: what's inside v.Serialize(ss) here? From first glance SerializeEventMessage just encodes payload passed from inner protocol serializer impl and sends that to the wire io.Writer, it's a little obscured to understand what happens inside the Serialize() without an implementation/test for now

What I understand here is input v contains the payload that will be serialized/encoded to eventmessage/http request, the runtime serialize workflow is decided by the type of v

Madrigal · 2026-03-23T19:28:53Z

transport/http/protocol.go

+	DeserializeResponse(ctx context.Context, schema *smithy.OperationSchema, types *smithy.TypeRegistry, resp *Response, out smithy.Deserializable) error
+
+	// event stream APIs
+	HasInitialMessages() bool


I understand what this means, but from the ClientProtocol interface it's weird to read "HasInitialMessage" since yeah, there are some messages that are "initial". Maybe something like HasInitialEventMessage would make it way more clear that this just event stream related

Madrigal · 2026-03-23T19:29:57Z

schema.go

 }

 // String returns the IDL microformat for the shape ID.
-func (s *ShapeID) String() string {


any reason to not make this a pointer receiver?

Madrigal · 2026-03-23T19:36:08Z

...egen/src/main/java/software/amazon/smithy/go/codegen/serde2/Serde2EventStreamMiddleware.java

+            Symbol opEventStreamConstructor,
+            Symbol opEventStreamSymbol) {
+
+        // For v2 (early-return) event streams, we must:


Madrigal · 2026-03-23T19:40:47Z

eventstream/codec.go

+// DeserializeInitialResponse reads the first event stream message and
+// deserializes it as the operation output.
+func (c *Codec) DeserializeInitialResponse(schema *smithy.Schema, r io.Reader, out smithy.Deserializable) error {
+	c.payloadBuf = c.payloadBuf[0:0]


TIL this pattern

Madrigal · 2026-03-23T19:43:21Z

eventstream/deserializer.go

+}
+
+func (d *ShapeDeserializer) ReadString(s *smithy.Schema, v *string) error {
+	if d.inBindings {


mostly a question for me, but why inBindings mean look into event stream stuff?

Madrigal · 2026-03-23T19:46:39Z

eventstream/deserializer.go

+	if d.inBindings && isEventHeader(s) {
+		return readEventHeaderInt(d, s, v)
+	}
+	return d.inner.ReadInt8(s, v)


should we do any bounds check?

Madrigal · 2026-03-23T19:54:23Z

.../smithy-go-codegen/src/main/java/software/amazon/smithy/go/codegen/EventStreamGenerator.java

+        var unionSymbol = symbolProvider.toSymbol(union);
+        var implName = StringUtils.uncapitalize(union.getId().getName(serviceShape)) + "Reader";
+        var ifaceName = getEventStreamReaderInterfaceName(serviceShape, union);
+        var members = union.members().stream()


throw a sorted when iterating through members, I noticed that our current code generation basically iterates this at random, so we generate a lot more releases of protocol test than we should https://github.com/aws/aws-sdk-go-v2/commits/main/internal/protocoltest/awsrestjson/deserializers.go

Madrigal · 2026-03-23T20:07:37Z

...egen/src/main/java/software/amazon/smithy/go/codegen/serde2/Serde2EventStreamMiddleware.java

+
+                output, ok := out.Result.($output:P)
+                if out.Result != nil && !ok {
+                    return out, md, $fmtErrorf:T("unexpected output result type: %T", out.Result)


Should we also add "Expected type X got Y"?

implement schema-serde event streams

b886bb1

lucix-aws requested review from a team as code owners March 19, 2026 14:50

wty-Bryant reviewed Mar 20, 2026

View reviewed changes

Madrigal mentioned this pull request Mar 23, 2026

Reading EventStream header values fails with unexpected event header ... with type ... aws/aws-sdk-go-v2#3358

Open

3 tasks

Madrigal reviewed Mar 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement schema-serde event streams#641

implement schema-serde event streams#641
lucix-aws wants to merge 1 commit intofeat-serde2-eventstreamfrom
feat-serde2-eventstream-impl

lucix-aws commented Mar 19, 2026 •

edited

Loading

Uh oh!

lucix-aws commented Mar 19, 2026

Uh oh!

wty-Bryant Mar 20, 2026 •

edited

Loading

Uh oh!

wty-Bryant Mar 20, 2026 •

edited

Loading

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Madrigal Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

lucix-aws commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucix-aws commented Mar 19, 2026

Uh oh!

wty-Bryant Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wty-Bryant Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lucix-aws commented Mar 19, 2026 •

edited

Loading

wty-Bryant Mar 20, 2026 •

edited

Loading

wty-Bryant Mar 20, 2026 •

edited

Loading