Skip to content

[bot] OpenAI ResponseStream text method bypasses instrumentation #155

@braintrust-bot

Description

@braintrust-bot

Summary

The OpenAI ResponseStream instrumentation only wraps the each method, but the upstream OpenAI::Helpers::Streaming::ResponseStream class also exposes a text convenience method that bypasses each and returns text deltas directly. Users who consume the Responses API stream via stream.text.each { |t| ... } get zero tracing.

This is inconsistent within the same SDK: the Chat Completions integration wraps both each and text on ChatCompletionStream, but the Responses integration only wraps each on ResponseStream.

What is missing

ResponseStream#text is not wrapped. The upstream SDK's text method returns a fused chain yielding ResponseTextDeltaEvent content that reads from the underlying HTTP stream directly, bypassing the prepended each method on the ResponseStream object.

How the Chat integration handles this (for reference)

The Chat integration wraps both methods on ChatCompletionStream at lib/braintrust/contrib/openai/instrumentation/chat.rb:152-169:

module ChatCompletionStream
  module InstanceMethods
    def each(&block)        # line 152 — wrapped ✓
      ...
    end

    def text                # line 159 — wrapped ✓
      ...
      Enumerator.new do |y|
        trace_consumption(ctx) do
          original_enum.each { |t| y << t }
        end
      end
    end
  end
end

The same pattern should be applied to ResponseStream. The ctx[:consumed] guard already prevents double-tracing when both text and each are wrapped on the same object.

Upstream ResponseStream methods

The upstream class (OpenAI::Helpers::Streaming::ResponseStream) exposes:

  • each — wrapped ✓
  • textNOT wrapped ✗ — bypasses each, reads from internal stream directly
  • until_done — covered (calls each internally)
  • get_final_response — covered (consumes stream via each)
  • get_output_text — covered (delegates to get_final_response)

Only text needs a wrapper because it is the only consumption method that bypasses each.

Braintrust docs status

not_found — Braintrust docs do not mention text as a stream consumption method or differentiate between stream consumption patterns.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/openai/instrumentation/responses.rb:118-183ResponseStream module only wraps each (line 128)
  • lib/braintrust/contrib/openai/instrumentation/chat.rb:140-225ChatCompletionStream module wraps both each (line 152) and text (line 159)
  • lib/braintrust/contrib/openai/patcher.rb:113-119ResponsesPatcher.patch_response_stream patches ResponseStream class

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions