[bot] Streaming aggregator and output types cannot represent or capture logprobs data

[braintrust-bot]

Summary

The OutputChoice type has a logprobs field typed as Option<()> (always None), and the streaming aggregator's StreamChoice/StreamDelta structs do not parse the logprobs field from streaming chunks. When a Chat Completions request uses logprobs: true (with optional top_logprobs), the per-token log probability data is silently dropped from the aggregated span output.

What is missing

OpenAI Chat Completions streaming includes a logprobs field on each choice containing per-token log probabilities:

{"choices": [{"index": 0, "delta": {"content": "Hello"}, "logprobs": {"content": [{"token": "Hello", "logprob": -0.31725305, "bytes": [72,101,108,108,111], "top_logprobs": [{"token": "Hello", "logprob": -0.31725305, "bytes": [72,101,108,108,111]}, {"token": "Hi", "logprob": -1.3862944, "bytes": [72,105]}]}]}}]}

Currently in the SDK:

1. OutputChoice (src/stream.rs:356-363) defines logprobs: Option<()> — the type () cannot hold any data; it is always serialized as null and always set to None in both the builder (line 434) and constructor (line 376)
2. StreamChoice (src/stream.rs:658-664) does not have a logprobs field, so logprobs data from streaming chunks is discarded during deserialization
3. aggregate() (src/stream.rs:727-807) has no logic to collect or merge per-token logprobs across chunks
4. There are no logprobs types defined anywhere in the codebase (no LogprobContent, TopLogprob, etc.)

This means when users request logprobs for:

• Confidence scoring — measuring model certainty on classifications or extractions
• Calibration — evaluating whether model-reported probabilities match actual correctness rates
• Content filtering confidence — assessing how confidently the model produced specific tokens
• Token-level analysis — debugging model behavior at the token level

…the logprobs data is present in the stream but lost from the Braintrust span.

Braintrust docs status

unclear — Braintrust's OpenAI integration page does not mention logprobs specifically. Other Braintrust SDKs (TypeScript, Python) capture logprobs as part of the full response object via their wrapOpenAI/wrap_openai wrappers, but there is no explicit documentation about logprobs tracing support.

• https://www.braintrust.dev/docs/integrations/ai-providers/openai

Upstream sources

• OpenAI Chat Completions logprobs parameter: https://platform.openai.com/docs/api-reference/chat/create — logprobs: boolean, top_logprobs: integer (0-5)
• OpenAI streaming logprobs: https://developers.openai.com/api/reference/resources/chat/subresources/completions/streaming-events — streaming choices include logprobs with content[] array of per-token data
• OpenAI logprobs cookbook: https://cookbook.openai.com/examples/using_logprobs — documents use cases including classification confidence and autocomplete

Relationship to existing issues

• Distinct from [BOT ISSUE] Streaming aggregator drops tool calls from Chat Completions deltas #30 (streaming tool call aggregation): [BOT ISSUE] Streaming aggregator drops tool calls from Chat Completions deltas #30 covers the missing tool_calls field in StreamDelta. This is about the missing logprobs field — a different choice property for a different feature.
• Distinct from [BOT ISSUE] Streaming usage metrics drop reasoning tokens and token detail breakdowns #32 (streaming usage detail loss): [BOT ISSUE] Streaming usage metrics drop reasoning tokens and token detail breakdowns #32 is about missing fields in StreamUsage. This is about missing per-token probability data in the choice output, not usage/billing metrics.
• Distinct from [bot] Streaming aggregator and output types drop the structured output refusal field #50 (refusal field): [bot] Streaming aggregator and output types drop the structured output refusal field #50 covers the refusal message field. This covers the logprobs choice-level field — different structural location and purpose.

Local files inspected

• src/stream.rs:356-363 — OutputChoice struct has logprobs: Option<()> (cannot hold data)
• src/stream.rs:376 — OutputChoice::new() hardcodes logprobs: None
• src/stream.rs:434 — OutputChoiceBuilder::build() hardcodes logprobs: None
• src/stream.rs:658-664 — StreamChoice struct has no logprobs field
• src/stream.rs:727-807 — aggregate() has no logprobs collection logic
• Full codebase grep for logprobs — only hits are the Option<()> placeholder in OutputChoice