Skip to content

[BOT ISSUE] No embeddings usage extraction or span helpers (Python and TS SDKs support this) #43

@braintrust-bot

Description

@braintrust-bot

Summary

The Braintrust Rust SDK has no embeddings-specific instrumentation, while both the Braintrust Python SDK (EmbeddingV1Wrapper in py/src/braintrust/oai.py) and TypeScript SDK (wrapEmbeddings via openAIChannels.embeddingsCreate in js/src/wrappers/oai.ts) automatically instrument OpenAI embeddings.create() calls.

What is missing

  1. No embeddings usage extractor — The existing extract_openai_usage() function in src/extractors.rs handles chat completion responses but is not tested or documented for embeddings responses. OpenAI embeddings responses have a different usage shape (prompt_tokens + total_tokens, no completion_tokens) and the extractor may produce misleading metrics (e.g. total_tokens fallback computation requires both prompt_tokens and completion_tokens).

  2. No embeddings span helper — There is no helper analogous to wrap_stream_with_span (in src/stream.rs) for embeddings calls. The Python SDK's EmbeddingV1Wrapper and TypeScript SDK's wrapEmbeddings automatically capture embedding model, input text, dimensions, and token usage in traced spans. The Rust SDK provides no equivalent.

  3. No embeddings response types — The src/stream.rs output types (FinalizedStream, OutputChoice, ChatMessage) are all chat-completion-specific. There are no types for representing embeddings responses (vectors, dimensions, encoding format).

Why this matters

Embeddings are a core AI execution surface. Users building RAG applications, semantic search, or similarity-based evaluation in Rust cannot get automatic instrumentation for their embedding calls through this SDK, unlike users of the Python or TypeScript SDKs.

Braintrust docs status

supported — The Braintrust proxy routes /embeddings requests and supports caching them (proxy docs). The OpenAI integration docs state "All API calls are automatically logged" when using the wrapper, though embeddings are not explicitly called out. The Python and TypeScript SDKs both implement embeddings wrapping.

Upstream sources

Local files inspected

  • src/extractors.rsextract_openai_usage() and extract_anthropic_usage() handle chat completion usage; no embeddings-specific extraction or tests
  • src/stream.rsBraintrustStream, FinalizedStream, OutputChoice, ChatMessage are all chat-completion-specific; no embeddings types
  • src/types.rsUsageMetrics struct has no embeddings-specific fields
  • src/lib.rs — public API exports; no embeddings references
  • Cargo.toml — no embeddings-related dependencies
  • tests/ — no embeddings test scenarios
  • Full codebase search for "embedding", "embed" — zero results in instrumentation code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions