Skip to content

[BOT ISSUE] langchainrb gem not instrumented #148

@braintrust-bot

Description

@braintrust-bot

Summary

The langchainrb gem is the most widely adopted Ruby AI framework (~1.1M downloads, v0.19.5, May 2025) and is not instrumented by this SDK. It is the Ruby analog of Python's LangChain, providing chat completions, embeddings, tool calling, RAG, and assistant/agent patterns across multiple providers.

What is missing

No instrumentation exists for any langchainrb execution surface. Key APIs that should be instrumented:

Chat Completions

  • Langchain::LLM::*#chat(messages:, ...) — unified chat interface across all providers (OpenAI, Anthropic, Google Gemini, Ollama, AWS Bedrock, Azure OpenAI, Cohere, HuggingFace, Mistral, Replicate)
  • Langchain::LLM::*#complete(prompt:, ...) — text completion

Embeddings

  • Langchain::LLM::*#embed(text:, ...) — vector embeddings generation across providers

Tool / Function Calling

  • Tool definitions and automatic tool execution within chat flows

Structured Outputs

  • Langchain::OutputParsers::StructuredOutputParser — JSON schema-based structured extraction from LLM responses

Assistants / Agents

  • Langchain::Assistant — agent framework with tool execution, conversation management, and multi-turn interaction

Braintrust docs status

not_found — No mention of langchainrb as a Ruby integration anywhere in Braintrust documentation. The SDK integrations page at https://www.braintrust.dev/docs/integrations/sdk-integrations lists a general "LangChain" integration but it is not Ruby-specific and does not cover this gem.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib/*/integration.rb — only 4 integrations registered: OpenAI, RubyOpenAI, RubyLLM, Anthropic. No langchainrb integration.
  • lib/braintrust/contrib/ — no directory or files for langchainrb or langchain
  • Appraisals — no appraisal scenarios for langchainrb
  • braintrust.gemspec — no mention of langchainrb
  • Grep for langchain across lib/braintrust/ returns zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions