Skip to content

Instrumentation Middleware fails to populate model_provider #23

@anfinil

Description

@anfinil

Description
In version 3.7.1 of ruby_llm-agents, the Pipeline::Middleware::Instrumentation fails to extract and store several key fields into the ruby_llm_agents_executions table, even though these fields are included in the gem's standard migration template.

Issues Identified

  1. model_provider is never set: The migration defines t.string :model_provider, but the instrumentation middleware only records the model_id. This makes it difficult to run analytics and cost reports grouped by provider (e.g., OpenAI vs. Gemini).
  2. Missing Tracing Fields: request_id and trace_id are present in the schema but are not extracted from the agent's metadata or the execution context, which breaks distributed tracing for LLM calls.
  3. Hierarchy Fallbacks: The middleware doesn't always fallback to checking agent metadata for parent_execution_id or root_execution_id if they aren't explicitly present in the pipeline context.

Suggested Resolution
Update Instrumentation#build_running_execution_data and Instrumentation#build_completion_data to automatically resolve the provider and extract tracing IDs.

Example Fix for model_provider:

# Inside Instrumentation middleware
data[:model_provider] = RubyLLM::Models.find(context.model_used || context.model).provider rescue nil

Example Fix for Tracing IDs:

agent_meta = safe_agent_metadata(context)
data[:request_id] = agent_meta[:request_id] if agent_meta[:request_id]
data[:trace_id] = agent_meta[:trace_id] if agent_meta[:trace_id]

Environment

  • ruby_llm-agents: 3.7.1
  • ruby_llm: 1.12.1
  • ruby_llm-schema: 0.3.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions