Add opentelemetry-instrumentation-litellm

[yucheng-berri]

Summary

Adds a new GenAI instrumentation for litellm under instrumentation-genai/opentelemetry-instrumentation-litellm/.

Wraps the four module-level entry points litellm.completion, litellm.acompletion, litellm.embedding, and litellm.aembedding on top of the shared opentelemetry-util-genai TelemetryHandler — following the boundary rules in instrumentation-genai/CLAUDE.md.

What it emits

• gen_ai.* spans (chat <model> / embeddings <model>) with request/response model, temperature, top_p, max_tokens, seed, stop_sequences, response id, finish_reasons, and input/output token counts.
• Duration and token-usage metrics via util-genai's InvocationMetricsRecorder.
• Prompt and output-message content on the span when OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY (or SPAN_AND_EVENT) and OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental are set.
• Streaming reuses the shared SyncStreamWrapper / AsyncStreamWrapper bases from util-genai; only litellm-specific chunk parsing and finalization live here (stream_wrappers.py).

Why litellm needs custom handling

LiteLLM is a unified interface for 100+ providers, so gen_ai.provider.name cannot be hard-coded. It's resolved per-call from litellm's custom_llm_provider:

1. Explicit custom_llm_provider kwarg the caller passed.
2. _hidden_params.custom_llm_provider on the response (populated by litellm after the call resolves).
3. litellm.get_llm_provider(model) fallback (for models with a provider prefix, e.g. anthropic/claude-3-5-sonnet).

Known providers map to the semconv GenAiProviderNameValues enum; unknown ones pass through verbatim so the instrumentation stays useful for the long tail of providers litellm supports.

Base pattern

Modelled after opentelemetry-instrumentation-openai-v2's post-util-genai refactor (litellm's response shape is OpenAI-compatible, so the extractors transfer cleanly). Divergences:

• No client-instance introspection — litellm's entry points are module-level. server.address / server.port come from api_base / base_url kwargs when the caller uses a proxy.
• Provider resolution is per-call rather than a fixed constant.
• Only the "v_new" util-genai path is implemented; no legacy _v_old variant.

Verification

• Ruff: clean under repo pyproject.toml config (ruff check + ruff format --check).

• Pylint 10.00/10 with repo .pylintrc.

• 12 hermetic unit tests pass on Python 3.12 with the workspace opentelemetry-util-genai installed. Tests stub litellm in sys.modules so they don't need the real package or network credentials.

• 40/40 live end-to-end assertions passed against real OpenAI + Anthropic HTTP endpoints with litellm==1.52.0. Matrix:

  Scenario	OpenAI (gpt-4o-mini)	Anthropic (claude-haiku-4-5)
  Sync completion (non-streaming)	✅ 7/7 assertions	✅ 7/7 assertions
  Sync streaming	✅ 4/4 (11 chunks reassembled)	✅ 4/4 (3 chunks)
  Async acompletion (non-streaming)	✅ 2/2	✅ 2/2
  Async streaming	✅ 2/2 (9 chunks)	✅ 2/2 (3 chunks)
  Embedding (text-embedding-3-small, dim 1536)	✅ 5/5	n/a — Anthropic has no embedding API
  Error path (real 401 from invalid key)	✅ 3/3 — span ERROR + error.type set	—
  Metrics aggregate	✅ both gen_ai.client.operation.duration + gen_ai.client.token.usage populated

  Multi-provider gen_ai.provider.name resolution verified end-to-end: OpenAI calls surface openai, Anthropic calls surface anthropic, both derived from the real _hidden_params["custom_llm_provider"] populated by litellm at response time.

• Shape audit against real litellm 1.52.0:

  • ModelResponse.choices[0].message.{role,content} — pydantic attributes ✅
  • .usage.prompt_tokens / .completion_tokens — ✅
  • ._hidden_params["custom_llm_provider"] — dict, populated at 5+ call sites in litellm source ✅
  • get_llm_provider(model) — returns (model, provider, api_key, api_base) ✅
  • EmbeddingResponse.data[0] is a dict (not object) — handled via get_property_value
  • Streaming: StreamingChoices.delta.{role,content,tool_calls} — ✅

Repo-wide changes

• tox.ini: added litellm-{oldest,latest} factors, lint-instrumentation-litellm, and test-instrumentation-litellm / lint-instrumentation-litellm commands mirroring openai-v2's shape. Added to the typecheck env's dep list.
• .github/workflows/{test,lint,core_contrib_test}.yml: regenerated via tox -e generate-workflows — adds 30 test-matrix jobs (5 python × oldest/latest × 3 refs each) and a lint job.
• .github/workflows/package-{release,prepare-release,prepare-patch-release}.yml: added opentelemetry-instrumentation-litellm to the release choice list.
• .github/component_owners.yml: added ownership entry (currently pointing at yucheng-berri while this lives on the fork; upstream would want a maintainer here).
• pyproject.toml (repo root): added to dev-dependencies, uv workspace sources, pyright include list, and pyright exclude for tests/examples.
• uv.lock: regenerated via uv lock; adds litellm 1.91.0 and transitive deps.
• scripts/generate_instrumentation_bootstrap.py: added to packages_to_exclude (early-dev, same pattern as anthropic + claude-agent-sdk) and independent_packages.
• docs/index.rst: added the editable-install pip line.
• docs/instrumentation-genai/litellm.rst: new auto-module doc file.

Coverage of higher-level litellm surface

The wrappers sit on litellm.{completion,acompletion,embedding,aembedding} at the module root. Higher-level litellm APIs delegate through those functions and are therefore instrumented incidentally, without needing their own wrappers:

API	Coverage
litellm.completion / acompletion / embedding / aembedding	✅ direct wrapping
litellm.batch_completion (non-vLLM path)	✅ incidental — fans out to litellm.completion via ThreadPoolExecutor.submit(litellm.completion, ...), so a batch of N messages produces N spans
Router.completion / Router.acompletion	✅ incidental — Router._completion calls litellm.completion at router.py:722
Router.embedding / Router.aembedding	✅ incidental — Router._embedding calls litellm.embedding/aembedding at router.py:2154/2224
litellm.batch_completion (vLLM path)	❌ bypasses litellm.completion and calls vllm.batch_completions directly — no telemetry today
Aggregate parent span for batch_completion	❌ not emitted — individual calls each get their own span, but there is no wrapping span for the batch as a whole

Known caveats

• pyright under repo strict mode surfaces ~128 reportUnknownXxx warnings, which is in line with (and below) other genai packages included in the pyright include list. Not treated as blocking.

Assisted-by: Claude Opus 4.7

🤖 Generated with Claude Code