Skip to content

DRAFT feature(llmobs): enabling llmobs annotate() and custom span processor to modify a wider scope of span parameters#17176

Draft
bharbron wants to merge 1 commit intoDataDog:mainfrom
bharbron:bharbron/llmobs-expanded-span-processing
Draft

DRAFT feature(llmobs): enabling llmobs annotate() and custom span processor to modify a wider scope of span parameters#17176
bharbron wants to merge 1 commit intoDataDog:mainfrom
bharbron:bharbron/llmobs-expanded-span-processing

Conversation

@bharbron
Copy link
Copy Markdown

Description

Expands the scope of LLMObs span fields that can be read and mutated — both via LLMObs.annotate() and via custom span processors registered with enable() or register_processor().

LLMObs.annotate() new parameters: name, model_name, model_provider, session_id, ml_app

LLMObsSpan (span processor object) new fields: name, model_name, model_provider, session_id, ml_app, metadata, metrics, tool_definitions, prompt, and _span_kind (read-only via `get_span_kind()``). All mutable fields are written back to the outgoing span event.

Motivation: Auto-instrumentation currently captures a wide range of span data but exposes only a narrow slice of it for user modification. This gap creates two common pain points:

  1. Model aliasing (e.g. AWS Bedrock application inference profiles): Bedrock best practice is to call models via application inference profile IDs (e.g. arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/a1b2c3d4e5f6) for cost attribution. Auto-instrumentation records this opaque ID ("a1b2c3d4e5f6") as the model name and "custom" as the provider, breaking Datadog's token-to-dollar-cost estimates. Users now have a first-class way to override model_name and model_provider with the underlying model identity while keeping auto-instrumentation intact.
  2. Post-response span enrichment: In cases where span annotations depend on the LLM response (e.g. tagging a span end_conversation:true based on response content), auto-instrumented spans are already committed before user code can inspect the result. A custom span processor is the natural solution, but was previously too limited in what it could modify. Users can now mutate the full set of span fields from a processor.

In both cases, manual instrumentation was technically possible but undesirable — it requires reimplementing conversion logic (Bedrock JSON → LLMObs message format, tool schema normalization, token count normalization, etc.) that already exists inside the auto-instrumentation integration and is not exposed as public helpers.

Testing

Unit tests added in tests/llmobs/test_llmobs.py covering:

  • Processor read/mutate round-trips for each new LLMObsSpan field (metadata, metrics, tool_definitions, model_name, model_provider, session_id, ml_app, name)
  • Read-only enforcement for span_kind
  • annotate() parameter tests for each new argument (name, model_name, model_provider, session_id, ml_app)
  • name vs _name precedence and backwards-compatibility

Risks

  • _name backwards compatibility: The existing private _name parameter on annotate() is preserved; name takes precedence when both are provided. Existing callers using _name are unaffected.
  • Write-back side effects: Processor mutations to model_provider are lowercased before write-back, consistent with existing normalization. Processors that set an empty string for name, session_id, or ml_app fall back to the original value (the or fallback guards in the event builder).

Additional Notes

_span_kind is intentionally read-only (exposed via get_span_kind()). Allowing processors to change the span kind would silently break I/O serialization (LLM vs. embedding vs. retrieval shapes differ), so mutation of that field is ignored.

@bharbron bharbron force-pushed the bharbron/llmobs-expanded-span-processing branch from 0ffd81e to 52f3eae Compare March 27, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant