DRAFT feature(llmobs): enabling llmobs annotate() and custom span processor to modify a wider scope of span parameters#17176
Draft
bharbron wants to merge 1 commit intoDataDog:mainfrom
Conversation
… to modify a wider scope of span parameters
0ffd81e to
52f3eae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Expands the scope of LLMObs span fields that can be read and mutated — both via
LLMObs.annotate()and via custom span processors registered withenable()orregister_processor().LLMObs.annotate()new parameters:name,model_name,model_provider,session_id,ml_appLLMObsSpan(span processor object) new fields:name,model_name,model_provider,session_id,ml_app,metadata,metrics,tool_definitions,prompt, and_span_kind(read-only via `get_span_kind()``). All mutable fields are written back to the outgoing span event.Motivation: Auto-instrumentation currently captures a wide range of span data but exposes only a narrow slice of it for user modification. This gap creates two common pain points:
arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/a1b2c3d4e5f6) for cost attribution. Auto-instrumentation records this opaque ID ("a1b2c3d4e5f6") as the model name and "custom" as the provider, breaking Datadog's token-to-dollar-cost estimates. Users now have a first-class way to override model_name and model_provider with the underlying model identity while keeping auto-instrumentation intact.end_conversation:truebased on response content), auto-instrumented spans are already committed before user code can inspect the result. A custom span processor is the natural solution, but was previously too limited in what it could modify. Users can now mutate the full set of span fields from a processor.In both cases, manual instrumentation was technically possible but undesirable — it requires reimplementing conversion logic (Bedrock JSON → LLMObs message format, tool schema normalization, token count normalization, etc.) that already exists inside the auto-instrumentation integration and is not exposed as public helpers.
Testing
Unit tests added in
tests/llmobs/test_llmobs.pycovering:LLMObsSpanfield (metadata,metrics,tool_definitions,model_name,model_provider,session_id,ml_app,name)annotate()parameter tests for each new argument (name,model_name,model_provider,session_id,ml_app)namevs_nameprecedence and backwards-compatibilityRisks
_namebackwards compatibility: The existing private_nameparameter on annotate() is preserved; name takes precedence when both are provided. Existing callers using_nameare unaffected.Additional Notes
_span_kindis intentionally read-only (exposed viaget_span_kind()). Allowing processors to change the span kind would silently break I/O serialization (LLM vs. embedding vs. retrieval shapes differ), so mutation of that field is ignored.