Skip to content

FR: add request model to LLM metric attributes - #24

Merged
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
fr-metrics-request-model-labels
May 22, 2026
Merged

FR: add request model to LLM metric attributes#24
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
fr-metrics-request-model-labels

Conversation

@sayonfortify

@sayonfortify sayonfortify commented May 22, 2026

Copy link
Copy Markdown

Description
This PR fixes a Metrics-tab consistency issue found during the Dashboard Provider Cost / GroupBy RCA validation.

After PR-A2 backend enrichment, Metrics Provider group-by correctly uses canonical fortifyroot.provider, but Metrics Model group-by can only canonicalize to the same model values as Events/Traces if metric datapoints include the request model. Several instrumentors only emitted gen_ai.response.model on metric datapoints. For routed/provider-backed calls, response model can be dated or provider-specific, while Events/Traces use request-model normalization.

Adds gen_ai.request.model to direct LLM metric emitters currently found in the fork: OpenAI, Anthropic, Google GenAI, Bedrock, and LangChain/LangGraph. LiteLLM/LlamaIndex are covered through underlying provider instrumentation when they do not emit native LLM metrics themselves.

What changed

  • Adds gen_ai.request.model to LLM token/duration metric attributes for MVP metric emitters:
    • OpenAI
    • Anthropic
    • Google GenAI
    • Bedrock
    • LangChain / LangGraph OpenAI path
  • Keeps existing gen_ai.response.model attributes unchanged.
  • Does not touch SDK vendored files directly; this change belongs in the OpenLLMetry fork and will be vendored into the SDK after tagging.
  • Does not add billing_provider; that remains PR-C scope.

Why
This lets backend metric enrichment use the same request-model semantics as LLM Events/Traces/Dashboard, so Metrics tab Provider/Model group-by does not drift into raw response-model labels like dated model IDs.

Validation

  • git diff --check
  • py_compile on all touched files
  • Focused tests:
    • OpenAI metrics: 5 passed
    • Anthropic completion/message metrics: 4 passed
    • Google GenAI metric: 1 passed
    • Bedrock metric: 1 passed
    • LangGraph metric: 1 passed

Next
After merge, tag fork release fr-v0.52.6.14, then vendor that tag into fortifyroot-sdk-py.

@sayonfortify
sayonfortify force-pushed the fr-metrics-request-model-labels branch from 9e41c1a to 491c305 Compare May 22, 2026 14:12
@sayonfortify
sayonfortify merged commit 2850fbb into fr-v0.52.6.x May 22, 2026
9 checks passed
@sayonfortify
sayonfortify deleted the fr-metrics-request-model-labels branch May 22, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant