Skip to content

fix(litellm): copy streaming latency attrs to canonical attempt span - #41

Merged
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
fix/litellm-streaming-latency-attempt-span
Jun 20, 2026
Merged

fix(litellm): copy streaming latency attrs to canonical attempt span#41
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
fix/litellm-streaming-latency-attempt-span

Conversation

@sayonfortify

Copy link
Copy Markdown

Summary

LiteLLM streaming TTFT/STTG is measured on FortifyRoot's fortifyroot.litellm.safety wrapper span, but retry-aware extraction treats fortifyroot.litellm.attempt_N as the canonical LLM usage span. In live fixture validation, the wrapper and attempt spans can be processed separately, so backend parent inheritance is not sufficient.

This change copies wrapper-owned streaming latency attrs onto the active canonical attempt span:

  • fortifyroot.llm.streaming.time_to_first_token_ms
  • fortifyroot.llm.streaming.time_to_generate_ms

The streaming wrapper accepts an optional canonical-span setter; the LiteLLM instrumentor supplies a setter that finds active attempt spans under the safety wrapper, sets the attrs immediately, and stores them so finalization re-applies them before span end.

Validation

  • python3 -m py_compile on changed source/tests
  • conda run -n py3.12 python -m pytest packages/opentelemetry-instrumentation-litellm/tests/test_safety_hooks.py packages/opentelemetry-instrumentation-litellm/tests/test_retry_attempt_emission.py -k 'streaming_wrapper_sets_streaming_latency_attrs or single_attempt_emits_one_retry_attempt_under_parent' -q
    • 3 passed, 42 deselected
  • Local vendored ocelle mirror + fr-system-tests dev validation:
    • FR_PROVIDER_MODE=fixture ./scripts/run-tests.sh --telemetry -- -k 'streaming_latency_litellm_single_canonical_usage_event or streaming_latency_llamaindex_single_canonical_usage_event' -rs
    • analytics_roundtrip 2 passed, 29 deselected; overall 13 passed, 0 failed

Notes

No provider calls/cost in the system-test validation: provider mode was fixture. The system-test failure that motivated this showed canonical LiteLLM usage rows existed, but TTFT/STTG were absent until these attrs were copied to the canonical attempt span.

@sayonfortify
sayonfortify merged commit 02adf49 into fr-v0.52.6.x Jun 20, 2026
10 checks passed
@sayonfortify
sayonfortify deleted the fix/litellm-streaming-latency-attempt-span branch June 20, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant