Skip to content

feat(llamaindex): emit streaming latency span attrs - #40

Merged
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
feat/llamaindex-streaming-latency-attrs
Jun 19, 2026
Merged

feat(llamaindex): emit streaming latency span attrs#40
sayonfortify merged 1 commit into
fr-v0.52.6.xfrom
feat/llamaindex-streaming-latency-attrs

Conversation

@sayonfortify

Copy link
Copy Markdown

Adds FortifyRoot TTFT/STTG span attributes to the existing LlamaIndex streaming safety wrapper.

What changed:

  • LlamaIndexStreamingSafety now records:
    • fortifyroot.llm.streaming.time_to_first_token_ms on the first output-bearing streamed delta
    • fortifyroot.llm.streaming.time_to_generate_ms at stream completion, only if a first token arrived
  • Sync and async stream wrappers call those hooks while preserving the existing pending-item safety/flush behavior.
  • Tests cover hook dispatch, integer-ms attr values, and empty-stream no-attribute behavior.

Notes:

  • For LlamaIndex/OpenAI delegated calls, the backend usage row is expected to come from the child OpenAI provider span (the wrapper span is intentionally marked llm_wrapper and skipped by the extractor).
  • For non-delegated LlamaIndex provider paths, these wrapper attrs provide the RDS extraction source.

Validation:

  • python3 -m py_compile .../streaming_safety.py .../tests/test_streaming_safety.py
  • python3 scripts/apply-fr-modification-notices.py --check
  • conda py3.12 editable install of this package, plus focused timing smoke: TTFT=125ms and STTG=1125ms were set on the span.

Local full pytest caveat:

  • pytest packages/opentelemetry-instrumentation-llamaindex/tests/test_streaming_safety.py -q is blocked in the shared py3.12 env before test collection because this checkout lacks the package test extras (chromadb, vector-store/cohere extras, onnxruntime). CI should run the repository test environment.

@sayonfortify
sayonfortify force-pushed the feat/llamaindex-streaming-latency-attrs branch from 8c57ab1 to 3d1b6b9 Compare June 19, 2026 17:55
@sayonfortify
sayonfortify merged commit 01b2d02 into fr-v0.52.6.x Jun 19, 2026
10 checks passed
@sayonfortify
sayonfortify deleted the feat/llamaindex-streaming-latency-attrs branch June 19, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant