Python: Consolidate OTel GenAI Semantic Conventions versions - #7673
Python: Consolidate OTel GenAI Semantic Conventions versions#7673Tao Chen (TaoChenOSU) wants to merge 1 commit into
Conversation
Tao Chen (TaoChenOSU)
added
python
GitHub Actions
Active
GitHub Actions
Active
GitHub Actions
Active
github-actions
Bot
changed the title
agent-framework-automation
Bot
added
the
documentation
There was a problem hiding this comment.
Pull request overview
Adds configurable OpenTelemetry GenAI semantic-convention versioning to Python observability.
Changes:
- Adds stable/experimental convention and message-event settings.
- Gates provider, message, and tool-call telemetry attributes.
- Adds documentation and tests for version-specific behavior.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
python/samples/02-agents/observability/README.md |
Documents semantic-convention settings. |
python/packages/core/tests/core/test_tools.py |
Tests stable tool telemetry. |
python/packages/core/tests/core/test_observability.py |
Tests convention selection and message capture. |
python/packages/core/tests/conftest.py |
Resets new environment variables. |
python/packages/core/agent_framework/observability.py |
Implements convention-aware telemetry. |
python/packages/core/agent_framework/_tools.py |
Gates experimental tool attributes. |
python/packages/core/agent_framework/_harness/_background_agents.py |
Applies formatting-only changes. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
| json.dumps(otel_messages, ensure_ascii=False), | ||
| ) | ||
| emit_events = OBSERVABILITY_SETTINGS.enable_message_events | ||
| emit_span_attribute = OBSERVABILITY_SETTINGS.use_latest_experimental_gen_ai_semconv |
| | Provider-identifying attribute | `gen_ai.system` | `gen_ai.provider.name` | | ||
| | Tool call arguments/results on `execute_tool` spans | Not emitted (introduced in v1.38.0) | `gen_ai.tool.call.arguments` / `gen_ai.tool.call.result` | | ||
|
|
||
| `invoke_agent` spans always use `INTERNAL` span kind (the OTel default), regardless of semconv version. The v1.41.0 spec defines `CLIENT` for agents that are themselves a remote service and `INTERNAL` for agents that run in-process (no `server.address`/`server.port`/token-usage attributes, since the actual network call happens on a nested `chat` span instead). Agent Framework's own agents run in-process — `agent.run()` orchestrates a locally-running chat client, which creates its own nested `chat` span for the actual network call — so `INTERNAL` applies uniformly, without needing to classify each agent implementation across packages. What's **not yet covered** by this flag is the rest of the v1.41.0 attribute-group split: under the conventions above v1.36.0, the `invoke_agent` client span is defined to drop `gen_ai.response.id`, `gen_ai.response.model`, and `gen_ai.response.finish_reasons` and add `gen_ai.agent.version` instead. Agent Framework still emits the former three unconditionally on `invoke_agent` spans and does not emit `gen_ai.agent.version` at all under either semconv version. |
| # Prerequisite for both examples below: capture message content at all. | ||
| ENABLE_SENSITIVE_DATA=true | ||
|
|
||
| # Opt into the stable v1.36.0 conventions only (events, no gen_ai.input.messages/output.messages span attributes) | ||
| OTEL_SEMCONV_STABILITY_OPT_IN="" | ||
|
|
||
| # Conventions above v1.36.0 only, no message events (strict spec compliance) | ||
| ENABLE_MESSAGE_EVENTS=false |
| for index, message in enumerate(normalized_messages): | ||
| # Reuse the otel message representation for logging instead of calling to_dict() | ||
| # to avoid expensive Pydantic serialization overhead | ||
| otel_message = _to_otel_message(message) |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
MAF Automated Review — Iteration 1
Result: Findings reported
Scope: full PR (1 commit(s)): 8c429189690c
Model: gpt-5.6-sol
Overview
The PR centralizes GenAI semantic-convention selection, preserves the existing experimental default, and gates sensitive tool payloads and message attributes with focused regression coverage. Provider-name switching and comma-separated opt-in parsing are well guarded. Two gaps remain in stable mode: output events do not satisfy the v1.36 choice-event contract, and spans can still contain attributes from newer conventions.
Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
2 verified findings remained after source verification (2 medium) across 1 file. Details are attached to the affected lines below.
Affected areas: python/packages/core/agent_framework/observability.py
| otel_message = _to_otel_message(message) | ||
| if emit_events: | ||
| logger.info( | ||
| otel_message, |
There was a problem hiding this comment.
In stable mode this object becomes the sole gen_ai.choice payload, but _to_otel_message() produces the experimental {role, parts} shape rather than the v1.36 choice body with index, finish_reason, and nested message. Stable collectors therefore cannot decode assistant outputs according to the selected contract. Please construct the role-specific v1.36 event body here, including the mapped finish reason for output choices, while retaining the existing representation for experimental span attributes.
| json.dumps(otel_messages, ensure_ascii=False), | ||
| ) | ||
| emit_events = OBSERVABILITY_SETTINGS.enable_message_events | ||
| emit_span_attribute = OBSERVABILITY_SETTINGS.use_latest_experimental_gen_ai_semconv |
There was a problem hiding this comment.
This switch only gates input/output message attributes; stable mode still emits newer attributes through _capture_system_instructions() and the unconditional tool definitions in _get_span_attributes(). A caller selecting v1.36 can therefore receive mixed-version spans that strict schema consumers cannot process consistently. Please apply the selected semconv version to all attributes introduced after v1.36, not only message and tool-call payload attributes.
Motivation & Context
Description & Review Guide
Related Issue
Fixes #7657
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.