Skip to content

fix(weave): align integration otel keys and retain instrumentation scope - #7645

Open
gtarpenning wants to merge 4 commits into
masterfrom
griffin/otel-scope-attribution
Open

fix(weave): align integration otel keys and retain instrumentation scope#7645
gtarpenning wants to merge 4 commits into
masterfrom
griffin/otel-scope-attribution

Conversation

@gtarpenning

@gtarpenning gtarpenning commented Jul 28, 2026

Copy link
Copy Markdown
Member

Summary

  • Python as_otel_attributes() emitted integration.* while the node SDK emits weave.integration.*, so a consumer keyed on either spelling missed the other SDK's spans. Python now emits the weave.integration.* form AGENTS.md:322 documents as canonical.
  • Instrumentation scope was parsed and then discarded for every ingested span. ScopeSpans.from_proto was the only site threading it, and it is reachable only from TracesData.from_proto, which has no non-test callers; all three real ingest loops dropped it. iter_proto_spans now pairs each span with its enclosing scope in one place, so the three loops cannot disagree.
  • Scope absence is a Scope | None mirroring resource rather than "", so "sender omitted the scope" and "sender sent an empty scope" stay distinguishable in the persisted otel_dump. This adds attributes.otel_span.scope as a permanent user-visible key.
  • gen_ai_aliases -> wire_aliases: it already held non-gen_ai keys (ERROR_TYPE takes error.type), so the name carries the contract instead of a docstring carve-out.

No server-side ladder accepting both spellings: no weave server code consumes either key, so converging the producers now means a consumer needs one key rather than two forever. Note the old spelling is already persisted as custom attrs in real projects, so those will list both until the old spans age out, and a saved filter on integration.name stops matching new spans. #7644 should rebase and drop its dual-key rung.

Testing

The scope assertion runs through the real otel_export path, not TracesData.from_proto; reverting the threading in clickhouse_trace_server_batched.py fails it with assert None == {...}. A parity test reads the node literal out of sdks/node/src/genai/semconv.ts so the SDKs cannot drift again; the import-linter forbids sharing the constant, so the literals are restated per the google_adk/_semconv.py precedent.

Four independent bugs on the flattened OTel attribution path.

Python `as_otel_attributes()` emitted `integration.*` while the node SDK
emits `weave.integration.*`, so a consumer keyed on either spelling
silently missed the other SDK's spans. Python now emits the
`weave.integration.*` form `AGENTS.md` already documents as canonical.
The literals are restated locally because the import-linter forbids
sharing them, following the `google_adk/_semconv.py` precedent, and a
parity test reads the node literal so the two cannot drift again.

`ScopeSpans.from_proto` set `scope` on the container but built its spans
via `Span.from_proto(s, resource)`, so the instrumentation scope was
parsed and then discarded for every span. It now threads onto `Span` and
into `otel_dump`, the only route by which a call retains it.

The two docstrings described behavior that does not exist: the trace
server never rebuilt a nested `attributes["integration"]` dict, and
`gen_ai_aliases` already holds non-`gen_ai.*` keys via `ERROR_TYPE`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@w-b-hivemind

w-b-hivemind Bot commented Jul 28, 2026

Copy link
Copy Markdown

HiveMind Sessions

1 session · 6m · $2.24

Session Agent Duration Tokens Cost Lines
Fix OTel Integration Provenance Attribute Bugs
019fa9be-c75e-7b70-9f30-a941c9b5d70d
codex 6m 125.1K $2.24 +107 -33
Total 6m 125.1K $2.24 +107 -33

View all sessions in HiveMind →

Run codex resume 019fa9be-c75e-7b70-9f30-a941c9b5d70d to pickup where you left off.

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.86364% with 23 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
weave/trace_server/in_memory_trace_server.py 5.00% 19 Missing ⚠️
...ve/trace_server/clickhouse_trace_server_batched.py 80.95% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

gtarpenning and others added 3 commits July 28, 2026 11:04
… paths

ScopeSpans.from_proto was the only site threading scope onto Span, and it is
reachable only from TracesData.from_proto, which has no non-test callers. All
three real ingest loops walked scope_spans themselves and dropped the scope,
so no ingested span carried its instrumentation-library identity.

Adds iter_proto_spans so the walk pairs each span with its enclosing scope in
one place, and replaces the flat scope_name/scope_version "" sentinels with a
Scope dataclass mirroring Resource, so an omitted scope persists as null
instead of being indistinguishable from an empty one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gen_ai_aliases already held non-gen_ai keys (ERROR_TYPE takes error.type), so
the name carries the contract instead of a docstring carve-out that goes stale
on the next non-gen_ai alias.

Drops comments that narrate where a change came from rather than describe
current behavior, and removes the as_otel_attributes claim about the trace
server not re-nesting these keys: unflatten_key_values nests every dotted key,
so the claim was wrong as well as out of place.

Also removes the parity test's pytest.skip, which would silently drop the
cross-SDK drift guard in any checkout without the node SDK.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gtarpenning
gtarpenning marked this pull request as ready for review July 28, 2026 22:03
@gtarpenning
gtarpenning requested a review from a team as a code owner July 28, 2026 22:03

@jtschoonhoven jtschoonhoven left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want a stamp from the sdk team on this, but LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants