Skip to content

feat(tracer): add DD_TRACE_OTEL_SEMANTICS_ENABLED for OTLP export#4889

Open
link04 wants to merge 1 commit into
mainfrom
maximo/otel-trace-compatibility
Open

feat(tracer): add DD_TRACE_OTEL_SEMANTICS_ENABLED for OTLP export#4889
link04 wants to merge 1 commit into
mainfrom
maximo/otel-trace-compatibility

Conversation

@link04

@link04 link04 commented Jun 12, 2026

Copy link
Copy Markdown

What does this PR do?

Adds an opt-in DD_TRACE_OTEL_SEMANTICS_ENABLED that aligns OTLP-exported spans (and the OTel bridge) with the pure OpenTelemetry SDK by suppressing Datadog-specific attributes and stopping DD-convention remaps. Mirrors dd-trace-py #18495 and libdatadog #2091.

Behavior when DD_TRACE_OTEL_SEMANTICS_ENABLED=true

  • Omit operation.name, resource.name, span.type (DD-only span attributes)
  • Omit error.message/type/stack/handling_stack from the span (they live on the exception span event for Error Tracking)
  • Omit span.kind (already carried by the first-class OTLP SpanKind field)
  • Keep http.response.status_code under its OTel key instead of remapping to the legacy http.status_code string tag
  • The OTel bridge no longer interprets Datadog reserved tags (operation.name, analytics.event, http.response.status_code) in SetAttributes — they pass through as provided
  • SetName updates the Datadog resource (which the OTLP exporter emits as the span name) instead of the DD-only operation name

Default (flag off) behavior is unchanged.

Why SetName is gated here (moved from #4887)

Changing SetName to update the resource is the correct OTel-aligned behavior, but doing it unconditionally is a breaking change: it shifts both the operation name and resource name, which changes users' RED metrics. Per team discussion (matching the dd-trace-js decision), this belongs behind the semantics flag rather than as a default change in a minor release. This supersedes #4887.

Performance

The flag is read once per export batch in otlpTraceWriter.add() and resolved once at provider creation for the bridge (cached on the oteltracer), then threaded as a parameter / read as a field — no per-span/per-attribute global config lookup.

Testing

TestConvertSpanAttributesOtelSemantics, TestRemapStatusCodeOtelSemantics, TestToReservedAttributesOtelSemantics, and TestSpanSetNameUpdatesResourceOtelSemantics (flag on) plus the existing TestSpanSetName (flag off, legacy operation-name behavior). Full ddtrace/tracer, ddtrace/opentelemetry, and internal/config suites pass.

🤖 Generated with Claude Code

@datadog-prod-us1-5

This comment has been minimized.

@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.28571% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.67%. Comparing base (08a41d6) to head (b23cf4e).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
ddtrace/tracer/span_to_otlp.go 53.84% 3 Missing and 3 partials ⚠️
internal/config/config.go 20.00% 4 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
ddtrace/opentelemetry/span.go 90.05% <100.00%> (-2.65%) ⬇️
ddtrace/opentelemetry/tracer.go 94.50% <ø> (ø)
ddtrace/opentelemetry/tracer_provider.go 100.00% <100.00%> (ø)
ddtrace/tracer/otlp_writer.go 96.42% <100.00%> (-1.17%) ⬇️
internal/config/config.go 59.70% <20.00%> (-4.78%) ⬇️
ddtrace/tracer/span_to_otlp.go 88.28% <53.84%> (-4.38%) ⬇️

... and 287 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter

pr-commenter Bot commented Jun 12, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-06-16 15:26:51

Comparing candidate commit 6dda781 in PR branch maximo/otel-trace-compatibility with baseline commit 08a41d6 in branch main.

Found 3 performance improvements and 0 performance regressions! Performance is the same for 294 metrics, 2 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:BenchmarkOTLPTraceWriterConcurrent/concurrency_2

  • 🟩 execution_time [-272.079ns; -203.321ns] or [-3.438%; -2.569%]

scenario:BenchmarkOTLPTraceWriterConcurrent/concurrency_8

  • 🟩 execution_time [-1112.144ns; -766.256ns] or [-3.668%; -2.527%]

scenario:BenchmarkPayloadVersions/simple_1000spans/v1.0

  • 🟩 execution_time [-3.865µs; -3.473µs] or [-2.498%; -2.244%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

  • scenario:BenchmarkOTLPTraceWriterFlush

Comment thread ddtrace/opentelemetry/span.go Outdated
Comment thread ddtrace/opentelemetry/span.go
@link04 link04 force-pushed the maximo/otel-trace-compatibility branch 3 times, most recently from 4194e3a to b23cf4e Compare June 12, 2026 20:59
Opt-in mode that aligns OTLP-exported spans with the pure OpenTelemetry SDK by
suppressing Datadog-specific attributes and stopping DD-convention remaps.
Mirrors dd-trace-py #18495 and libdatadog #2091.

When DD_TRACE_OTEL_SEMANTICS_ENABLED=true:
- omit operation.name, resource.name, span.type (DD-only span attributes)
- omit error.message/type/stack/handling_stack from the span (they live on the
  exception span event for Error Tracking)
- omit span.kind (already carried by the first-class OTLP SpanKind field)
- keep http.response.status_code under its OTel key instead of remapping to the
  legacy http.status_code string tag
- the OTel bridge no longer interprets Datadog reserved tags (operation.name,
  analytics.event, http.response.status_code) in SetAttributes
- SetName updates the Datadog resource (the OTLP span name) instead of the DD-only
  operation name; gated here because changing it by default shifts RED metrics

Registers the key in supported_configurations so env.Get resolves it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@link04 link04 force-pushed the maximo/otel-trace-compatibility branch from b23cf4e to 6dda781 Compare June 16, 2026 15:02
@link04 link04 marked this pull request as ready for review June 16, 2026 15:08
@link04 link04 requested review from a team as code owners June 16, 2026 15:08

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6dda781634

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

func toReservedAttributes(k string, v attribute.Value, otelSemantics bool) (string, any) {
if otelSemantics {
// Under OTel semantics, set every attribute as-is (no DD reserved-tag remapping).
return k, v.AsInterface()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve OTel attrs instead of DD pseudo-tags

With DD_TRACE_OTEL_SEMANTICS_ENABLED=true, this path stores keys such as resource.name, service.name, span.type, and operation.name unchanged in s.attributes, but End still forwards every entry through s.DD.SetTag, where those exact names are Datadog pseudo-tags rather than regular attributes. For example, SetAttributes(attribute.String("resource.name", "x")) changes the exported OTLP span name/resource and emits no resource.name attribute, so the new flag does not actually preserve OTel attributes as-is for these inputs.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants