feat(openfeature): add flag evaluation metrics by sameerank · Pull Request #17029 · DataDog/dd-trace-py

sameerank · 2026-03-19T20:58:50Z

Description

Adds feature_flag.evaluations OTel counter metric emitted on every flag evaluation, following the Go implementation pattern.

Requirements:

DD_METRICS_OTEL_ENABLED=true must be set for metrics to emit
openfeature-sdk>=0.8.0 (required for finally_after hook to receive evaluation details)

Changes:

Implements FlagEvalMetrics class and FlagEvalHook for metrics tracking
Fixes flag not found behavior to return Reason.ERROR with ErrorCode.FLAG_NOT_FOUND when flag is not in existing config (aligns with Go/iOS SDKs)
Returns Reason.DEFAULT only when no configuration is loaded (preserving existing behavior)

Testing

Unit test parity with Go SDK

The following tests mirror the Go SDK flageval_metrics_test.go:

Go Test	Python Test
`TestRecord` (targeting match)	`test_record_basic_attributes`
`TestRecord` (allocation key)	`test_record_with_allocation_key`
`TestRecord` (empty allocation key)	`test_record_empty_allocation_key_not_included`
`TestRecord` (error flag not found)	`test_record_with_error`
`TestRecord` (disabled flag)	`test_record_disabled_reason`
`TestRecordMultipleEvaluations`	`test_record_multiple_evaluations`
`TestRecordDifferentFlags`	`test_record_different_flags`
`TestRecordAllErrorTypes`	`test_record_all_error_types`
`TestIntegrationEvaluate` (type mismatch)	`test_type_conversion_error_records_type_mismatch`

Note: Go tests use a real OTel test meter provider. Python unit tests (TestFlagEvalMetrics) use mocks for faster isolated testing, while TestMetricsWithRealOTel validates behavior with real OTel runtime.

Python-specific tests (not in Go)

TestFlagEvalMetrics: OTel initialization tests (graceful handling when OTel not available), metrics disabled when DD_METRICS_OTEL_ENABLED=false, shutdown behavior
TestFlagEvalHook: Tests the hook mechanism (finally_after calls metrics.record with correct arguments)
TestProviderHooksIntegration: Tests provider hook registration, get_provider_hooks() returns correct hooks, cleanup on shutdown
TestMetricsWithRealOTel: Integration tests with real OTel runtime

System tests

System tests PR: [FFL-1942] feat(ffe): add eval metrics tests [python@main] [golang@sameerank/FFL-1972/fix-flag-evaluation-metrics-consistency-issues] system-tests#6545 (python tests passing)

Risks

API behavior change: Flag evaluations for non-existent flags now return Reason.ERROR instead of Reason.DEFAULT when configuration is available. Release note added.
Dependency upgrade: Minimum openfeature-sdk version increased from 0.6.0 to 0.8.0. Users on older versions will need to upgrade. Release note added.

Additional Notes

Reference files in Go SDK: flageval_metrics.go, flageval_metrics_test.go, provider.go, provider_test.go

cit-pr-commenter-54b7da · 2026-03-19T21:04:02Z

Codeowners resolved as

.riot/requirements/14fc413.txt                                          @DataDog/apm-python
.riot/requirements/1540c33.txt                                          @DataDog/apm-python
.riot/requirements/168ee03.txt                                          @DataDog/apm-python
.riot/requirements/18421e5.txt                                          @DataDog/apm-python
.riot/requirements/765862d.txt                                          @DataDog/apm-python
.riot/requirements/cdab08a.txt                                          @DataDog/apm-python
ddtrace/internal/openfeature/_flageval_metrics.py                       @DataDog/feature-flagging-and-experimentation-sdk
ddtrace/internal/openfeature/_provider.py                               @DataDog/feature-flagging-and-experimentation-sdk
ddtrace/internal/openfeature/_remoteconfiguration.py                    @DataDog/feature-flagging-and-experimentation-sdk
releasenotes/notes/openfeature-flag-not-found-error-d31bde19aa5e373f.yaml  @DataDog/apm-python
riotfile.py                                                             @DataDog/apm-python
tests/openfeature/test_flag_eval_metrics.py                             @DataDog/feature-flagging-and-experimentation-sdk
tests/openfeature/test_provider.py                                      @DataDog/feature-flagging-and-experimentation-sdk

ddtrace/internal/openfeature/_flageval_metrics.py

ddtrace/internal/openfeature/_provider.py

tests/openfeature/test_flag_eval_metrics.py

tests/openfeature/test_provider.py

ddtrace/internal/openfeature/_provider.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f4e7b65fe9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

ddtrace/internal/openfeature/_flageval_metrics.py

pr-commenter · 2026-03-20T17:45:12Z

Performance SLOs

Comparing candidate sameerank/FFL-1942/add-flag-eval-metrics (0951d14) with baseline main (ecc75ad)

📈 Performance Regressions (2 suites)

📈 iastaspects - 118/118

✅ add_aspect

Time: ✅ 103.332µs (SLO: <130.000µs 📉 -20.5%) vs baseline: +2.2%

Memory: ✅ 43.911MB (SLO: <46.000MB -4.5%) vs baseline: +5.0%

✅ add_inplace_aspect

Time: ✅ 100.563µs (SLO: <130.000µs 📉 -22.6%) vs baseline: -0.4%

Memory: ✅ 44.064MB (SLO: <46.000MB -4.2%) vs baseline: +5.2%

✅ add_inplace_noaspect

Time: ✅ 28.067µs (SLO: <40.000µs 📉 -29.8%) vs baseline: ~same

Memory: ✅ 43.972MB (SLO: <46.000MB -4.4%) vs baseline: +5.0%

✅ add_noaspect

Time: ✅ 49.610µs (SLO: <70.000µs 📉 -29.1%) vs baseline: +1.8%

Memory: ✅ 43.947MB (SLO: <46.000MB -4.5%) vs baseline: +4.7%

✅ bytearray_aspect

Time: ✅ 248.816µs (SLO: <400.000µs 📉 -37.8%) vs baseline: -0.2%

Memory: ✅ 43.991MB (SLO: <46.000MB -4.4%) vs baseline: +5.0%

✅ bytearray_extend_aspect

Time: ✅ 650.192µs (SLO: <800.000µs 📉 -18.7%) vs baseline: -1.8%

Memory: ✅ 43.946MB (SLO: <46.000MB -4.5%) vs baseline: +4.8%

✅ bytearray_extend_noaspect

Time: ✅ 268.532µs (SLO: <400.000µs 📉 -32.9%) vs baseline: +1.7%

Memory: ✅ 43.851MB (SLO: <46.000MB -4.7%) vs baseline: +4.5%

✅ bytearray_noaspect

Time: ✅ 138.274µs (SLO: <300.000µs 📉 -53.9%) vs baseline: +0.1%

Memory: ✅ 44.051MB (SLO: <46.000MB -4.2%) vs baseline: +5.2%

✅ bytes_aspect

Time: ✅ 216.055µs (SLO: <300.000µs 📉 -28.0%) vs baseline: -0.8%

Memory: ✅ 43.874MB (SLO: <46.000MB -4.6%) vs baseline: +4.8%

✅ bytes_noaspect

Time: ✅ 133.468µs (SLO: <200.000µs 📉 -33.3%) vs baseline: +1.3%

Memory: ✅ 43.949MB (SLO: <46.000MB -4.5%) vs baseline: +5.1%

✅ bytesio_aspect

Time: ✅ 3.778ms (SLO: <5.000ms 📉 -24.4%) vs baseline: +0.2%

Memory: ✅ 43.850MB (SLO: <46.000MB -4.7%) vs baseline: +4.0%

✅ bytesio_noaspect

Time: ✅ 316.473µs (SLO: <420.000µs 📉 -24.6%) vs baseline: +0.9%

Memory: ✅ 43.835MB (SLO: <46.000MB -4.7%) vs baseline: +5.0%

✅ capitalize_aspect

Time: ✅ 88.053µs (SLO: <300.000µs 📉 -70.6%) vs baseline: +0.1%

Memory: ✅ 43.946MB (SLO: <46.000MB -4.5%) vs baseline: +5.0%

✅ capitalize_noaspect

Time: ✅ 248.220µs (SLO: <300.000µs 📉 -17.3%) vs baseline: +1.3%

Memory: ✅ 43.970MB (SLO: <46.000MB -4.4%) vs baseline: +4.8%

✅ casefold_aspect

Time: ✅ 88.170µs (SLO: <500.000µs 📉 -82.4%) vs baseline: +0.5%

Memory: ✅ 43.950MB (SLO: <46.000MB -4.5%) vs baseline: +5.2%

✅ casefold_noaspect

Time: ✅ 303.965µs (SLO: <500.000µs 📉 -39.2%) vs baseline: +0.6%

Memory: ✅ 43.874MB (SLO: <46.000MB -4.6%) vs baseline: +5.0%

✅ decode_aspect

Time: ✅ 86.970µs (SLO: <100.000µs 📉 -13.0%) vs baseline: +0.6%

Memory: ✅ 44.016MB (SLO: <46.000MB -4.3%) vs baseline: +4.8%

✅ decode_noaspect

Time: ✅ 153.130µs (SLO: <210.000µs 📉 -27.1%) vs baseline: +0.3%

Memory: ✅ 43.929MB (SLO: <46.000MB -4.5%) vs baseline: +4.9%

✅ encode_aspect

Time: ✅ 84.098µs (SLO: <200.000µs 📉 -58.0%) vs baseline: -0.9%

Memory: ✅ 43.919MB (SLO: <46.000MB -4.5%) vs baseline: +4.8%

✅ encode_noaspect

Time: ✅ 141.192µs (SLO: <200.000µs 📉 -29.4%) vs baseline: +1.0%

Memory: ✅ 43.947MB (SLO: <46.000MB -4.5%) vs baseline: +5.2%

✅ format_aspect

Time: ✅ 14.560ms (SLO: <19.200ms 📉 -24.2%) vs baseline: -0.5%

Memory: ✅ 44.015MB (SLO: <46.000MB -4.3%) vs baseline: +4.7%

✅ format_map_aspect

Time: ✅ 16.431ms (SLO: <21.500ms 📉 -23.6%) vs baseline: +0.6%

Memory: ✅ 44.102MB (SLO: <46.000MB -4.1%) vs baseline: +5.0%

✅ format_map_noaspect

Time: ✅ 375.012µs (SLO: <500.000µs 📉 -25.0%) vs baseline: +0.5%

Memory: ✅ 44.015MB (SLO: <46.000MB -4.3%) vs baseline: +5.1%

✅ format_noaspect

Time: ✅ 303.694µs (SLO: <500.000µs 📉 -39.3%) vs baseline: +0.5%

Memory: ✅ 43.836MB (SLO: <46.000MB -4.7%) vs baseline: +4.8%

✅ index_aspect

Time: ✅ 137.253µs (SLO: <300.000µs 📉 -54.2%) vs baseline: 📈 +11.4%

Memory: ✅ 43.986MB (SLO: <46.000MB -4.4%) vs baseline: +5.1%

✅ index_noaspect

Time: ✅ 40.502µs (SLO: <300.000µs 📉 -86.5%) vs baseline: -0.7%

Memory: ✅ 44.006MB (SLO: <46.000MB -4.3%) vs baseline: +5.3%

✅ join_aspect

Time: ✅ 208.389µs (SLO: <300.000µs 📉 -30.5%) vs baseline: -0.5%

Memory: ✅ 43.972MB (SLO: <46.000MB -4.4%) vs baseline: +5.2%

✅ join_noaspect

Time: ✅ 144.616µs (SLO: <300.000µs 📉 -51.8%) vs baseline: +0.8%

Memory: ✅ 44.047MB (SLO: <46.000MB -4.2%) vs baseline: +5.2%

✅ ljust_aspect

Time: ✅ 502.339µs (SLO: <700.000µs 📉 -28.2%) vs baseline: +0.1%

Memory: ✅ 43.916MB (SLO: <46.000MB -4.5%) vs baseline: +4.8%

✅ ljust_noaspect

Time: ✅ 270.667µs (SLO: <300.000µs -9.8%) vs baseline: +6.6%

Memory: ✅ 43.900MB (SLO: <46.000MB -4.6%) vs baseline: +4.8%

✅ lower_aspect

Time: ✅ 296.668µs (SLO: <500.000µs 📉 -40.7%) vs baseline: +0.8%

Memory: ✅ 43.909MB (SLO: <46.000MB -4.5%) vs baseline: +4.8%

✅ lower_noaspect

Time: ✅ 231.808µs (SLO: <300.000µs 📉 -22.7%) vs baseline: +1.0%

Memory: ✅ 43.913MB (SLO: <46.000MB -4.5%) vs baseline: +4.9%

✅ lstrip_aspect

Time: ✅ 0.273ms (SLO: <3.000ms 📉 -90.9%) vs baseline: -2.0%

Memory: ✅ 43.879MB (SLO: <46.000MB -4.6%) vs baseline: +5.1%

✅ lstrip_noaspect

Time: ✅ 0.177ms (SLO: <3.000ms 📉 -94.1%) vs baseline: +0.3%

Memory: ✅ 44.126MB (SLO: <46.000MB -4.1%) vs baseline: +5.6%

✅ modulo_aspect

Time: ✅ 14.267ms (SLO: <18.750ms 📉 -23.9%) vs baseline: ~same

Memory: ✅ 44.170MB (SLO: <46.000MB -4.0%) vs baseline: +5.2%

✅ modulo_aspect_for_bytearray_bytearray

Time: ✅ 14.764ms (SLO: <19.350ms 📉 -23.7%) vs baseline: -0.6%

Memory: ✅ 44.061MB (SLO: <46.000MB -4.2%) vs baseline: +4.8%

✅ modulo_aspect_for_bytes

Time: ✅ 14.323ms (SLO: <18.900ms 📉 -24.2%) vs baseline: -0.4%

Memory: ✅ 44.064MB (SLO: <46.000MB -4.2%) vs baseline: +4.6%

✅ modulo_aspect_for_bytes_bytearray

Time: ✅ 14.605ms (SLO: <19.150ms 📉 -23.7%) vs baseline: -0.4%

Memory: ✅ 44.050MB (SLO: <46.000MB -4.2%) vs baseline: +4.7%

✅ modulo_noaspect

Time: ✅ 0.356ms (SLO: <3.000ms 📉 -88.1%) vs baseline: ~same

Memory: ✅ 43.933MB (SLO: <46.000MB -4.5%) vs baseline: +5.0%

✅ replace_aspect

Time: ✅ 18.411ms (SLO: <24.000ms 📉 -23.3%) vs baseline: ~same

Memory: ✅ 44.027MB (SLO: <46.000MB -4.3%) vs baseline: +4.8%

✅ replace_noaspect

Time: ✅ 281.227µs (SLO: <400.000µs 📉 -29.7%) vs baseline: +0.1%

Memory: ✅ 43.971MB (SLO: <46.000MB -4.4%) vs baseline: +5.0%

✅ repr_aspect

Time: ✅ 318.352µs (SLO: <420.000µs 📉 -24.2%) vs baseline: -0.3%

Memory: ✅ 43.830MB (SLO: <46.000MB -4.7%) vs baseline: +4.5%

✅ repr_noaspect

Time: ✅ 46.520µs (SLO: <90.000µs 📉 -48.3%) vs baseline: -0.7%

Memory: ✅ 43.834MB (SLO: <46.000MB -4.7%) vs baseline: +4.7%

✅ rstrip_aspect

Time: ✅ 387.434µs (SLO: <500.000µs 📉 -22.5%) vs baseline: +1.1%

Memory: ✅ 43.921MB (SLO: <46.000MB -4.5%) vs baseline: +5.1%

✅ rstrip_noaspect

Time: ✅ 180.033µs (SLO: <300.000µs 📉 -40.0%) vs baseline: -1.6%

Memory: ✅ 43.777MB (SLO: <46.000MB -4.8%) vs baseline: +4.5%

✅ slice_aspect

Time: ✅ 185.496µs (SLO: <300.000µs 📉 -38.2%) vs baseline: +0.9%

Memory: ✅ 43.849MB (SLO: <46.000MB -4.7%) vs baseline: +4.3%

✅ slice_noaspect

Time: ✅ 54.174µs (SLO: <90.000µs 📉 -39.8%) vs baseline: -1.4%

Memory: ✅ 43.991MB (SLO: <46.000MB -4.4%) vs baseline: +5.1%

✅ stringio_aspect

Time: ✅ 3.775ms (SLO: <5.000ms 📉 -24.5%) vs baseline: -0.6%

Memory: ✅ 43.811MB (SLO: <46.000MB -4.8%) vs baseline: +4.2%

✅ stringio_noaspect

Time: ✅ 377.413µs (SLO: <500.000µs 📉 -24.5%) vs baseline: +9.5%

Memory: ✅ 43.990MB (SLO: <46.000MB -4.4%) vs baseline: +5.1%

✅ strip_aspect

Time: ✅ 273.072µs (SLO: <350.000µs 📉 -22.0%) vs baseline: -0.9%

Memory: ✅ 43.929MB (SLO: <46.000MB -4.5%) vs baseline: +5.0%

✅ strip_noaspect

Time: ✅ 175.766µs (SLO: <240.000µs 📉 -26.8%) vs baseline: +0.6%

Memory: ✅ 43.889MB (SLO: <46.000MB -4.6%) vs baseline: +5.0%

✅ swapcase_aspect

Time: ✅ 331.825µs (SLO: <500.000µs 📉 -33.6%) vs baseline: +0.3%

Memory: ✅ 43.851MB (SLO: <46.000MB -4.7%) vs baseline: +4.4%

✅ swapcase_noaspect

Time: ✅ 267.762µs (SLO: <400.000µs 📉 -33.1%) vs baseline: +0.7%

Memory: ✅ 43.966MB (SLO: <46.000MB -4.4%) vs baseline: +4.9%

✅ title_aspect

Time: ✅ 325.854µs (SLO: <500.000µs 📉 -34.8%) vs baseline: +1.9%

Memory: ✅ 43.890MB (SLO: <46.000MB -4.6%) vs baseline: +4.7%

✅ title_noaspect

Time: ✅ 255.330µs (SLO: <400.000µs 📉 -36.2%) vs baseline: +1.3%

Memory: ✅ 43.814MB (SLO: <46.000MB -4.8%) vs baseline: +4.7%

✅ translate_aspect

Time: ✅ 493.693µs (SLO: <700.000µs 📉 -29.5%) vs baseline: +0.8%

Memory: ✅ 43.853MB (SLO: <46.000MB -4.7%) vs baseline: +4.2%

✅ translate_noaspect

Time: ✅ 424.230µs (SLO: <500.000µs 📉 -15.2%) vs baseline: -0.8%

Memory: ✅ 43.939MB (SLO: <46.000MB -4.5%) vs baseline: +5.3%

✅ upper_aspect

Time: ✅ 298.712µs (SLO: <500.000µs 📉 -40.3%) vs baseline: +1.0%

Memory: ✅ 43.890MB (SLO: <46.000MB -4.6%) vs baseline: +4.7%

✅ upper_noaspect

Time: ✅ 230.561µs (SLO: <400.000µs 📉 -42.4%) vs baseline: ~same

Memory: ✅ 43.938MB (SLO: <46.000MB -4.5%) vs baseline: +5.1%

📈 iastaspectsospath - 24/24

✅ ospathbasename_aspect

Time: ✅ 508.138µs (SLO: <700.000µs 📉 -27.4%) vs baseline: 📈 +20.0%

Memory: ✅ 43.612MB (SLO: <46.000MB -5.2%) vs baseline: +4.3%

✅ ospathbasename_noaspect

Time: ✅ 430.508µs (SLO: <700.000µs 📉 -38.5%) vs baseline: -0.3%

Memory: ✅ 43.960MB (SLO: <46.000MB -4.4%) vs baseline: +4.5%

✅ ospathjoin_aspect

Time: ✅ 623.095µs (SLO: <700.000µs 📉 -11.0%) vs baseline: ~same

Memory: ✅ 43.845MB (SLO: <46.000MB -4.7%) vs baseline: +5.2%

✅ ospathjoin_noaspect

Time: ✅ 635.568µs (SLO: <700.000µs -9.2%) vs baseline: +0.2%

Memory: ✅ 43.975MB (SLO: <46.000MB -4.4%) vs baseline: +4.9%

✅ ospathnormcase_aspect

Time: ✅ 350.456µs (SLO: <700.000µs 📉 -49.9%) vs baseline: -0.9%

Memory: ✅ 43.681MB (SLO: <46.000MB -5.0%) vs baseline: +3.9%

✅ ospathnormcase_noaspect

Time: ✅ 362.650µs (SLO: <700.000µs 📉 -48.2%) vs baseline: +0.7%

Memory: ✅ 44.085MB (SLO: <46.000MB -4.2%) vs baseline: +5.2%

✅ ospathsplit_aspect

Time: ✅ 482.183µs (SLO: <700.000µs 📉 -31.1%) vs baseline: ~same

Memory: ✅ 43.580MB (SLO: <46.000MB -5.3%) vs baseline: +5.0%

✅ ospathsplit_noaspect

Time: ✅ 497.175µs (SLO: <700.000µs 📉 -29.0%) vs baseline: +0.3%

Memory: ✅ 43.660MB (SLO: <46.000MB -5.1%) vs baseline: +4.2%

✅ ospathsplitdrive_aspect

Time: ✅ 371.000µs (SLO: <700.000µs 📉 -47.0%) vs baseline: -0.8%

Memory: ✅ 43.810MB (SLO: <46.000MB -4.8%) vs baseline: +4.3%

✅ ospathsplitdrive_noaspect

Time: ✅ 72.890µs (SLO: <700.000µs 📉 -89.6%) vs baseline: +0.2%

Memory: ✅ 43.842MB (SLO: <46.000MB -4.7%) vs baseline: +4.3%

✅ ospathsplitext_aspect

Time: ✅ 456.281µs (SLO: <700.000µs 📉 -34.8%) vs baseline: -0.2%

Memory: ✅ 43.760MB (SLO: <46.000MB -4.9%) vs baseline: +4.1%

✅ ospathsplitext_noaspect

Time: ✅ 466.177µs (SLO: <700.000µs 📉 -33.4%) vs baseline: -0.3%

Memory: ✅ 43.984MB (SLO: <46.000MB -4.4%) vs baseline: +4.8%

🟡 Near SLO Breach (3 suites)

🟡 djangosimple - 30/30

✅ appsec

Time: ✅ 19.664ms (SLO: <22.300ms 📉 -11.8%) vs baseline: ~same

Memory: ✅ 68.798MB (SLO: <73.500MB -6.4%) vs baseline: +5.1%

✅ exception-replay-enabled

Time: ✅ 1.401ms (SLO: <1.450ms -3.4%) vs baseline: -0.3%

Memory: ✅ 66.882MB (SLO: <71.500MB -6.5%) vs baseline: +5.0%

✅ iast

Time: ✅ 19.684ms (SLO: <22.250ms 📉 -11.5%) vs baseline: -0.1%

Memory: ✅ 68.669MB (SLO: <75.000MB -8.4%) vs baseline: +4.8%

✅ profiler

Time: ✅ 15.014ms (SLO: <16.550ms -9.3%) vs baseline: -0.8%

Memory: ✅ 60.445MB (SLO: <61.000MB 🟡 -0.9%) vs baseline: +4.8%

✅ resource-renaming

Time: ✅ 19.599ms (SLO: <21.750ms -9.9%) vs baseline: -0.1%

Memory: ✅ 68.874MB (SLO: <73.500MB -6.3%) vs baseline: +5.2%

✅ span-code-origin

Time: ✅ 20.232ms (SLO: <28.200ms 📉 -28.3%) vs baseline: +2.3%

Memory: ✅ 68.755MB (SLO: <75.000MB -8.3%) vs baseline: +4.8%

✅ tracer

Time: ✅ 19.681ms (SLO: <21.750ms -9.5%) vs baseline: +0.3%

Memory: ✅ 68.824MB (SLO: <75.000MB -8.2%) vs baseline: +5.1%

✅ tracer-and-profiler

Time: ✅ 21.011ms (SLO: <23.500ms 📉 -10.6%) vs baseline: +0.6%

Memory: ✅ 70.949MB (SLO: <75.000MB -5.4%) vs baseline: +5.3%

✅ tracer-dont-create-db-spans

Time: ✅ 19.765ms (SLO: <21.500ms -8.1%) vs baseline: -0.2%

Memory: ✅ 68.802MB (SLO: <75.000MB -8.3%) vs baseline: +4.8%

✅ tracer-minimal

Time: ✅ 16.811ms (SLO: <17.500ms -3.9%) vs baseline: +0.2%

Memory: ✅ 68.817MB (SLO: <75.000MB -8.2%) vs baseline: +5.0%

✅ tracer-native

Time: ✅ 19.566ms (SLO: <21.750ms 📉 -10.0%) vs baseline: -0.5%

Memory: ✅ 68.805MB (SLO: <72.500MB -5.1%) vs baseline: +5.0%

✅ tracer-no-caches

Time: ✅ 17.613ms (SLO: <19.650ms 📉 -10.4%) vs baseline: +0.4%

Memory: ✅ 68.794MB (SLO: <75.000MB -8.3%) vs baseline: +5.1%

✅ tracer-no-databases

Time: ✅ 19.318ms (SLO: <20.100ms -3.9%) vs baseline: -0.1%

Memory: ✅ 68.869MB (SLO: <75.000MB -8.2%) vs baseline: +5.0%

✅ tracer-no-middleware

Time: ✅ 19.321ms (SLO: <21.500ms 📉 -10.1%) vs baseline: -0.3%

Memory: ✅ 68.765MB (SLO: <75.000MB -8.3%) vs baseline: +4.9%

✅ tracer-no-templates

Time: ✅ 19.551ms (SLO: <22.000ms 📉 -11.1%) vs baseline: +0.5%

Memory: ✅ 68.899MB (SLO: <73.500MB -6.3%) vs baseline: +5.2%

🟡 otelsdkspan - 24/24

✅ add-event

Time: ✅ 40.435ms (SLO: <42.000ms -3.7%) vs baseline: ~same

Memory: ✅ 39.046MB (SLO: <40.750MB -4.2%) vs baseline: +4.9%

✅ add-link

Time: ✅ 36.596ms (SLO: <38.550ms -5.1%) vs baseline: +0.2%

Memory: ✅ 39.184MB (SLO: <40.750MB -3.8%) vs baseline: +5.0%

✅ add-metrics

Time: ✅ 217.245ms (SLO: <232.000ms -6.4%) vs baseline: -0.9%

Memory: ✅ 39.243MB (SLO: <40.750MB -3.7%) vs baseline: +5.2%

✅ add-tags

Time: ✅ 211.196ms (SLO: <221.600ms -4.7%) vs baseline: -0.8%

Memory: ✅ 39.066MB (SLO: <40.750MB -4.1%) vs baseline: +4.8%

✅ get-context

Time: ✅ 29.233ms (SLO: <31.300ms -6.6%) vs baseline: +0.7%

Memory: ✅ 39.105MB (SLO: <40.750MB -4.0%) vs baseline: +4.8%

✅ is-recording

Time: ✅ 29.205ms (SLO: <31.000ms -5.8%) vs baseline: -0.4%

Memory: ✅ 39.184MB (SLO: <40.750MB -3.8%) vs baseline: +4.7%

✅ record-exception

Time: ✅ 63.272ms (SLO: <65.850ms -3.9%) vs baseline: ~same

Memory: ✅ 39.086MB (SLO: <40.750MB -4.1%) vs baseline: +4.8%

✅ set-status

Time: ✅ 31.800ms (SLO: <34.150ms -6.9%) vs baseline: -0.4%

Memory: ✅ 39.105MB (SLO: <40.750MB -4.0%) vs baseline: +4.6%

✅ start

Time: ✅ 29.578ms (SLO: <30.150ms 🟡 -1.9%) vs baseline: +1.5%

Memory: ✅ 39.105MB (SLO: <40.750MB -4.0%) vs baseline: +4.8%

✅ start-finish

Time: ✅ 34.025ms (SLO: <35.350ms -3.7%) vs baseline: +1.1%

Memory: ✅ 39.125MB (SLO: <40.750MB -4.0%) vs baseline: +5.1%

✅ start-finish-telemetry

Time: ✅ 34.000ms (SLO: <35.450ms -4.1%) vs baseline: ~same

Memory: ✅ 39.125MB (SLO: <40.750MB -4.0%) vs baseline: +4.9%

✅ update-name

Time: ✅ 31.201ms (SLO: <33.400ms -6.6%) vs baseline: ~same

Memory: ✅ 39.105MB (SLO: <40.750MB -4.0%) vs baseline: +4.8%

🟡 recursivecomputation - 8/8

✅ deep

Time: ✅ 311.011ms (SLO: <320.950ms -3.1%) vs baseline: ~same

Memory: ✅ 37.473MB (SLO: <38.750MB -3.3%) vs baseline: +4.9%

✅ deep-profiled

Time: ✅ 333.008ms (SLO: <359.150ms -7.3%) vs baseline: +0.2%

Memory: ✅ 43.726MB (SLO: <46.000MB -4.9%) vs baseline: +4.7%

✅ medium

Time: ✅ 7.280ms (SLO: <7.400ms 🟡 -1.6%) vs baseline: ~same

Memory: ✅ 36.235MB (SLO: <38.000MB -4.6%) vs baseline: +4.7%

✅ shallow

Time: ✅ 1.014ms (SLO: <1.050ms -3.4%) vs baseline: +1.6%

Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +4.9%

⚠️ Unstable Tests (2 suites)

⚠️

coreapiscenario - 10/10 (1 unstable)

⚠️ context_with_data_listeners

Time: ⚠️ 13.654µs (SLO: <20.000µs 📉 -31.7%) vs baseline: +0.7%

Memory: ✅ 36.235MB (SLO: <38.000MB -4.6%) vs baseline: +4.7%

✅ context_with_data_no_listeners

Time: ✅ 3.593µs (SLO: <10.000µs 📉 -64.1%) vs baseline: -0.8%

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +4.9%

✅ get_item_exists

Time: ✅ 0.580µs (SLO: <10.000µs 📉 -94.2%) vs baseline: -0.3%

Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +4.9%

✅ get_item_missing

Time: ✅ 0.636µs (SLO: <10.000µs 📉 -93.6%) vs baseline: -0.4%

Memory: ✅ 36.333MB (SLO: <38.000MB -4.4%) vs baseline: +4.5%

✅ set_item

Time: ✅ 24.754µs (SLO: <30.000µs 📉 -17.5%) vs baseline: +0.9%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.0%

⚠️

packagesupdateimporteddependencies - 24/24 (1 unstable)

✅ import_many

Time: ✅ 155.422µs (SLO: <170.000µs -8.6%) vs baseline: +0.3%

Memory: ✅ 41.207MB (SLO: <46.000MB 📉 -10.4%) vs baseline: +4.8%

✅ import_many_cached

Time: ✅ 121.179µs (SLO: <130.000µs -6.8%) vs baseline: -0.3%

Memory: ✅ 41.177MB (SLO: <46.000MB 📉 -10.5%) vs baseline: +4.9%

✅ import_many_stdlib

Time: ✅ 0.783ms (SLO: <1.750ms 📉 -55.3%) vs baseline: +0.2%

Memory: ✅ 41.104MB (SLO: <46.000MB 📉 -10.6%) vs baseline: +4.6%

⚠️ import_many_stdlib_cached

Time: ⚠️ 0.178ms (SLO: <1.100ms 📉 -83.9%) vs baseline: +0.3%

Memory: ✅ 41.288MB (SLO: <46.000MB 📉 -10.2%) vs baseline: +5.0%

✅ import_many_unknown

Time: ✅ 837.047µs (SLO: <890.000µs -5.9%) vs baseline: -0.5%

Memory: ✅ 41.185MB (SLO: <46.000MB 📉 -10.5%) vs baseline: +4.1%

✅ import_many_unknown_cached

Time: ✅ 789.958µs (SLO: <870.000µs -9.2%) vs baseline: +0.5%

Memory: ✅ 41.396MB (SLO: <46.000MB 📉 -10.0%) vs baseline: +5.0%

✅ import_one

Time: ✅ 19.728µs (SLO: <30.000µs 📉 -34.2%) vs baseline: -0.8%

Memory: ✅ 41.142MB (SLO: <46.000MB 📉 -10.6%) vs baseline: +4.8%

✅ import_one_cache

Time: ✅ 6.292µs (SLO: <10.000µs 📉 -37.1%) vs baseline: -0.5%

Memory: ✅ 41.205MB (SLO: <46.000MB 📉 -10.4%) vs baseline: +4.7%

✅ import_one_stdlib

Time: ✅ 18.732µs (SLO: <20.000µs -6.3%) vs baseline: +0.3%

Memory: ✅ 41.021MB (SLO: <46.000MB 📉 -10.8%) vs baseline: +4.6%

✅ import_one_stdlib_cache

Time: ✅ 6.258µs (SLO: <10.000µs 📉 -37.4%) vs baseline: ~same

Memory: ✅ 41.226MB (SLO: <46.000MB 📉 -10.4%) vs baseline: +4.9%

✅ import_one_unknown

Time: ✅ 45.491µs (SLO: <50.000µs -9.0%) vs baseline: ~same

Memory: ✅ 41.144MB (SLO: <46.000MB 📉 -10.6%) vs baseline: +4.8%

✅ import_one_unknown_cache

Time: ✅ 6.298µs (SLO: <10.000µs 📉 -37.0%) vs baseline: +0.3%

Memory: ✅ 41.207MB (SLO: <43.000MB -4.2%) vs baseline: +4.8%

✅ All Tests Passing (19 suites)

✅ codeprovenancefork - 2/2

✅ fork-10

Time: ✅ 2.140s (SLO: <2.300s -7.0%) vs baseline: +1.9%

Memory: ✅ 17.380MB (SLO: <20.000MB 📉 -13.1%) vs baseline: +4.7%

✅ errortrackingdjangosimple - 6/6

✅ errortracking-enabled-all

Time: ✅ 16.355ms (SLO: <19.850ms 📉 -17.6%) vs baseline: +0.2%

Memory: ✅ 68.675MB (SLO: <75.000MB -8.4%) vs baseline: +4.9%

✅ errortracking-enabled-user

Time: ✅ 16.342ms (SLO: <19.400ms 📉 -15.8%) vs baseline: ~same

Memory: ✅ 68.695MB (SLO: <75.000MB -8.4%) vs baseline: +4.9%

✅ tracer-enabled

Time: ✅ 16.324ms (SLO: <19.450ms 📉 -16.1%) vs baseline: -0.1%

Memory: ✅ 68.616MB (SLO: <75.000MB -8.5%) vs baseline: +5.3%

✅ errortrackingflasksqli - 6/6

✅ errortracking-enabled-all

Time: ✅ 2.082ms (SLO: <2.300ms -9.5%) vs baseline: -0.2%

Memory: ✅ 56.053MB (SLO: <60.000MB -6.6%) vs baseline: +4.7%

✅ errortracking-enabled-user

Time: ✅ 2.085ms (SLO: <2.250ms -7.3%) vs baseline: -0.1%

Memory: ✅ 55.994MB (SLO: <60.000MB -6.7%) vs baseline: +4.7%

✅ tracer-enabled

Time: ✅ 2.081ms (SLO: <2.300ms -9.5%) vs baseline: +0.2%

Memory: ✅ 55.581MB (SLO: <60.000MB -7.4%) vs baseline: +4.6%

✅ flasksimple - 18/18

✅ appsec-get

Time: ✅ 3.371ms (SLO: <4.750ms 📉 -29.0%) vs baseline: +0.5%

Memory: ✅ 55.847MB (SLO: <66.500MB 📉 -16.0%) vs baseline: +5.0%

✅ appsec-post

Time: ✅ 2.869ms (SLO: <6.750ms 📉 -57.5%) vs baseline: +0.5%

Memory: ✅ 56.001MB (SLO: <66.500MB 📉 -15.8%) vs baseline: +4.6%

✅ appsec-telemetry

Time: ✅ 3.358ms (SLO: <4.750ms 📉 -29.3%) vs baseline: +0.4%

Memory: ✅ 55.971MB (SLO: <66.500MB 📉 -15.8%) vs baseline: +5.0%

✅ debugger

Time: ✅ 1.876ms (SLO: <2.000ms -6.2%) vs baseline: ~same

Memory: ✅ 49.230MB (SLO: <51.500MB -4.4%) vs baseline: +4.9%

✅ iast-get

Time: ✅ 1.867ms (SLO: <2.000ms -6.6%) vs baseline: ~same

Memory: ✅ 46.052MB (SLO: <49.000MB -6.0%) vs baseline: +5.1%

✅ profiler

Time: ✅ 1.912ms (SLO: <2.100ms -9.0%) vs baseline: -0.4%

Memory: ✅ 52.316MB (SLO: <53.500MB -2.2%) vs baseline: +4.6%

✅ resource-renaming

Time: ✅ 3.334ms (SLO: <3.650ms -8.7%) vs baseline: -0.1%

Memory: ✅ 55.832MB (SLO: <60.000MB -6.9%) vs baseline: +4.9%

✅ tracer

Time: ✅ 3.351ms (SLO: <3.650ms -8.2%) vs baseline: +0.4%

Memory: ✅ 55.922MB (SLO: <60.000MB -6.8%) vs baseline: +4.9%

✅ tracer-native

Time: ✅ 3.343ms (SLO: <3.650ms -8.4%) vs baseline: +0.2%

Memory: ✅ 55.978MB (SLO: <60.000MB -6.7%) vs baseline: +5.1%

✅ flasksqli - 6/6

✅ appsec-enabled

Time: ✅ 2.073ms (SLO: <4.200ms 📉 -50.6%) vs baseline: -0.5%

Memory: ✅ 55.679MB (SLO: <66.000MB 📉 -15.6%) vs baseline: +4.6%

✅ iast-enabled

Time: ✅ 2.078ms (SLO: <2.800ms 📉 -25.8%) vs baseline: -0.2%

Memory: ✅ 55.758MB (SLO: <62.500MB 📉 -10.8%) vs baseline: +5.0%

✅ tracer-enabled

Time: ✅ 2.068ms (SLO: <2.250ms -8.1%) vs baseline: -0.2%

Memory: ✅ 55.719MB (SLO: <60.000MB -7.1%) vs baseline: +4.9%

✅ forktime - 4/4

✅ baseline

Time: ✅ 1.940ms (SLO: <3.000ms 📉 -35.3%) vs baseline: +3.8%

Memory: ✅ 29.236MB (SLO: <33.000MB 📉 -11.4%) vs baseline: +4.9%

✅ configured

Time: ✅ 8.714ms (SLO: <13.000ms 📉 -33.0%) vs baseline: +0.4%

Memory: ✅ 56.050MB (SLO: <60.000MB -6.6%) vs baseline: +4.9%

✅ httppropagationextract - 60/60

✅ all_styles_all_headers

Time: ✅ 80.401µs (SLO: <100.000µs 📉 -19.6%) vs baseline: +4.6%

Memory: ✅ 36.392MB (SLO: <38.000MB -4.2%) vs baseline: +4.9%

✅ b3_headers

Time: ✅ 12.806µs (SLO: <20.000µs 📉 -36.0%) vs baseline: +0.8%

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +5.0%

✅ b3_single_headers

Time: ✅ 11.939µs (SLO: <20.000µs 📉 -40.3%) vs baseline: +0.6%

Memory: ✅ 36.313MB (SLO: <38.000MB -4.4%) vs baseline: +4.9%

✅ datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match

Time: ✅ 61.935µs (SLO: <80.000µs 📉 -22.6%) vs baseline: ~same

Memory: ✅ 36.451MB (SLO: <38.000MB -4.1%) vs baseline: +4.9%

✅ datadog_tracecontext_tracestate_propagated_on_trace_id_match

Time: ✅ 63.324µs (SLO: <80.000µs 📉 -20.8%) vs baseline: +0.3%

Memory: ✅ 36.431MB (SLO: <38.000MB -4.1%) vs baseline: +5.1%

✅ empty_headers

Time: ✅ 1.302µs (SLO: <10.000µs 📉 -87.0%) vs baseline: +0.9%

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +5.1%

✅ full_t_id_datadog_headers

Time: ✅ 21.096µs (SLO: <30.000µs 📉 -29.7%) vs baseline: -0.9%

Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +4.7%

✅ invalid_priority_header

Time: ✅ 5.916µs (SLO: <10.000µs 📉 -40.8%) vs baseline: +0.4%

Memory: ✅ 36.294MB (SLO: <38.000MB -4.5%) vs baseline: +5.1%

✅ invalid_span_id_header

Time: ✅ 5.870µs (SLO: <10.000µs 📉 -41.3%) vs baseline: -0.3%

Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +4.8%

✅ invalid_tags_header

Time: ✅ 5.872µs (SLO: <10.000µs 📉 -41.3%) vs baseline: -0.5%

Memory: ✅ 36.274MB (SLO: <38.000MB -4.5%) vs baseline: +4.9%

✅ invalid_trace_id_header

Time: ✅ 5.917µs (SLO: <10.000µs 📉 -40.8%) vs baseline: +0.7%

Memory: ✅ 36.392MB (SLO: <38.000MB -4.2%) vs baseline: +5.4%

✅ large_header_no_matches

Time: ✅ 26.927µs (SLO: <30.000µs 📉 -10.2%) vs baseline: +0.6%

Memory: ✅ 36.235MB (SLO: <38.000MB -4.6%) vs baseline: +4.5%

✅ large_valid_headers_all

Time: ✅ 27.946µs (SLO: <40.000µs 📉 -30.1%) vs baseline: ~same

Memory: ✅ 36.313MB (SLO: <38.000MB -4.4%) vs baseline: +4.7%

✅ medium_header_no_matches

Time: ✅ 9.291µs (SLO: <20.000µs 📉 -53.5%) vs baseline: +0.4%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.2%

✅ medium_valid_headers_all

Time: ✅ 10.685µs (SLO: <20.000µs 📉 -46.6%) vs baseline: +0.3%

Memory: ✅ 36.392MB (SLO: <38.000MB -4.2%) vs baseline: +5.0%

✅ none_propagation_style

Time: ✅ 1.396µs (SLO: <10.000µs 📉 -86.0%) vs baseline: ~same

Memory: ✅ 36.431MB (SLO: <38.000MB -4.1%) vs baseline: +5.2%

✅ tracecontext_headers

Time: ✅ 32.858µs (SLO: <40.000µs 📉 -17.9%) vs baseline: ~same

Memory: ✅ 36.333MB (SLO: <38.000MB -4.4%) vs baseline: +4.7%

✅ valid_headers_all

Time: ✅ 5.908µs (SLO: <10.000µs 📉 -40.9%) vs baseline: +1.0%

Memory: ✅ 36.392MB (SLO: <38.000MB -4.2%) vs baseline: +5.0%

✅ valid_headers_basic

Time: ✅ 5.486µs (SLO: <10.000µs 📉 -45.1%) vs baseline: +0.5%

Memory: ✅ 36.313MB (SLO: <38.000MB -4.4%) vs baseline: +4.9%

✅ wsgi_empty_headers

Time: ✅ 1.304µs (SLO: <10.000µs 📉 -87.0%) vs baseline: +0.3%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.3%

✅ wsgi_invalid_priority_header

Time: ✅ 5.942µs (SLO: <10.000µs 📉 -40.6%) vs baseline: +0.3%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.2%

✅ wsgi_invalid_span_id_header

Time: ✅ 1.309µs (SLO: <10.000µs 📉 -86.9%) vs baseline: +0.4%

Memory: ✅ 36.313MB (SLO: <38.000MB -4.4%) vs baseline: +4.8%

✅ wsgi_invalid_tags_header

Time: ✅ 5.987µs (SLO: <10.000µs 📉 -40.1%) vs baseline: +0.7%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.2%

✅ wsgi_invalid_trace_id_header

Time: ✅ 5.932µs (SLO: <10.000µs 📉 -40.7%) vs baseline: -0.1%

Memory: ✅ 36.333MB (SLO: <38.000MB -4.4%) vs baseline: +5.0%

✅ wsgi_large_header_no_matches

Time: ✅ 28.114µs (SLO: <40.000µs 📉 -29.7%) vs baseline: ~same

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +4.3%

✅ wsgi_large_valid_headers_all

Time: ✅ 29.235µs (SLO: <40.000µs 📉 -26.9%) vs baseline: ~same

Memory: ✅ 36.412MB (SLO: <38.000MB -4.2%) vs baseline: +5.5%

✅ wsgi_medium_header_no_matches

Time: ✅ 9.446µs (SLO: <20.000µs 📉 -52.8%) vs baseline: -0.1%

Memory: ✅ 36.333MB (SLO: <38.000MB -4.4%) vs baseline: +5.1%

✅ wsgi_medium_valid_headers_all

Time: ✅ 11.061µs (SLO: <20.000µs 📉 -44.7%) vs baseline: +1.2%

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +4.6%

✅ wsgi_valid_headers_all

Time: ✅ 5.920µs (SLO: <10.000µs 📉 -40.8%) vs baseline: ~same

Memory: ✅ 36.294MB (SLO: <38.000MB -4.5%) vs baseline: +4.9%

✅ wsgi_valid_headers_basic

Time: ✅ 5.500µs (SLO: <10.000µs 📉 -45.0%) vs baseline: -0.8%

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +5.1%

✅ httppropagationinject - 16/16

✅ ids_only

Time: ✅ 20.837µs (SLO: <30.000µs 📉 -30.5%) vs baseline: +3.7%

Memory: ✅ 36.333MB (SLO: <38.000MB -4.4%) vs baseline: +4.8%

✅ with_all

Time: ✅ 27.105µs (SLO: <40.000µs 📉 -32.2%) vs baseline: -0.2%

Memory: ✅ 36.255MB (SLO: <38.000MB -4.6%) vs baseline: +4.6%

✅ with_dd_origin

Time: ✅ 23.919µs (SLO: <30.000µs 📉 -20.3%) vs baseline: ~same

Memory: ✅ 36.392MB (SLO: <38.000MB -4.2%) vs baseline: +5.1%

✅ with_priority_and_origin

Time: ✅ 23.168µs (SLO: <40.000µs 📉 -42.1%) vs baseline: -0.6%

Memory: ✅ 36.274MB (SLO: <38.000MB -4.5%) vs baseline: +4.9%

✅ with_sampling_priority

Time: ✅ 20.074µs (SLO: <30.000µs 📉 -33.1%) vs baseline: -0.5%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.2%

✅ with_tags

Time: ✅ 24.981µs (SLO: <40.000µs 📉 -37.5%) vs baseline: -1.0%

Memory: ✅ 36.313MB (SLO: <38.000MB -4.4%) vs baseline: +5.0%

✅ with_tags_invalid

Time: ✅ 26.389µs (SLO: <40.000µs 📉 -34.0%) vs baseline: -0.6%

Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.2%

✅ with_tags_max_size

Time: ✅ 25.460µs (SLO: <40.000µs 📉 -36.3%) vs baseline: -0.9%

Memory: ✅ 36.412MB (SLO: <38.000MB -4.2%) vs baseline: +5.1%

✅ iastaspectssplit - 12/12

✅ rsplit_aspect

Time: ✅ 160.548µs (SLO: <250.000µs 📉 -35.8%) vs baseline: +3.8%

Memory: ✅ 43.652MB (SLO: <46.000MB -5.1%) vs baseline: +4.6%

✅ rsplit_noaspect

Time: ✅ 156.220µs (SLO: <250.000µs 📉 -37.5%) vs baseline: -3.8%

Memory: ✅ 43.912MB (SLO: <46.000MB -4.5%) vs baseline: +5.4%

✅ split_aspect

Time: ✅ 152.690µs (SLO: <250.000µs 📉 -38.9%) vs baseline: +1.0%

Memory: ✅ 43.509MB (SLO: <46.000MB -5.4%) vs baseline: +4.4%

✅ split_noaspect

Time: ✅ 155.395µs (SLO: <250.000µs 📉 -37.8%) vs baseline: +0.5%

Memory: ✅ 43.909MB (SLO: <46.000MB -4.5%) vs baseline: +5.2%

✅ splitlines_aspect

Time: ✅ 150.603µs (SLO: <250.000µs 📉 -39.8%) vs baseline: +1.0%

Memory: ✅ 43.902MB (SLO: <46.000MB -4.6%) vs baseline: +5.3%

✅ splitlines_noaspect

Time: ✅ 154.943µs (SLO: <250.000µs 📉 -38.0%) vs baseline: +0.1%

Memory: ✅ 43.632MB (SLO: <46.000MB -5.1%) vs baseline: +5.0%

✅ iastpropagation - 8/8

✅ no-propagation

Time: ✅ 48.605µs (SLO: <60.000µs 📉 -19.0%) vs baseline: ~same

Memory: ✅ 40.088MB (SLO: <42.000MB -4.6%) vs baseline: +4.9%

✅ propagation_enabled

Time: ✅ 135.481µs (SLO: <190.000µs 📉 -28.7%) vs baseline: -0.6%

Memory: ✅ 40.108MB (SLO: <42.000MB -4.5%) vs baseline: +4.9%

✅ propagation_enabled_100

Time: ✅ 1.583ms (SLO: <2.300ms 📉 -31.2%) vs baseline: +1.0%

Memory: ✅ 40.108MB (SLO: <42.000MB -4.5%) vs baseline: +5.1%

✅ propagation_enabled_1000

Time: ✅ 29.202ms (SLO: <34.550ms 📉 -15.5%) vs baseline: -0.2%

Memory: ✅ 40.226MB (SLO: <42.000MB -4.2%) vs baseline: +5.3%

✅ otelspan - 22/22

✅ add-event

Time: ✅ 38.219ms (SLO: <47.150ms 📉 -18.9%) vs baseline: +0.5%

Memory: ✅ 41.315MB (SLO: <47.000MB 📉 -12.1%) vs baseline: +4.7%

✅ add-metrics

Time: ✅ 257.274ms (SLO: <344.800ms 📉 -25.4%) vs baseline: +0.2%

Memory: ✅ 45.696MB (SLO: <47.500MB -3.8%) vs baseline: +4.7%

✅ add-tags

Time: ✅ 305.476ms (SLO: <330.000ms -7.4%) vs baseline: +0.4%

Memory: ✅ 45.840MB (SLO: <47.500MB -3.5%) vs baseline: +4.8%

✅ get-context

Time: ✅ 80.663ms (SLO: <92.350ms 📉 -12.7%) vs baseline: ~same

Memory: ✅ 41.601MB (SLO: <46.500MB 📉 -10.5%) vs baseline: +4.8%

✅ is-recording

Time: ✅ 36.386ms (SLO: <44.500ms 📉 -18.2%) vs baseline: +0.3%

Memory: ✅ 41.392MB (SLO: <47.500MB 📉 -12.9%) vs baseline: +5.2%

✅ record-exception

Time: ✅ 56.666ms (SLO: <67.650ms 📉 -16.2%) vs baseline: ~same

Memory: ✅ 41.824MB (SLO: <47.000MB 📉 -11.0%) vs baseline: +5.2%

✅ set-status

Time: ✅ 43.195ms (SLO: <50.400ms 📉 -14.3%) vs baseline: +0.6%

Memory: ✅ 41.249MB (SLO: <47.000MB 📉 -12.2%) vs baseline: +4.8%

✅ start

Time: ✅ 37.183ms (SLO: <43.450ms 📉 -14.4%) vs baseline: +3.8%

Memory: ✅ 41.100MB (SLO: <47.000MB 📉 -12.6%) vs baseline: +4.4%

✅ start-finish

Time: ✅ 84.921ms (SLO: <90.000ms -5.6%) vs baseline: +0.5%

Memory: ✅ 38.830MB (SLO: <46.500MB 📉 -16.5%) vs baseline: +5.0%

✅ start-finish-telemetry

Time: ✅ 86.445ms (SLO: <91.000ms -5.0%) vs baseline: +0.1%

Memory: ✅ 38.732MB (SLO: <46.500MB 📉 -16.7%) vs baseline: +4.5%

✅ update-name

Time: ✅ 37.701ms (SLO: <45.150ms 📉 -16.5%) vs baseline: +0.6%

Memory: ✅ 41.304MB (SLO: <47.000MB 📉 -12.1%) vs baseline: +5.2%

✅ packagespackageforrootmodulemapping - 4/4

✅ cache_off

Time: ✅ 343.745ms (SLO: <354.300ms -3.0%) vs baseline: -0.4%

Memory: ✅ 41.949MB (SLO: <46.000MB -8.8%) vs baseline: +4.2%

✅ cache_on

Time: ✅ 0.382µs (SLO: <10.000µs 📉 -96.2%) vs baseline: ~same

Memory: ✅ 40.275MB (SLO: <46.000MB 📉 -12.4%) vs baseline: +5.3%

✅ rand - 2/2

✅ rand128bits

Time: ✅ 0.183µs (SLO: <21.000µs 📉 -99.1%) vs baseline: +0.3%

✅ rand64bits

Time: ✅ 0.122µs (SLO: <15.000µs 📉 -99.2%) vs baseline: ~same

✅ ratelimiter - 12/12

✅ defaults

Time: ✅ 2.349µs (SLO: <10.000µs 📉 -76.5%) vs baseline: +0.2%

Memory: ✅ 36.549MB (SLO: <38.000MB -3.8%) vs baseline: +4.6%

✅ high_rate_limit

Time: ✅ 2.409µs (SLO: <10.000µs 📉 -75.9%) vs baseline: +0.2%

Memory: ✅ 36.412MB (SLO: <38.000MB -4.2%) vs baseline: +4.1%

✅ long_window

Time: ✅ 2.363µs (SLO: <10.000µs 📉 -76.4%) vs baseline: -0.1%

Memory: ✅ 36.608MB (SLO: <38.000MB -3.7%) vs baseline: +4.7%

✅ low_rate_limit

Time: ✅ 2.373µs (SLO: <10.000µs 📉 -76.3%) vs baseline: +0.4%

Memory: ✅ 36.471MB (SLO: <38.000MB -4.0%) vs baseline: +4.3%

✅ no_rate_limit

Time: ✅ 0.841µs (SLO: <10.000µs 📉 -91.6%) vs baseline: +2.0%

Memory: ✅ 36.628MB (SLO: <38.000MB -3.6%) vs baseline: +4.9%

✅ short_window

Time: ✅ 2.486µs (SLO: <10.000µs 📉 -75.1%) vs baseline: -0.1%

Memory: ✅ 36.648MB (SLO: <38.000MB -3.6%) vs baseline: +4.9%

✅ samplingrules - 8/8

✅ average_match

Time: ✅ 148.906µs (SLO: <290.000µs 📉 -48.7%) vs baseline: +1.1%

Memory: ✅ 36.176MB (SLO: <38.000MB -4.8%) vs baseline: +4.1%

✅ high_match

Time: ✅ 192.004µs (SLO: <480.000µs 📉 -60.0%) vs baseline: -0.1%

Memory: ✅ 36.313MB (SLO: <38.000MB -4.4%) vs baseline: +5.0%

✅ low_match

Time: ✅ 99.094µs (SLO: <120.000µs 📉 -17.4%) vs baseline: -1.1%

Memory: ✅ 701.711MB (SLO: <780.000MB 📉 -10.0%) vs baseline: +4.9%

✅ very_low_match

Time: ✅ 2.863ms (SLO: <8.500ms 📉 -66.3%) vs baseline: -0.5%

Memory: ✅ 78.880MB (SLO: <85.000MB -7.2%) vs baseline: +4.9%

✅ sethttpmeta - 32/32

✅ all-disabled

Time: ✅ 9.827µs (SLO: <20.000µs 📉 -50.9%) vs baseline: -0.3%

Memory: ✅ 37.120MB (SLO: <38.750MB -4.2%) vs baseline: +4.4%

✅ all-enabled

Time: ✅ 40.239µs (SLO: <50.000µs 📉 -19.5%) vs baseline: +1.7%

Memory: ✅ 37.297MB (SLO: <38.750MB -3.8%) vs baseline: +5.0%

✅ collectipvariant_exists

Time: ✅ 40.207µs (SLO: <50.000µs 📉 -19.6%) vs baseline: -0.3%

Memory: ✅ 37.198MB (SLO: <38.750MB -4.0%) vs baseline: +4.9%

✅ no-collectipvariant

Time: ✅ 39.551µs (SLO: <50.000µs 📉 -20.9%) vs baseline: ~same

Memory: ✅ 37.120MB (SLO: <38.750MB -4.2%) vs baseline: +4.4%

✅ no-useragentvariant

Time: ✅ 38.232µs (SLO: <50.000µs 📉 -23.5%) vs baseline: -0.4%

Memory: ✅ 37.218MB (SLO: <38.750MB -4.0%) vs baseline: +5.1%

✅ obfuscation-no-query

Time: ✅ 39.885µs (SLO: <50.000µs 📉 -20.2%) vs baseline: ~same

Memory: ✅ 37.159MB (SLO: <38.750MB -4.1%) vs baseline: +5.0%

✅ obfuscation-regular-case-explicit-query

Time: ✅ 76.031µs (SLO: <90.000µs 📉 -15.5%) vs baseline: ~same

Memory: ✅ 37.493MB (SLO: <38.750MB -3.2%) vs baseline: +4.9%

✅ obfuscation-regular-case-implicit-query

Time: ✅ 76.385µs (SLO: <90.000µs 📉 -15.1%) vs baseline: -0.2%

Memory: ✅ 37.532MB (SLO: <38.750MB -3.1%) vs baseline: +5.0%

✅ obfuscation-send-querystring-disabled

Time: ✅ 154.291µs (SLO: <170.000µs -9.2%) vs baseline: -0.2%

Memory: ✅ 37.513MB (SLO: <38.750MB -3.2%) vs baseline: +5.0%

✅ obfuscation-worst-case-explicit-query

Time: ✅ 148.781µs (SLO: <160.000µs -7.0%) vs baseline: -0.1%

Memory: ✅ 37.552MB (SLO: <38.750MB -3.1%) vs baseline: +4.9%

✅ obfuscation-worst-case-implicit-query

Time: ✅ 155.249µs (SLO: <170.000µs -8.7%) vs baseline: +0.2%

Memory: ✅ 37.552MB (SLO: <38.750MB -3.1%) vs baseline: +4.8%

✅ useragentvariant_exists_1

Time: ✅ 39.121µs (SLO: <50.000µs 📉 -21.8%) vs baseline: -0.4%

Memory: ✅ 37.159MB (SLO: <38.750MB -4.1%) vs baseline: +5.1%

✅ useragentvariant_exists_2

Time: ✅ 39.849µs (SLO: <50.000µs 📉 -20.3%) vs baseline: -0.6%

Memory: ✅ 37.159MB (SLO: <38.750MB -4.1%) vs baseline: +5.0%

✅ useragentvariant_exists_3

Time: ✅ 39.486µs (SLO: <50.000µs 📉 -21.0%) vs baseline: ~same

Memory: ✅ 37.120MB (SLO: <38.750MB -4.2%) vs baseline: +4.6%

✅ useragentvariant_not_exists_1

Time: ✅ 39.052µs (SLO: <50.000µs 📉 -21.9%) vs baseline: -0.2%

Memory: ✅ 37.198MB (SLO: <38.750MB -4.0%) vs baseline: +4.7%

✅ useragentvariant_not_exists_2

Time: ✅ 39.048µs (SLO: <50.000µs 📉 -21.9%) vs baseline: ~same

Memory: ✅ 37.198MB (SLO: <38.750MB -4.0%) vs baseline: +5.1%

✅ span - 26/26

✅ add-event

Time: ✅ 17.548ms (SLO: <22.500ms 📉 -22.0%) vs baseline: +0.3%

Memory: ✅ 38.589MB (SLO: <53.000MB 📉 -27.2%) vs baseline: +4.8%

✅ add-metrics

Time: ✅ 86.370ms (SLO: <93.500ms -7.6%) vs baseline: ~same

Memory: ✅ 43.011MB (SLO: <53.000MB 📉 -18.8%) vs baseline: +4.9%

✅ add-tags

Time: ✅ 145.349ms (SLO: <155.000ms -6.2%) vs baseline: +0.1%

Memory: ✅ 43.016MB (SLO: <53.000MB 📉 -18.8%) vs baseline: +4.5%

✅ get-context

Time: ✅ 16.864ms (SLO: <20.500ms 📉 -17.7%) vs baseline: ~same

Memory: ✅ 38.344MB (SLO: <53.000MB 📉 -27.7%) vs baseline: +4.7%

✅ is-recording

Time: ✅ 16.879ms (SLO: <20.500ms 📉 -17.7%) vs baseline: -0.1%

Memory: ✅ 38.354MB (SLO: <53.000MB 📉 -27.6%) vs baseline: +4.4%

✅ record-exception

Time: ✅ 35.524ms (SLO: <41.000ms 📉 -13.4%) vs baseline: ~same

Memory: ✅ 39.128MB (SLO: <53.000MB 📉 -26.2%) vs baseline: +5.3%

✅ set-status

Time: ✅ 18.581ms (SLO: <22.000ms 📉 -15.5%) vs baseline: -0.2%

Memory: ✅ 38.548MB (SLO: <53.000MB 📉 -27.3%) vs baseline: +4.7%

✅ start

Time: ✅ 17.361ms (SLO: <20.500ms 📉 -15.3%) vs baseline: +3.8%

Memory: ✅ 38.384MB (SLO: <53.000MB 📉 -27.6%) vs baseline: +4.3%

✅ start-finish

Time: ✅ 53.703ms (SLO: <56.000ms -4.1%) vs baseline: +0.3%

Memory: ✅ 36.549MB (SLO: <38.000MB -3.8%) vs baseline: +4.7%

✅ start-finish-telemetry

Time: ✅ 54.912ms (SLO: <58.000ms -5.3%) vs baseline: +0.4%

Memory: ✅ 36.687MB (SLO: <38.000MB -3.5%) vs baseline: +5.9%

✅ start-finish-traceid128

Time: ✅ 55.912ms (SLO: <60.000ms -6.8%) vs baseline: ~same

Memory: ✅ 36.569MB (SLO: <38.000MB -3.8%) vs baseline: +5.7%

✅ start-traceid128

Time: ✅ 16.746ms (SLO: <22.500ms 📉 -25.6%) vs baseline: +0.7%

Memory: ✅ 38.440MB (SLO: <53.000MB 📉 -27.5%) vs baseline: +4.5%

✅ update-name

Time: ✅ 17.198ms (SLO: <22.000ms 📉 -21.8%) vs baseline: ~same

Memory: ✅ 38.481MB (SLO: <53.000MB 📉 -27.4%) vs baseline: +5.0%

✅ telemetryaddmetric - 30/30

✅ 1-count-metric-1-times

Time: ✅ 2.315µs (SLO: <20.000µs 📉 -88.4%) vs baseline: +7.7%

Memory: ✅ 36.353MB (SLO: <38.000MB -4.3%) vs baseline: +5.0%

✅ 1-count-metrics-100-times

Time: ✅ 153.318µs (SLO: <220.000µs 📉 -30.3%) vs baseline: -0.6%

Memory: ✅ 36.589MB (SLO: <38.000MB -3.7%) vs baseline: +5.9%

✅ 1-distribution-metric-1-times

Time: ✅ 2.447µs (SLO: <20.000µs 📉 -87.8%) vs baseline: -0.3%

Memory: ✅ 36.608MB (SLO: <38.000MB -3.7%) vs baseline: +5.7%

✅ 1-distribution-metrics-100-times

Time: ✅ 164.950µs (SLO: <230.000µs 📉 -28.3%) vs baseline: -0.1%

Memory: ✅ 36.510MB (SLO: <38.000MB -3.9%) vs baseline: +4.6%

✅ 1-gauge-metric-1-times

Time: ✅ 1.986µs (SLO: <20.000µs 📉 -90.1%) vs baseline: -0.9%

Memory: ✅ 36.392MB (SLO: <38.000MB -4.2%) vs baseline: +5.4%

✅ 1-gauge-metrics-100-times

Time: ✅ 137.637µs (SLO: <150.000µs -8.2%) vs baseline: -0.2%

Memory: ✅ 36.549MB (SLO: <38.000MB -3.8%) vs baseline: +5.4%

✅ 1-rate-metric-1-times

Time: ✅ 2.294µs (SLO: <20.000µs 📉 -88.5%) vs baseline: -0.1%

Memory: ✅ 36.648MB (SLO: <38.000MB -3.6%) vs baseline: +5.5%

✅ 1-rate-metrics-100-times

Time: ✅ 168.334µs (SLO: <250.000µs 📉 -32.7%) vs baseline: +0.3%

Memory: ✅ 36.431MB (SLO: <38.000MB -4.1%) vs baseline: +5.3%

✅ 100-count-metrics-100-times

Time: ✅ 15.211ms (SLO: <22.000ms 📉 -30.9%) vs baseline: -0.5%

Memory: ✅ 36.569MB (SLO: <38.000MB -3.8%) vs baseline: +4.9%

✅ 100-distribution-metrics-100-times

Time: ✅ 1.719ms (SLO: <2.550ms 📉 -32.6%) vs baseline: +0.6%

Memory: ✅ 36.648MB (SLO: <38.000MB -3.6%) vs baseline: +4.8%

✅ 100-gauge-metrics-100-times

Time: ✅ 1.392ms (SLO: <1.550ms 📉 -10.2%) vs baseline: ~same

Memory: ✅ 36.589MB (SLO: <38.000MB -3.7%) vs baseline: +4.7%

✅ 100-rate-metrics-100-times

Time: ✅ 1.716ms (SLO: <2.550ms 📉 -32.7%) vs baseline: -0.4%

Memory: ✅ 36.549MB (SLO: <38.000MB -3.8%) vs baseline: +4.9%

✅ flush-1-metric

Time: ✅ 3.624µs (SLO: <20.000µs 📉 -81.9%) vs baseline: -0.5%

Memory: ✅ 36.707MB (SLO: <38.000MB -3.4%) vs baseline: +5.0%

✅ flush-100-metrics

Time: ✅ 176.030µs (SLO: <250.000µs 📉 -29.6%) vs baseline: +0.3%

Memory: ✅ 36.707MB (SLO: <38.000MB -3.4%) vs baseline: +5.0%

✅ flush-1000-metrics

Time: ✅ 2.186ms (SLO: <2.500ms 📉 -12.6%) vs baseline: -0.3%

Memory: ✅ 37.513MB (SLO: <38.750MB -3.2%) vs baseline: +5.1%

✅ tracer - 6/6

✅ large

Time: ✅ 31.297ms (SLO: <32.950ms -5.0%) vs baseline: +0.2%

Memory: ✅ 37.434MB (SLO: <39.250MB -4.6%) vs baseline: +4.8%

✅ medium

Time: ✅ 3.094ms (SLO: <3.200ms -3.3%) vs baseline: +0.3%

Memory: ✅ 36.215MB (SLO: <38.750MB -6.5%) vs baseline: +4.7%

✅ small

Time: ✅ 357.986µs (SLO: <370.000µs -3.2%) vs baseline: +3.2%

Memory: ✅ 36.196MB (SLO: <38.750MB -6.6%) vs baseline: +4.7%

ℹ️ Scenarios Missing SLO Configuration (46 scenarios)

The following scenarios exist in candidate data but have no SLO thresholds configured:

coreapiscenario-core_dispatch_listeners
coreapiscenario-core_dispatch_no_listeners
coreapiscenario-core_dispatch_with_results_listeners
coreapiscenario-core_dispatch_with_results_no_listeners
djangosimple-baseline
errortrackingdjangosimple-baseline
errortrackingflasksqli-baseline
flasksimple-baseline
flasksqli-baseline
iast_aspects-re_expand_aspect
iast_aspects-re_expand_noaspect
iast_aspects-re_findall_aspect
iast_aspects-re_findall_noaspect
iast_aspects-re_finditer_aspect
iast_aspects-re_finditer_noaspect
iast_aspects-re_fullmatch_aspect
iast_aspects-re_fullmatch_noaspect
iast_aspects-re_group_aspect
iast_aspects-re_group_noaspect
iast_aspects-re_groups_aspect
iast_aspects-re_groups_noaspect
iast_aspects-re_match_aspect
iast_aspects-re_match_noaspect
iast_aspects-re_search_aspect
iast_aspects-re_search_noaspect
iast_aspects-re_sub_aspect
iast_aspects-re_sub_noaspect
iast_aspects-re_subn_aspect
iast_aspects-re_subn_noaspect
sethttpmeta-obfuscation-disabled
startup-baseline
startup-baseline_django
startup-baseline_flask
startup-ddtrace_run
startup-ddtrace_run_appsec
startup-ddtrace_run_profiling
startup-ddtrace_run_runtime_metrics
startup-ddtrace_run_send_span
startup-ddtrace_run_telemetry_disabled
startup-ddtrace_run_telemetry_enabled
startup-import_ddtrace
startup-import_ddtrace_auto
startup-import_ddtrace_auto_django
startup-import_ddtrace_auto_flask
startup-import_ddtrace_django
startup-import_ddtrace_flask

sameerank · 2026-03-20T17:47:52Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b292329966

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

ddtrace/internal/openfeature/_provider.py

ddtrace/internal/openfeature/_flageval_metrics.py

riotfile.py

ddtrace/internal/openfeature/_flageval_metrics.py

leoromanovsky

Let me know what you decide in terms of unknown for missing reason; I can make the change in go (or feel free to). Same for the lowercasing.

ddtrace/internal/openfeature/_flageval_metrics.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6dd59ec52a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

ddtrace/internal/openfeature/_provider.py

ddtrace/internal/openfeature/_flageval_metrics.py

emmettbutler

Core and test sources look good to me

tests/openfeature/test_provider.py

tests/openfeature/test_flag_eval_metrics.py

Update minimum SDK version to 0.8.0 which adds the `details` parameter to the `finally_after` hook stage. This is required for the flag evaluation metrics feature to access variant, reason, and error_code.

…ENABLED=true Check the _otel_metrics_enabled config before creating the OTel counter to avoid overhead when OTel metrics are not enabled. This follows the established pattern in ddtrace for gating OTel metrics features.

…protection

… fix config deletion handling

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch from c241395 to 93848cb Compare March 19, 2026 21:03

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch from 93848cb to b08bbaa Compare March 19, 2026 21:08

This comment has been minimized.

Sign in to view

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch 5 times, most recently from 5585746 to f4e7b65 Compare March 20, 2026 01:12

sameerank commented Mar 20, 2026

View reviewed changes

sameerank marked this pull request as ready for review March 20, 2026 16:20

sameerank requested review from a team as code owners March 20, 2026 16:20

sameerank requested review from P403n1x87, dd-oleksii, leoromanovsky and rachelyangdog March 20, 2026 16:20

chatgpt-codex-connector bot reviewed Mar 20, 2026

View reviewed changes

ddtrace/internal/openfeature/_flageval_metrics.py Show resolved Hide resolved

sameerank marked this pull request as draft March 20, 2026 16:44

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch 3 times, most recently from b1b46c7 to b292329 Compare March 20, 2026 17:05

chatgpt-codex-connector bot reviewed Mar 20, 2026

View reviewed changes

ddtrace/internal/openfeature/_provider.py Show resolved Hide resolved

ddtrace/internal/openfeature/_flageval_metrics.py Show resolved Hide resolved

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch 2 times, most recently from ed503c2 to 70eb5ba Compare March 20, 2026 20:24

leoromanovsky reviewed Mar 21, 2026

View reviewed changes

riotfile.py Show resolved Hide resolved

ddtrace/internal/openfeature/_flageval_metrics.py Outdated Show resolved Hide resolved

ddtrace/internal/openfeature/_flageval_metrics.py Show resolved Hide resolved

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch 2 times, most recently from 810d4c8 to 74f4110 Compare March 23, 2026 07:55

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch from 74f4110 to 6dd59ec Compare March 23, 2026 07:57

leoromanovsky approved these changes Mar 23, 2026

View reviewed changes

ddtrace/internal/openfeature/_flageval_metrics.py Show resolved Hide resolved

ddtrace/internal/openfeature/_flageval_metrics.py Show resolved Hide resolved

sameerank marked this pull request as ready for review March 23, 2026 16:16

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

ddtrace/internal/openfeature/_provider.py Show resolved Hide resolved

ddtrace/internal/openfeature/_flageval_metrics.py Show resolved Hide resolved

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch from c435e25 to 0ce14d8 Compare March 24, 2026 00:46

emmettbutler approved these changes Mar 25, 2026

View reviewed changes

tests/openfeature/test_provider.py Outdated Show resolved Hide resolved

tests/openfeature/test_flag_eval_metrics.py Outdated Show resolved Hide resolved

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch from 7410042 to c540472 Compare March 25, 2026 17:06

sameerank added 11 commits March 26, 2026 08:42

feat(openfeature): add flag evaluation metrics

8425d43

feat(openfeature): return ERROR reason for flag not found

390fadb

feat(openfeature): clarify flag not found behavior and add release note

b1c6a91

feat(openfeature): remove dead code and fix linting

a5a865b

test(openfeature): add flag eval metrics tests

5543ba1

chore(openfeature): require openfeature-sdk >= 0.8.0

cf8289c

Update minimum SDK version to 0.8.0 which adds the `details` parameter to the `finally_after` hook stage. This is required for the flag evaluation metrics feature to access variant, reason, and error_code.

address review feedback: use ErrorCode.value directly, add TypeError …

ec3be44

…protection

fix(openfeature): correct config import and use lowercase error codes

3965a42

fix(openfeature): return PROVIDER_NOT_READY when no config loaded and…

259bc97

… fix config deletion handling

test(openfeature): add debug messages to assertions for easier debugging

0951d14

sameerank force-pushed the sameerank/FFL-1942/add-flag-eval-metrics branch from c540472 to 0951d14 Compare March 26, 2026 15:42

gh-worker-dd-mergequeue-cf854d bot merged commit 62e5bae into main Mar 26, 2026
1113 checks passed

gh-worker-dd-mergequeue-cf854d bot deleted the sameerank/FFL-1942/add-flag-eval-metrics branch March 26, 2026 17:36

Conversation

sameerank commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Unit test parity with Go SDK

Python-specific tests (not in Go)

System tests

Risks

Additional Notes

Uh oh!

cit-pr-commenter-54b7da bot commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

pr-commenter bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance SLOs

✅ add_aspect

✅ add_inplace_aspect

✅ add_inplace_noaspect

✅ add_noaspect

✅ bytearray_aspect

✅ bytearray_extend_aspect

✅ bytearray_extend_noaspect

✅ bytearray_noaspect

✅ bytes_aspect

✅ bytes_noaspect

✅ bytesio_aspect

✅ bytesio_noaspect

✅ capitalize_aspect

✅ capitalize_noaspect

✅ casefold_aspect

✅ casefold_noaspect

✅ decode_aspect

✅ decode_noaspect

✅ encode_aspect

✅ encode_noaspect

✅ format_aspect

✅ format_map_aspect

✅ format_map_noaspect

✅ format_noaspect

✅ index_aspect

✅ index_noaspect

✅ join_aspect

✅ join_noaspect

✅ ljust_aspect

✅ ljust_noaspect

✅ lower_aspect

✅ lower_noaspect

✅ lstrip_aspect

✅ lstrip_noaspect

✅ modulo_aspect

✅ modulo_aspect_for_bytearray_bytearray

✅ modulo_aspect_for_bytes

✅ modulo_aspect_for_bytes_bytearray

✅ modulo_noaspect

✅ replace_aspect

✅ replace_noaspect

✅ repr_aspect

✅ repr_noaspect

✅ rstrip_aspect

✅ rstrip_noaspect

✅ slice_aspect

✅ slice_noaspect

✅ stringio_aspect

✅ stringio_noaspect

✅ strip_aspect

✅ strip_noaspect

sameerank commented Mar 19, 2026 •

edited

Loading

cit-pr-commenter-54b7da bot commented Mar 19, 2026 •

edited

Loading

pr-commenter bot commented Mar 20, 2026 •

edited

Loading