fix(dsm): Grab correct span in RabbitMQ/OTel for DSM checkpoints#8362
fix(dsm): Grab correct span in RabbitMQ/OTel for DSM checkpoints#8362robcarlan-datadog wants to merge 2 commits intomasterfrom
Conversation
BenchmarksBenchmark execution time: 2026-03-25 14:55:33 Comparing candidate commit 3c5e1d7 in PR branch Found 6 performance improvements and 14 performance regressions! Performance is the same for 253 metrics, 15 unstable metrics.
|
InternalActiveScope and ActiveScope both return the same scope stack — the OTel scope sits on top either way. The scope chain walk via scope.Parent is what actually fixes the issue. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8362) and master.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Metric | Master (Mean ± 95% CI) | Current (Mean ± 95% CI) | Change | Status |
|---|---|---|---|---|
| .NET Framework 4.8 - Baseline | ||||
| duration | 192.79 ± (192.85 - 193.66) ms | 220.87 ± (219.93 - 221.96) ms | +14.6% | ❌⬆️ |
| .NET Framework 4.8 - Bailout | ||||
| duration | 196.20 ± (195.92 - 196.62) ms | 219.62 ± (219.84 - 221.07) ms | +11.9% | ❌⬆️ |
| .NET Framework 4.8 - CallTarget+Inlining+NGEN | ||||
| duration | 1158.67 ± (1160.42 - 1168.44) ms | 1256.33 ± (1254.11 - 1261.26) ms | +8.4% | ❌⬆️ |
Full Metrics Comparison
FakeDbCommand
| Metric | Master (Mean ± 95% CI) | Current (Mean ± 95% CI) | Change | Status |
|---|---|---|---|---|
| .NET Framework 4.8 - Baseline | ||||
| duration | 68.64 ± (68.83 - 69.29) ms | 71.79 ± (71.89 - 72.24) ms | +4.6% | ✅⬆️ |
| .NET Framework 4.8 - Bailout | ||||
| duration | 79.83 ± (78.88 - 80.41) ms | 76.11 ± (76.01 - 76.33) ms | -4.7% | ✅ |
| .NET Framework 4.8 - CallTarget+Inlining+NGEN | ||||
| duration | 1047.72 ± (1050.50 - 1058.26) ms | 1071.83 ± (1074.25 - 1083.51) ms | +2.3% | ✅⬆️ |
| .NET Core 3.1 - Baseline | ||||
| process.internal_duration_ms | 21.78 ± (21.75 - 21.81) ms | 22.24 ± (22.21 - 22.28) ms | +2.1% | ✅⬆️ |
| process.time_to_main_ms | 79.31 ± (79.15 - 79.47) ms | 83.22 ± (83.02 - 83.41) ms | +4.9% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 10.90 ± (10.90 - 10.91) MB | 10.91 ± (10.90 - 10.91) MB | +0.0% | ✅⬆️ |
| runtime.dotnet.threads.count | 12 ± (12 - 12) | 12 ± (12 - 12) | +0.0% | ✅ |
| .NET Core 3.1 - Bailout | ||||
| process.internal_duration_ms | 21.73 ± (21.71 - 21.75) ms | 22.27 ± (22.24 - 22.31) ms | +2.5% | ✅⬆️ |
| process.time_to_main_ms | 80.66 ± (80.54 - 80.78) ms | 84.61 ± (84.40 - 84.82) ms | +4.9% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 10.94 ± (10.93 - 10.94) MB | 10.94 ± (10.94 - 10.94) MB | +0.0% | ✅⬆️ |
| runtime.dotnet.threads.count | 13 ± (13 - 13) | 13 ± (13 - 13) | +0.0% | ✅ |
| .NET Core 3.1 - CallTarget+Inlining+NGEN | ||||
| process.internal_duration_ms | 216.87 ± (215.60 - 218.14) ms | 224.60 ± (223.47 - 225.74) ms | +3.6% | ✅⬆️ |
| process.time_to_main_ms | 520.24 ± (519.26 - 521.21) ms | 533.50 ± (532.24 - 534.76) ms | +2.5% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 48.24 ± (48.21 - 48.26) MB | 48.25 ± (48.22 - 48.28) MB | +0.0% | ✅⬆️ |
| runtime.dotnet.threads.count | 28 ± (28 - 28) | 28 ± (28 - 28) | -0.2% | ✅ |
| .NET 6 - Baseline | ||||
| process.internal_duration_ms | 20.58 ± (20.55 - 20.60) ms | 21.02 ± (20.98 - 21.05) ms | +2.1% | ✅⬆️ |
| process.time_to_main_ms | 68.89 ± (68.75 - 69.03) ms | 72.78 ± (72.60 - 72.97) ms | +5.7% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 10.59 ± (10.59 - 10.59) MB | 10.63 ± (10.63 - 10.63) MB | +0.3% | ✅⬆️ |
| runtime.dotnet.threads.count | 10 ± (10 - 10) | 10 ± (10 - 10) | +0.0% | ✅ |
| .NET 6 - Bailout | ||||
| process.internal_duration_ms | 20.59 ± (20.56 - 20.62) ms | 20.97 ± (20.94 - 21.00) ms | +1.8% | ✅⬆️ |
| process.time_to_main_ms | 69.81 ± (69.67 - 69.94) ms | 73.69 ± (73.51 - 73.87) ms | +5.6% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 10.66 ± (10.65 - 10.66) MB | 10.73 ± (10.73 - 10.73) MB | +0.7% | ✅⬆️ |
| runtime.dotnet.threads.count | 11 ± (11 - 11) | 11 ± (11 - 11) | +0.0% | ✅ |
| .NET 6 - CallTarget+Inlining+NGEN | ||||
| process.internal_duration_ms | 208.33 ± (207.01 - 209.64) ms | 210.74 ± (209.68 - 211.80) ms | +1.2% | ✅⬆️ |
| process.time_to_main_ms | 518.76 ± (517.69 - 519.83) ms | 533.74 ± (532.41 - 535.06) ms | +2.9% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 50.00 ± (49.95 - 50.05) MB | 50.13 ± (50.10 - 50.16) MB | +0.3% | ✅⬆️ |
| runtime.dotnet.threads.count | 29 ± (29 - 29) | 29 ± (29 - 29) | +0.1% | ✅⬆️ |
| .NET 8 - Baseline | ||||
| process.internal_duration_ms | 18.74 ± (18.71 - 18.77) ms | 19.29 ± (19.25 - 19.33) ms | +2.9% | ✅⬆️ |
| process.time_to_main_ms | 67.94 ± (67.80 - 68.09) ms | 71.82 ± (71.63 - 72.00) ms | +5.7% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 7.67 ± (7.66 - 7.67) MB | 7.68 ± (7.67 - 7.68) MB | +0.1% | ✅⬆️ |
| runtime.dotnet.threads.count | 10 ± (10 - 10) | 10 ± (10 - 10) | +0.0% | ✅ |
| .NET 8 - Bailout | ||||
| process.internal_duration_ms | 18.71 ± (18.68 - 18.73) ms | 19.36 ± (19.31 - 19.40) ms | +3.5% | ✅⬆️ |
| process.time_to_main_ms | 69.13 ± (69.02 - 69.24) ms | 73.11 ± (72.91 - 73.31) ms | +5.8% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 7.76 ± (7.75 - 7.77) MB | 7.72 ± (7.72 - 7.73) MB | -0.5% | ✅ |
| runtime.dotnet.threads.count | 11 ± (11 - 11) | 11 ± (11 - 11) | +0.0% | ✅ |
| .NET 8 - CallTarget+Inlining+NGEN | ||||
| process.internal_duration_ms | 153.29 ± (152.58 - 154.01) ms | 158.93 ± (158.27 - 159.59) ms | +3.7% | ✅⬆️ |
| process.time_to_main_ms | 477.64 ± (476.73 - 478.56) ms | 492.47 ± (491.50 - 493.44) ms | +3.1% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 0 ± (0 - 0) | 0 ± (0 - 0) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 36.90 ± (36.87 - 36.92) MB | 36.93 ± (36.91 - 36.95) MB | +0.1% | ✅⬆️ |
| runtime.dotnet.threads.count | 28 ± (28 - 28) | 28 ± (28 - 28) | -0.1% | ✅ |
HttpMessageHandler
| Metric | Master (Mean ± 95% CI) | Current (Mean ± 95% CI) | Change | Status |
|---|---|---|---|---|
| .NET Framework 4.8 - Baseline | ||||
| duration | 192.79 ± (192.85 - 193.66) ms | 220.87 ± (219.93 - 221.96) ms | +14.6% | ❌⬆️ |
| .NET Framework 4.8 - Bailout | ||||
| duration | 196.20 ± (195.92 - 196.62) ms | 219.62 ± (219.84 - 221.07) ms | +11.9% | ❌⬆️ |
| .NET Framework 4.8 - CallTarget+Inlining+NGEN | ||||
| duration | 1158.67 ± (1160.42 - 1168.44) ms | 1256.33 ± (1254.11 - 1261.26) ms | +8.4% | ❌⬆️ |
| .NET Core 3.1 - Baseline | ||||
| process.internal_duration_ms | 187.03 ± (186.69 - 187.37) ms | 215.24 ± (214.41 - 216.08) ms | +15.1% | ✅⬆️ |
| process.time_to_main_ms | 80.51 ± (80.27 - 80.75) ms | 92.41 ± (92.05 - 92.77) ms | +14.8% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 3 ± (3 - 3) | 3 ± (3 - 3) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 16.11 ± (16.07 - 16.14) MB | 15.85 ± (15.84 - 15.87) MB | -1.6% | ✅ |
| runtime.dotnet.threads.count | 20 ± (20 - 20) | 20 ± (20 - 20) | +1.5% | ✅⬆️ |
| .NET Core 3.1 - Bailout | ||||
| process.internal_duration_ms | 186.97 ± (186.60 - 187.34) ms | 215.41 ± (214.77 - 216.05) ms | +15.2% | ✅⬆️ |
| process.time_to_main_ms | 82.01 ± (81.85 - 82.17) ms | 93.79 ± (93.53 - 94.05) ms | +14.4% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 3 ± (3 - 3) | 3 ± (3 - 3) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 16.16 ± (16.13 - 16.18) MB | 15.95 ± (15.93 - 15.97) MB | -1.3% | ✅ |
| runtime.dotnet.threads.count | 21 ± (20 - 21) | 21 ± (21 - 21) | +1.9% | ✅⬆️ |
| .NET Core 3.1 - CallTarget+Inlining+NGEN | ||||
| process.internal_duration_ms | 391.58 ± (390.14 - 393.02) ms | 428.88 ± (427.21 - 430.55) ms | +9.5% | ✅⬆️ |
| process.time_to_main_ms | 521.25 ± (520.06 - 522.45) ms | 573.21 ± (571.47 - 574.95) ms | +10.0% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 3 ± (3 - 3) | 3 ± (3 - 3) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 58.43 ± (58.22 - 58.64) MB | 59.18 ± (59.12 - 59.23) MB | +1.3% | ✅⬆️ |
| runtime.dotnet.threads.count | 30 ± (30 - 30) | 30 ± (30 - 30) | +0.2% | ✅⬆️ |
| .NET 6 - Baseline | ||||
| process.internal_duration_ms | 191.92 ± (191.61 - 192.24) ms | 220.73 ± (220.01 - 221.45) ms | +15.0% | ✅⬆️ |
| process.time_to_main_ms | 70.05 ± (69.87 - 70.23) ms | 80.02 ± (79.80 - 80.24) ms | +14.2% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 4 ± (4 - 4) | 4 ± (4 - 4) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 16.06 ± (15.91 - 16.22) MB | 16.18 ± (16.16 - 16.19) MB | +0.7% | ✅⬆️ |
| runtime.dotnet.threads.count | 18 ± (18 - 18) | 20 ± (20 - 20) | +7.7% | ✅⬆️ |
| .NET 6 - Bailout | ||||
| process.internal_duration_ms | 190.73 ± (190.44 - 191.01) ms | 221.58 ± (220.70 - 222.46) ms | +16.2% | ✅⬆️ |
| process.time_to_main_ms | 70.86 ± (70.73 - 70.98) ms | 81.54 ± (81.31 - 81.77) ms | +15.1% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 4 ± (4 - 4) | 4 ± (4 - 4) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 16.29 ± (16.19 - 16.40) MB | 16.17 ± (16.15 - 16.18) MB | -0.8% | ✅ |
| runtime.dotnet.threads.count | 19 ± (19 - 20) | 21 ± (20 - 21) | +6.2% | ✅⬆️ |
| .NET 6 - CallTarget+Inlining+NGEN | ||||
| process.internal_duration_ms | 413.43 ± (412.12 - 414.73) ms | 456.05 ± (451.84 - 460.26) ms | +10.3% | ✅⬆️ |
| process.time_to_main_ms | 523.97 ± (522.53 - 525.41) ms | 576.75 ± (575.50 - 578.00) ms | +10.1% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 4 ± (4 - 4) | 4 ± (4 - 4) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 60.94 ± (60.90 - 60.99) MB | 60.85 ± (60.79 - 60.91) MB | -0.2% | ✅ |
| runtime.dotnet.threads.count | 31 ± (30 - 31) | 31 ± (31 - 31) | +1.5% | ✅⬆️ |
| .NET 8 - Baseline | ||||
| process.internal_duration_ms | 190.20 ± (189.80 - 190.61) ms | 232.89 ± (231.58 - 234.19) ms | +22.4% | ✅⬆️ |
| process.time_to_main_ms | 69.52 ± (69.33 - 69.72) ms | 80.28 ± (80.00 - 80.56) ms | +15.5% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 4 ± (4 - 4) | 4 ± (4 - 4) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 11.75 ± (11.73 - 11.78) MB | 11.44 ± (11.42 - 11.46) MB | -2.6% | ✅ |
| runtime.dotnet.threads.count | 18 ± (18 - 18) | 19 ± (19 - 19) | +4.3% | ✅⬆️ |
| .NET 8 - Bailout | ||||
| process.internal_duration_ms | 188.96 ± (188.44 - 189.48) ms | 223.87 ± (222.67 - 225.06) ms | +18.5% | ✅⬆️ |
| process.time_to_main_ms | 70.72 ± (70.52 - 70.92) ms | 80.73 ± (80.49 - 80.98) ms | +14.2% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 4 ± (4 - 4) | 4 ± (4 - 4) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 11.76 ± (11.69 - 11.83) MB | 11.48 ± (11.47 - 11.50) MB | -2.3% | ✅ |
| runtime.dotnet.threads.count | 19 ± (19 - 19) | 20 ± (20 - 20) | +4.1% | ✅⬆️ |
| .NET 8 - CallTarget+Inlining+NGEN | ||||
| process.internal_duration_ms | 340.13 ± (339.04 - 341.22) ms | 474.64 ± (472.84 - 476.44) ms | +39.5% | ✅⬆️ |
| process.time_to_main_ms | 478.95 ± (478.09 - 479.81) ms | 530.67 ± (529.32 - 532.03) ms | +10.8% | ✅⬆️ |
| runtime.dotnet.exceptions.count | 4 ± (4 - 4) | 4 ± (4 - 4) | +0.0% | ✅ |
| runtime.dotnet.mem.committed | 48.87 ± (48.83 - 48.91) MB | 50.66 ± (50.62 - 50.71) MB | +3.7% | ✅⬆️ |
| runtime.dotnet.threads.count | 30 ± (30 - 30) | 30 ± (30 - 30) | +0.5% | ✅⬆️ |
Comparison explanation
Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
- Welch test with statistical test for significance of 5%
- Only results indicating a difference greater than 5% and 5 ms are considered.
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.
Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).
Duration charts
FakeDbCommand (.NET Framework 4.8)
gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (72ms) : 69, 75
master - mean (69ms) : 66, 72
section Bailout
This PR (8362) - mean (76ms) : 74, 78
master - mean (80ms) : 69, 91
section CallTarget+Inlining+NGEN
This PR (8362) - mean (1,079ms) : 1011, 1147
master - mean (1,054ms) : 999, 1109
FakeDbCommand (.NET Core 3.1)
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (112ms) : 108, 116
master - mean (107ms) : 104, 110
section Bailout
This PR (8362) - mean (113ms) : 110, 116
master - mean (108ms) : 106, 110
section CallTarget+Inlining+NGEN
This PR (8362) - mean (796ms) : 769, 824
master - mean (777ms) : 751, 802
FakeDbCommand (.NET 6)
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (100ms) : 97, 103
master - mean (95ms) : 92, 97
section Bailout
This PR (8362) - mean (101ms) : crit, 97, 104
master - mean (95ms) : 93, 98
section CallTarget+Inlining+NGEN
This PR (8362) - mean (783ms) : 763, 802
master - mean (760ms) : 736, 784
FakeDbCommand (.NET 8)
gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (99ms) : 95, 103
master - mean (93ms) : 90, 96
section Bailout
This PR (8362) - mean (100ms) : crit, 95, 105
master - mean (94ms) : 93, 96
section CallTarget+Inlining+NGEN
This PR (8362) - mean (690ms) : 664, 716
master - mean (661ms) : 645, 677
HttpMessageHandler (.NET Framework 4.8)
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (221ms) : 206, 236
master - mean (193ms) : 189, 197
section Bailout
This PR (8362) - mean (220ms) : crit, 211, 230
master - mean (196ms) : 193, 200
section CallTarget+Inlining+NGEN
This PR (8362) - mean (1,258ms) : crit, 1207, 1309
master - mean (1,164ms) : 1107, 1222
HttpMessageHandler (.NET Core 3.1)
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (318ms) : 303, 333
master - mean (276ms) : 270, 282
section Bailout
This PR (8362) - mean (319ms) : crit, 307, 331
master - mean (277ms) : 272, 282
section CallTarget+Inlining+NGEN
This PR (8362) - mean (1,045ms) : crit, 1019, 1072
master - mean (942ms) : 923, 961
HttpMessageHandler (.NET 6)
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (310ms) : 298, 322
master - mean (270ms) : 266, 274
section Bailout
This PR (8362) - mean (312ms) : crit, 297, 327
master - mean (269ms) : 265, 274
section CallTarget+Inlining+NGEN
This PR (8362) - mean (1,069ms) : crit, 1005, 1132
master - mean (965ms) : 943, 988
HttpMessageHandler (.NET 8)
gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8362) - mean (324ms) : 304, 344
master - mean (269ms) : 263, 275
section Bailout
This PR (8362) - mean (316ms) : crit, 292, 340
master - mean (269ms) : 262, 276
section CallTarget+Inlining+NGEN
This PR (8362) - mean (1,041ms) : crit, 991, 1092
master - mean (850ms) : 829, 871
Summary of changes
Fix DSM header injection for RabbitMQ / OTel. OTel emits a span which causes us to skip DSM checkpoint logic.
Reason for change
Customer issue
Implementation details
OnMethodEnd used
tracer.ActiveScope?.Spanto find the RabbitMQ producer span. WhenDD_TRACE_OTEL_ENABLED=true, RabbitMQ.Client v7 emits its own OTel Activity which gets converted to a
scope with OpenTelemetryTags that sits on top of the dd-trace RabbitMQ scope (RabbitMQTags). The
pattern match
activeSpan is not Span { Tags: RabbitMQTags }fails on the OTel span, so the methodbails out, skipping both DSM checkpoint and header injection.
OTel generated span from here
RabbitMQ span created from here
Test coverage
Other details