Summary
e2e_modules::profiler_dump_analysis_full_stack::test_profiler_object_graph_roots_inspect_and_diff_full_stack fails intermittently on the assertion at tests/e2e_modules/profiler_dump_analysis_full_stack.rs:117:
baseline heap dump must contain StringBuilder instances (ProfileTarget allocates them constantly)
Observed 1 failure in 3 consecutive full runs of Rust shard 1 on the same commit and machine (Windows, make _test-rust-shard SHARD=1 SHARD_COUNT=2). The failing run reached the assertion in 4.86s; the passing runs took the same path. Nothing in the tree changed between runs.
Why it flakes
The assertion depends on ProfileTarget having actually allocated StringBuilders before the baseline dump is captured. The comment ("allocates them constantly") is true in steady state, but there is a race between the target reaching that steady state and dotnet-dump capturing the heap. On a loaded machine the dump can land in the startup window, where the assertion is legitimately false.
A distinct, earlier failure mode of the same test is a missing dotnet-dump global tool — that produces a much faster failure (~0.4s) and is an environment problem, not this race. CI installs dotnet-trace, dotnet-counters, and dotnet-dump explicitly in ci-rust.yml, so only the race applies there.
Suggested fix
Do not assert on an unsynchronised allocation. Either:
- have
ProfileTarget signal readiness (stdout marker / file touch) once it has allocated the objects the assertion needs, and capture the baseline only after that, or
- poll: capture and inspect until
StringBuilder appears or a timeout elapses, so the assertion reflects steady state rather than whenever the dump happened to land.
Per the repo's test rules the assertion must not be weakened or removed — it should be made deterministic.
Summary
e2e_modules::profiler_dump_analysis_full_stack::test_profiler_object_graph_roots_inspect_and_diff_full_stackfails intermittently on the assertion attests/e2e_modules/profiler_dump_analysis_full_stack.rs:117:Observed 1 failure in 3 consecutive full runs of Rust shard 1 on the same commit and machine (Windows,
make _test-rust-shard SHARD=1 SHARD_COUNT=2). The failing run reached the assertion in 4.86s; the passing runs took the same path. Nothing in the tree changed between runs.Why it flakes
The assertion depends on
ProfileTargethaving actually allocatedStringBuilders before the baseline dump is captured. The comment ("allocates them constantly") is true in steady state, but there is a race between the target reaching that steady state anddotnet-dumpcapturing the heap. On a loaded machine the dump can land in the startup window, where the assertion is legitimately false.A distinct, earlier failure mode of the same test is a missing
dotnet-dumpglobal tool — that produces a much faster failure (~0.4s) and is an environment problem, not this race. CI installsdotnet-trace,dotnet-counters, anddotnet-dumpexplicitly inci-rust.yml, so only the race applies there.Suggested fix
Do not assert on an unsynchronised allocation. Either:
ProfileTargetsignal readiness (stdout marker / file touch) once it has allocated the objects the assertion needs, and capture the baseline only after that, orStringBuilderappears or a timeout elapses, so the assertion reflects steady state rather than whenever the dump happened to land.Per the repo's test rules the assertion must not be weakened or removed — it should be made deterministic.