Register anonymous executable mappings for pre-existing processes#295
Draft
brianrob wants to merge 1 commit into
Draft
Register anonymous executable mappings for pre-existing processes#295brianrob wants to merge 1 commit into
brianrob wants to merge 1 commit into
Conversation
write_environment_modules() synthesizes MMAP2 records from /proc/<pid>/maps at capture start so an already-running process's mappings are known to the unwinder. It skipped every entry without a backing file path, which discarded anonymous executable mappings such as .NET JIT code heaps (used when W^X is disabled or no usable memfd is available, e.g. in some container sandboxes). For a long-running process that JIT code exists before capture starts and is never reported via a live MMAP2 record, so it was left unregistered. CPU samples landing in it then produced single-frame stacks that PerfView/TraceEvent reports as BROKEN (observed in 44-99% of samples in affected processes). Stop skipping path-less mappings; emit anonymous executable mappings with an empty filename so the existing MMAP2 handler registers them as anonymous code (UnwindType::Prolog), matching how live anonymous JIT mappings are handled. Non-executable mappings are still skipped unless all-mmaps capture is enabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Profiling already-running .NET (or other JIT) processes on Linux yields a large fraction of BROKEN stacks in PerfView/TraceEvent — single-frame stacks (just the sampled IP, no callers). A customer saw 44.9% and 75.1% BROKEN in two services.
Root cause
At capture start,
PerfSession::write_environment_modulessynthesizesMMAP2records from/proc/<pid>/mapsso an already-running process's mappings are known to the unwinder. It skipped every entry with no backing file path:.NET JIT code heaps are anonymous executable mappings (no path) when W^X is disabled (
DOTNET_EnableWriteXorExecute=0) or no usablememfdis available (e.g. container seccomp profiles blockingmemfd_create). With W^X on (default), JIT lives in/memfd:doublemappermappings, which do have a path in procfs and were already registered — which is why this often stayed hidden.If a process's JIT code was created before tracing started, no live
MMAP2record is ever emitted for it. With the synthetic enumeration skipping it, the region is never registered, the unwinder can't start there, and every sample in JIT'd code becomes a single-frame BROKEN stack. This is not a PerfView/TraceEvent or version issue — the stacks are genuinely truncated.Fix
Stop skipping path-less mappings in
write_environment_modules; emit anonymous executable mappings with an empty filename so the existingadd_mmap_exechandler classifies them as anonymous code (UnwindType::Prolog), exactly as it already does for live anonymous JIT mappings. Non-executable mappings are still skipped unless all-mmaps capture is requested (captures_all).Notes:
/proc/<pid>/mapsgives an empty path for plain anonymous mappings (the kernel uses//anononly in liveMMAP2records);add_mmap_execalready treats an empty filename as anonymous.MMAP2hook filters only onPROT_EXEC(not path), and processes started during capture get their mappings via that same live path. This synthetic enumeration was the only/proc/mapsreader feeding the unwinder.Verification (record-trace, on-CPU, .NET 10; broken rate measured via TraceEvent)
Repro steps
DOTNET_EnableWriteXorExecute=0and let it warm up so its JIT code is created before tracing starts.record-trace --on-cpusystem-wide.Before this change the process is dominated by a
BROKENnode; after, stacks reach the thread base.