From cd2b088cd301fe8e6ad82f3306f435b13a48072a Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Fri, 8 May 2026 17:10:13 -0400 Subject: [PATCH 01/13] draft a plan on how to do dwarf rewriting --- DWARF-plan.md | 245 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 245 insertions(+) create mode 100644 DWARF-plan.md diff --git a/DWARF-plan.md b/DWARF-plan.md new file mode 100644 index 00000000..febc4068 --- /dev/null +++ b/DWARF-plan.md @@ -0,0 +1,245 @@ +# DWARF Debug-Info Rewriting + +Design doc for issue #169. The goal: when wirm instruments a Wasm +module that carries DWARF debug info, the output module's DWARF must +remain coherent — original instructions still map to their original +`(file, line, col)`; injected instructions inherit their anchor's +`(file, line, col)` so a debugger never "stops inside" wirm-injected +code. + +## TODO + +Work through these in order. Each box is roughly one PR-sized unit. + +- [ ] **0. Validate the testing approach.** Sketch the property/diff + test harness against a single hand-written `.wat` + DWARF input, + end-to-end, before committing to the full design. Spike, throw + it away if needed. +- [ ] **1. Parse-aside.** Add `gimli` dep. Add opt-in flag (`with_dwarf` + on `Module::parse`). When set, recognize `.debug_*` custom + sections and pull them into `ModuleDebugData` (currently a stub + at `src/ir/dwarf.rs`). All other sections continue to flow + through `custom_sections` unchanged. No rewriting yet — encode + should round-trip the parsed-aside DWARF byte-identically. +- [ ] **2. Capture new per-op PCs during encode.** Modify + `encode_function` (`src/ir/module/mod.rs:1202`) to record the + `wasm_encoder::Function::byte_len()` after each emitted op, + producing a `Vec` of new in-function PCs aligned with the + emitted op order. Gated on the same opt-in flag. +- [ ] **3. Track new code-section start.** During `encode_internal`, + record the byte offset where the code section begins in the + output (header bytes + sizes of preceding sections). DWARF PCs + are relative to the code section start. +- [ ] **4. Post-resolve anchor-walk.** After + `resolve_special_instrumentation`, walk every function's + `InstrumentationFlag.{before, after, alternate}` and produce + `Vec<(emit_order_idx, anchor_instr_idx)>` per function. Original + ops map to themselves; injected ops map to their host + instruction. +- [ ] **5. `.debug_line` rewriter.** Walk the input's line program; + emit a new program where each original PC is translated via the + maps from steps 2–4. For every injected PC, emit a row with the + anchor's `(file, line, col, is_stmt, ...)`. +- [ ] **6. `.debug_info` and friends.** Use + `gimli::write::Dwarf::from(&read, &addr_translator)` with an + address translator that consults the same maps. This handles + `DW_AT_low_pc`/`high_pc`, `DW_AT_ranges`, location lists, + aranges, frame info — anything that holds a PC. +- [ ] **7. Re-emit DWARF as custom sections.** Encoded DWARF goes into + the output module's custom sections in the conventional order. +- [ ] **8. Unit/regression tests.** Five or six hand-written `.wat` + inputs run through `wasm-tools parse --generate-dwarf full`, + with hand-checked expectations. Cover: locals added, nop + injected before every op, replacement, block_alt, multi-function + module, `func_exit` cloned to multiple return sites. +- [ ] **9. Differential test.** Two offset-discovery paths + (in-encode capture vs. re-parse-after-encode), assert + byte-identical maps. Catches off-by-ones in step 2. +- [ ] **10. Property test.** `proptest` over (small dwarf-bearing + corpus) × (random instrumentation plans). Single invariant: + `lookup(new_pc)` in output equals `lookup(anchor_orig_pc)` in + input, for every emitted op. +- [ ] **11. Real-binary tests.** Small Rust + C debug builds checked + in, end-to-end DWARF round-trip. +- [ ] **12. libfuzzer target (corpus-based).** Reuse the parse-aside + seeds from steps 8/11; let libfuzzer generate the + instrumentation plan via `Arbitrary`. +- [ ] **13. libfuzzer target (synthesized DWARF).** Helper that emits + a stub `.debug_line` on top of `wasm-smith` output so we get + unbounded variation. Optional, deferred until earlier layers + are stable. +- [ ] **14. Warn on adjacent debug sections.** When DWARF rewriting is + opted in, detect `external_debug_info` and `sourceMappingURL` + custom sections during parse and emit a `log::warn!` (the `log` + crate is already a dep). The user opted into DWARF rewriting, + so they care about debug info coherence — they need to know + these sections exist and that we won't fix them. See "Adjacent + conventions" below. + +## Decisions already made + +These are settled — recorded here so we don't relitigate. + +- **Anchor inheritance policy.** Every instruction injected as + `Before`/`After`/`Alternate` on a target op (whether attached + directly or expanded from a special mode) inherits that target op's + `(file, line, col)`. A debugger stepping through the rewritten + module never sees a stop "inside" wirm-injected code — instrumentation + appears to belong to the adjacent original instruction. +- **Anchor mapping happens *after* `resolve_special_instrumentation`.** + By that point every injection is pinned to a specific target + instruction via `InstrumentationFlag.{before, after, alternate}`, + regardless of whether it started as a plain mode or a special one + (`SemanticAfter`, `BlockEntry`, `BlockExit`, `BlockAlt`, func-entry, + func-exit). A single uniform walk over the resolved layout + produces the anchor map. Doing it inside resolve would either miss + the plain modes or duplicate the bookkeeping. +- **Opt-in.** DWARF rewriting adds `gimli` parse/encode work and + per-op offset capture. Both have nontrivial cost. Gated behind a + `with_dwarf: bool` flag on `Module::parse` (sibling of + `with_offsets`). Default is off. +- **For func-exit clones**, the natural anchor is the local op the + clone was pinned to: a `return` for return-site clones, the + function's last op for the implicit fall-through clone. Same + principle for any other special mode that fans out. + +## Current-state notes + +- `src/ir/dwarf.rs` is a one-line stub (`use gimli::read::Dwarf`) and + `gimli` is not yet a dependency. `ir/mod.rs` doesn't pull it in. +- DWARF sections currently pass through opaquely. + `parse_internal` (`src/ir/module/mod.rs:430`) sticks unrecognized + custom sections into `custom_sections`; `encode_internal` + (`src/ir/module/mod.rs:1866`) re-emits them byte-for-byte. After + instrumentation, the output's `.debug_line` points at the wrong + addresses. +- The "original instruction → original PC" half is already done. + `Module::parse(_, _, with_offsets)` in `src/ir/module/mod.rs:148` + threads through to `Instructions::new` + (`src/ir/types.rs:1578`), which records `offset - locals_start` per + op. Exposed via `Instructions::lookup_pc_offset_for`. +- Special-mode resolution lives in + `Module::resolve_special_instrumentation` + (`src/ir/module/mod.rs:730`). It expands + `SemanticAfter`/`BlockEntry`/`BlockExit`/`BlockAlt` plus func-entry + and func-exit into concrete `Before`/`After`/`Alternate` injections + pinned to specific target instructions. The anchor walk runs after + this returns. + +## Maps the rewriter operates on + +For each local function: + +- `orig_instr_idx → orig_pc_in_func`: from `with_offsets`-tracked + parse-time offsets, accounting for locals-bytes subtraction + (already done in `Instructions::new`). +- `emit_order_idx → new_pc_in_func`: from per-op `byte_len()` capture + in `encode_function`. +- `emit_order_idx → anchor_instr_idx`: from the post-resolve walk. + Trivially the identity for original ops. + +Module-level: + +- `orig_code_section_start`: known from parse. +- `new_code_section_start`: tracked during encode. +- `orig_func_start_in_code_section`, `new_func_start_in_code_section`: + per-function. Computed from cumulative function body sizes plus the + function-body-size LEB128 length. + +Together these give: `orig_pc_in_module → new_pc_in_module` for every +original instruction, and `injected_new_pc → anchor_orig_pc` for every +injected op. + +## Sections handled by the rewrite + +PC-bearing (must rewrite): + +- `.debug_line` — line-number program. Walk and rebuild against the + new PC map. +- `.debug_info` — DIEs with `DW_AT_low_pc`, `DW_AT_high_pc`, + `DW_AT_ranges`, location-list references. +- `.debug_loc`, `.debug_loclists` — location lists. +- `.debug_ranges`, `.debug_rnglists` — range lists. +- `.debug_aranges` — address-range lookup. +- `.debug_frame` — call-frame info. + +Pass-through (no PCs): + +- `.debug_abbrev`, `.debug_str`, `.debug_line_str`, `.debug_str_offsets`, + `.debug_pubnames`, `.debug_pubtypes`. + +`gimli::write::Dwarf::from(&read, &translator)` covers the bulk of the +PC-bearing sections; we hand-roll `.debug_line` because the inheritance +logic for injected ops is custom. + +## Open questions + +- Version pinning. `gimli` releases are independent of + `wasm-encoder`/`wasmparser`. Need to confirm a version triple that + agrees on object representations (or use `gimli`'s standalone + read/write types and not try to share types with `wasm-encoder`). +- DWARF v4 vs v5. Real-world Wasm produced by current Rust/clang is + mostly v4. `gimli` handles both, but the rewriter has to round-trip + the input version, not normalize. +- Inlined functions (`DW_TAG_inlined_subroutine`) carry their own + `low_pc`/`high_pc`. Should fall out of the address translator + naturally — flag if testing shows otherwise. +- Cross-CU references. Some DWARF emitters split debug info across + CUs. `gimli::write::Dwarf::from` is supposed to handle this; verify. + +## Components + +DWARF in Wasm is a core-module-level convention. The +[tool-conventions Debugging.md](https://github.com/WebAssembly/tool-conventions/blob/main/Debugging.md) +defines DWARF embedding only for modules — `.debug_*` custom sections +inside a Wasm module, with addresses interpreted as offsets into that +module's code section. The component-model spec does not define a +component-level DWARF convention, and there's no in-flight proposal +for one (verified May 2026). + +For components: each embedded core module carries its own `.debug_*` +custom sections independently. wirm's component path +(`src/ir/component`) should, when DWARF rewriting is enabled, recurse +into each contained core module and apply the module-level rewriter. +Cross-module glue (lifts/lowers/canonical-ABI shims) is component +machinery, not user code — debuggers don't care about source lines +for it, so there's nothing to translate at the component level +itself. + +Component support is additive: do the module path first, then call +into it from the component path. + +## Adjacent conventions (out of scope, but related) + +Both are core-module-level, both currently flow through wirm's +`custom_sections` opaquely, and both have the same +"addresses-go-stale-after-instrumentation" problem. Both are +detectable by custom-section name during parse — when DWARF rewriting +is opted in we should `log::warn!` so the user knows their debug info +is partially incoherent (see TODO #14). The user opted in *because* +they care about debug coherence; silently leaving these stale would +be a footgun. + +- `external_debug_info` custom section: holds a URL pointing to a + side-file containing the DWARF. If the input uses this, the main + module has no `.debug_*` sections to rewrite — but the side-file is + now incoherent with the rewritten module. Options when detected: + (a) refuse to rewrite when this section is present, (b) fetch + + rewrite + re-emit the side-file, (c) strip the section. For now: + warn loudly and pass through. Decide between (a)/(b)/(c) when a + real input forces the question. +- `sourceMappingURL` custom section: source-map-based debugging. + Source maps are byte-offset-based, so they go stale identically to + DWARF. Different format (JSON), separate rewriter. Out of scope for + issue #169 — but warn so the user knows their source map is + pointing at the wrong bytes after instrumentation. + +## Non-goals + +- Generating DWARF for code that didn't have it. If the input has no + `.debug_*` sections, the output has none. +- Generating DWARF for wirm-injected code. The whole point of the + inheritance policy is that injected code is invisible to the + debugger. +- Rewriting `sourceMappingURL` source maps. Tracked above as adjacent. +- Component-level DWARF. No such thing today. From 64645e416133d5d3972a8a010b3cef5d76c4babb Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Fri, 29 May 2026 14:38:17 -0400 Subject: [PATCH 02/13] add opt-in with_dwarf parsing to pull out .debug_* sections --- Cargo.lock | 1 + Cargo.toml | 1 + DWARF-plan.md | 52 +++++++++- fuzz/fuzz_targets/component_concretize.rs | 2 +- fuzz/fuzz_targets/component_instrument.rs | 2 +- fuzz/fuzz_targets/component_roundtrip.rs | 2 +- fuzz/fuzz_targets/component_walks.rs | 2 +- fuzz/fuzz_targets/module_instrument.rs | 2 +- fuzz/fuzz_targets/module_roundtrip.rs | 2 +- src/ir/component/mod.rs | 13 ++- src/ir/component/tests.rs | 24 +++-- src/ir/component/visitor/resolution_tests.rs | 4 +- src/ir/component/visitor/tests.rs | 2 +- src/ir/dwarf.rs | 43 +++++++-- src/ir/mod.rs | 1 + src/ir/module/mod.rs | 47 ++++++++- src/ir/module/test.rs | 42 ++++---- src/iterator/component_iterator.rs | 2 +- src/iterator/module_iterator.rs | 2 +- src/lib.rs | 6 +- tests/common/mod.rs | 6 +- tests/dwarf.rs | 91 ++++++++++++++++++ tests/func_builder.rs | 12 +-- tests/instrumentation/component_level.rs | 4 +- tests/instrumentation/instrumentation_test.rs | 4 +- tests/instrumentation/test_module.rs | 50 +++++----- tests/iterator_test.rs | 22 ++--- tests/round_trip_component.rs | 2 +- tests/round_trip_module.rs | 4 +- tests/round_trip_wast.rs | 4 +- tests/test_inputs/handwritten/dwarf/add.wasm | Bin 0 -> 409 bytes tests/test_inputs/handwritten/dwarf/add.wat | 8 ++ 32 files changed, 343 insertions(+), 116 deletions(-) create mode 100644 tests/dwarf.rs create mode 100644 tests/test_inputs/handwritten/dwarf/add.wasm create mode 100644 tests/test_inputs/handwritten/dwarf/add.wat diff --git a/Cargo.lock b/Cargo.lock index a0e0d16e..ca02043f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1646,6 +1646,7 @@ checksum = "5a5364e9d77fcdeeaa6062ced926ee3381faa2ee02d3eb83a5c27a8825540829" name = "wirm" version = "5.0.1" dependencies = [ + "gimli", "log", "paste", "rayon", diff --git a/Cargo.toml b/Cargo.toml index d901189a..14333052 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -13,6 +13,7 @@ keywords = ["wasm", "WebAssembly", "instrumentation"] include = ["src/**/*.rs", "Cargo.toml"] [dependencies] +gimli = "0.33" log = "0.4.22" paste = "1" rayon = { version = "1.8", optional = true } diff --git a/DWARF-plan.md b/DWARF-plan.md index febc4068..9dad54b6 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -11,10 +11,9 @@ code. Work through these in order. Each box is roughly one PR-sized unit. -- [ ] **0. Validate the testing approach.** Sketch the property/diff - test harness against a single hand-written `.wat` + DWARF input, - end-to-end, before committing to the full design. Spike, throw - it away if needed. +- [x] **0. Validate the testing approach.** Spike confirmed the design + is viable; findings recorded under "Spike findings (step 0)" + below. - [ ] **1. Parse-aside.** Add `gimli` dep. Add opt-in flag (`with_dwarf` on `Module::parse`). When set, recognize `.debug_*` custom sections and pull them into `ModuleDebugData` (currently a stub @@ -51,7 +50,11 @@ Work through these in order. Each box is roughly one PR-sized unit. inputs run through `wasm-tools parse --generate-dwarf full`, with hand-checked expectations. Cover: locals added, nop injected before every op, replacement, block_alt, multi-function - module, `func_exit` cloned to multiple return sites. + module, `func_exit` cloned to multiple return sites. Inputs must + be designed to *expose* miscoherence — see "Spike findings" — so + prefer one row per opcode, distinct (line, col) per opcode, and + either multi-byte injections or enough cumulative shift that no + original op stays inside its old line-program range. - [ ] **9. Differential test.** Two offset-discovery paths (in-encode capture vs. re-parse-after-encode), assert byte-identical maps. Catches off-by-ones in step 2. @@ -234,6 +237,45 @@ be a footgun. issue #169 — but warn so the user knows their source map is pointing at the wrong bytes after instrumentation. +## Spike findings (step 0) + +Spike lived in `/tmp/dwarf-spike{,-rt}` (throwaway). Pipeline verified: +`wasm-tools parse --generate-dwarf full` on a hand-written `.wat` +produces a module with `.debug_*` custom sections; `wasmparser` +collects them and `gimli::DwarfSections::load` parses them. `gimli` +0.33 + `wasmparser` 0.247 cooperate without friction on the read side. + +Two findings shape the rest of the work: + +1. **Row equality is the wrong invariant.** Injecting a single nop + leaves `.debug_line` byte-identical (we don't rewrite it yet), but + every original opcode has moved by one byte — the rows now name + different ops. A naive "input rows == output rows" assertion would + pass. The right invariant is semantic: + `output_lookup(new_pc) == input_lookup(anchor_orig_pc)` for every + emitted op. The test harness in step 8 is built around this, not + row equality. +2. **The semantic check can still pass by accident on tiny inputs.** + Line-program rows define implicit ranges (a row's `(file, line, + col)` is in effect until the next row's address). A small shift can + land each op inside the *next* row's range, which often has the + same `(file, line, col)` by coincidence. Regression inputs must be + designed to break this — distinct `(line, col)` per opcode, line + programs with one row per opcode, and either multi-byte injections + or enough cumulative shift that ops escape their original buckets. + +Implications for downstream steps: + +- The harness can't run automatically on instrumented modules until + the per-op new-PC map (step 2) and the anchor walk (step 4) exist. + Step 0's harness only checks the noop baseline automatically; the + injection case was hand-mocked. +- Step 8 input authoring needs explicit thought, not boilerplate. +- DWARF address convention in wasm-tools-generated output appears to + be "offset from the function-size LEB", not "offset from the code + section payload start". Worth pinning down before step 5 starts + emitting addresses. + ## Non-goals - Generating DWARF for code that didn't have it. If the input has no diff --git a/fuzz/fuzz_targets/component_concretize.rs b/fuzz/fuzz_targets/component_concretize.rs index 5bb605d5..2bf9b0c4 100644 --- a/fuzz/fuzz_targets/component_concretize.rs +++ b/fuzz/fuzz_targets/component_concretize.rs @@ -25,7 +25,7 @@ fuzz_target!(|smith: SmithComponent| { return; } - let comp = match wirm::Component::parse(&bytes, false, false) { + let comp = match wirm::Component::parse(&bytes, false, false, false) { Ok(c) => c, Err(_) => return, }; diff --git a/fuzz/fuzz_targets/component_instrument.rs b/fuzz/fuzz_targets/component_instrument.rs index 5fcb861a..3f2ee93c 100644 --- a/fuzz/fuzz_targets/component_instrument.rs +++ b/fuzz/fuzz_targets/component_instrument.rs @@ -128,7 +128,7 @@ fuzz_target!(|input: (SmithComponent, u8, u8, u8, u8, u8, u8, u8, u8)| { sub.finish() }; - let mut comp = match Component::parse(&bytes, false, false) { + let mut comp = match Component::parse(&bytes, false, false, false) { Ok(c) => c, Err(_) => return, }; diff --git a/fuzz/fuzz_targets/component_roundtrip.rs b/fuzz/fuzz_targets/component_roundtrip.rs index 88522907..75cb675e 100644 --- a/fuzz/fuzz_targets/component_roundtrip.rs +++ b/fuzz/fuzz_targets/component_roundtrip.rs @@ -26,7 +26,7 @@ fuzz_target!(|smith: SmithComponent| { return; } - let comp = match wirm::Component::parse(&bytes, false, false) { + let comp = match wirm::Component::parse(&bytes, false, false, false) { Ok(c) => c, Err(_) => return, }; diff --git a/fuzz/fuzz_targets/component_walks.rs b/fuzz/fuzz_targets/component_walks.rs index 35b6b8f8..6075a1e3 100644 --- a/fuzz/fuzz_targets/component_walks.rs +++ b/fuzz/fuzz_targets/component_walks.rs @@ -438,7 +438,7 @@ fuzz_target!(|smith: SmithComponent| { return; } - let comp = match wirm::Component::parse(&bytes, false, false) { + let comp = match wirm::Component::parse(&bytes, false, false, false) { Ok(c) => c, Err(_) => return, }; diff --git a/fuzz/fuzz_targets/module_instrument.rs b/fuzz/fuzz_targets/module_instrument.rs index 51c88614..a024d16f 100644 --- a/fuzz/fuzz_targets/module_instrument.rs +++ b/fuzz/fuzz_targets/module_instrument.rs @@ -46,7 +46,7 @@ fuzz_target!(|smith: SmithModule| { return; } - let mut module = match wirm::Module::parse(&bytes, false, false) { + let mut module = match wirm::Module::parse(&bytes, false, false, false) { Ok(m) => m, Err(_) => return, }; diff --git a/fuzz/fuzz_targets/module_roundtrip.rs b/fuzz/fuzz_targets/module_roundtrip.rs index 68ddaa5e..58df3a02 100644 --- a/fuzz/fuzz_targets/module_roundtrip.rs +++ b/fuzz/fuzz_targets/module_roundtrip.rs @@ -28,7 +28,7 @@ fuzz_target!(|smith: SmithModule| { return; } - let module = match wirm::Module::parse(&bytes, false, false) { + let module = match wirm::Module::parse(&bytes, false, false, false) { Ok(m) => m, Err(_) => return, }; diff --git a/src/ir/component/mod.rs b/src/ir/component/mod.rs index 54193895..64282ace 100644 --- a/src/ir/component/mod.rs +++ b/src/ir/component/mod.rs @@ -197,6 +197,7 @@ impl<'a> Component<'a> { bytes, false, false, + false, Parser::new(0), 0, sub_space_id, @@ -707,6 +708,9 @@ impl<'a> Component<'a> { /// Set enable_multi_memory to `true` to support parsing modules using multiple memories. /// Set with_offsets to `true` to save opcode pc offset metadata during parsing /// (can be used to determine the static pc offset inside a function body of the start of any opcode). + /// Set with_dwarf to `true` to lift `.debug_*` custom sections aside in each + /// contained core module so they can be rewritten coherently with the rest + /// of the module. See [`crate::Module::parse`] for details. /// /// # Example /// @@ -715,12 +719,13 @@ impl<'a> Component<'a> { /// /// let file = "path_to_file"; /// let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - /// let comp = Component::parse(&buff, false, false).unwrap(); + /// let comp = Component::parse(&buff, false, false, false).unwrap(); /// ``` pub fn parse( wasm: &'_ [u8], enable_multi_memory: bool, with_offsets: bool, + with_dwarf: bool, ) -> Result, Error> { let parser = Parser::new(0); @@ -732,6 +737,7 @@ impl<'a> Component<'a> { wasm, enable_multi_memory, with_offsets, + with_dwarf, parser, 0, space_id, @@ -746,6 +752,7 @@ impl<'a> Component<'a> { wasm: &'a [u8], enable_multi_memory: bool, with_offsets: bool, + with_dwarf: bool, parser: Parser, start: usize, space_id: ScopeId, @@ -996,6 +1003,7 @@ impl<'a> Component<'a> { &wasm[unchecked_range.start - start..unchecked_range.end - start], enable_multi_memory, with_offsets, + with_dwarf, parser, )?; store_handle.borrow_mut().assign_assumed_id( @@ -1025,6 +1033,7 @@ impl<'a> Component<'a> { &wasm[unchecked_range.start - start..unchecked_range.end - start], enable_multi_memory, with_offsets, + with_dwarf, parser, unchecked_range.start, sub_space_id, @@ -1244,7 +1253,7 @@ impl<'a> Component<'a> { /// /// let file = "path/to/file.wasm"; /// let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - /// let mut comp = Component::parse(&buff, false, false).unwrap(); + /// let mut comp = Component::parse(&buff, false, false, false).unwrap(); /// let result = comp.encode(); /// ``` pub fn encode(&self) -> crate::ir::types::Result> { diff --git a/src/ir/component/tests.rs b/src/ir/component/tests.rs index 491d6629..88943fe6 100644 --- a/src/ir/component/tests.rs +++ b/src/ir/component/tests.rs @@ -14,7 +14,7 @@ fn bytes(wat: &str) -> Vec { } fn parsed(b: &[u8]) -> Component<'_> { - Component::parse(b, false, false).unwrap() + Component::parse(b, false, false, false).unwrap() } /// Resolve the ref carried by `comp.exports[export_idx]` against `comp`'s own index space. @@ -317,7 +317,9 @@ fn concretize_export_all_patterns_same_signature() { // Round-trip through owned bytes to satisfy the `'static` bound. let bytes = wat::parse_str(wat).expect("WAT parse failed"); let bytes: &'static [u8] = Box::leak(bytes.into_boxed_slice()); - let comp = Box::leak(Box::new(Component::parse(bytes, false, false).unwrap())); + let comp = Box::leak(Box::new( + Component::parse(bytes, false, false, false).unwrap(), + )); let Some(ConcreteType::Instance { mut funcs, .. }) = comp.concretize_export("iface") else { panic!("expected Instance"); }; @@ -461,7 +463,9 @@ fn first_param_type(wat: &str) -> ConcreteValType<'_> { let bytes = bytes(wat); // Safety: we box-leak to get 'static for simplicity in tests. let bytes: &'static [u8] = Box::leak(bytes.into_boxed_slice()); - let comp = Box::leak(Box::new(Component::parse(bytes, false, false).unwrap())); + let comp = Box::leak(Box::new( + Component::parse(bytes, false, false, false).unwrap(), + )); let Some(ConcreteType::Instance { funcs, .. }) = comp.concretize_import("iface") else { panic!("expected Instance"); }; @@ -718,10 +722,10 @@ fn server_and_middleware_concretize_to_same_func_type() { let server_b_s: &'static [u8] = Box::leak(server_b.into_boxed_slice()); let middleware_a_s: &'static [u8] = Box::leak(middleware_a.into_boxed_slice()); let sb = Box::leak(Box::new( - Component::parse(server_b_s, false, false).unwrap(), + Component::parse(server_b_s, false, false, false).unwrap(), )); let ma = Box::leak(Box::new( - Component::parse(middleware_a_s, false, false).unwrap(), + Component::parse(middleware_a_s, false, false, false).unwrap(), )); let Some(ConcreteType::Instance { @@ -800,10 +804,10 @@ fn server_and_middleware_same_func_type_explicit_type_decl() { let server_b_s: &'static [u8] = Box::leak(server_b.into_boxed_slice()); let middleware_a_s: &'static [u8] = Box::leak(middleware_a.into_boxed_slice()); let sb = Box::leak(Box::new( - Component::parse(server_b_s, false, false).unwrap(), + Component::parse(server_b_s, false, false, false).unwrap(), )); let ma = Box::leak(Box::new( - Component::parse(middleware_a_s, false, false).unwrap(), + Component::parse(middleware_a_s, false, false, false).unwrap(), )); let Some(ConcreteType::Instance { @@ -869,7 +873,9 @@ fn concretize_func_param_via_alias_to_imported_instance_type_direct() { )"#, ); let b_s: &'static [u8] = Box::leak(b.into_boxed_slice()); - let comp = Box::leak(Box::new(Component::parse(b_s, false, false).unwrap())); + let comp = Box::leak(Box::new( + Component::parse(b_s, false, false, false).unwrap(), + )); let Some(ConcreteType::Func(ft)) = comp.concretize_import("handle") else { panic!("expected ConcreteType::Func for 'handle' import"); @@ -1026,7 +1032,7 @@ fn count_wirm_sections_by_kind( /// payloads of that kind. Returns the parsed component for any /// follow-up assertions the caller wants to make. fn assert_section_count_invariant<'a>(bytes: &'a [u8]) -> Component<'a> { - let comp = Component::parse(bytes, false, false).expect("wirm parse"); + let comp = Component::parse(bytes, false, false, false).expect("wirm parse"); let binary_counts = count_binary_sections_by_kind(bytes); let wirm_counts = count_wirm_sections_by_kind(&comp); assert_eq!( diff --git a/src/ir/component/visitor/resolution_tests.rs b/src/ir/component/visitor/resolution_tests.rs index b02f7d46..b5fab797 100644 --- a/src/ir/component/visitor/resolution_tests.rs +++ b/src/ir/component/visitor/resolution_tests.rs @@ -188,7 +188,7 @@ impl<'a> ComponentVisitor<'a> for ParanoidVisitor { } fn run_on_bytes(bytes: Vec) -> usize { - let comp = Component::parse(&bytes, false, false).expect("component parse failed"); + let comp = Component::parse(&bytes, false, false, false).expect("component parse failed"); let mut structural = ParanoidVisitor::default(); walk_structural(&comp, &mut structural); @@ -222,7 +222,7 @@ fn run_paranoid_file(path: &str) { /// Parse `wat` and run `walk_structural` with `visitor`. fn walk_wat ComponentVisitor<'a>>(wat: &str, visitor: &mut V) { let bytes = wat::parse_str(wat).expect("WAT parse failed"); - let comp = Component::parse(&bytes, false, false).expect("component parse failed"); + let comp = Component::parse(&bytes, false, false, false).expect("component parse failed"); walk_structural(&comp, visitor); } diff --git a/src/ir/component/visitor/tests.rs b/src/ir/component/visitor/tests.rs index e534d4ef..6d57486c 100644 --- a/src/ir/component/visitor/tests.rs +++ b/src/ir/component/visitor/tests.rs @@ -592,7 +592,7 @@ fn test_event_generation(label: &str, bytes: &[u8]) { let original = wasmprinter::print_bytes(bytes).expect("couldn't convert original Wasm to wat"); println!("original: {:?}", original); - let comp = Component::parse(bytes, false, false).expect("Unable to parse"); + let comp = Component::parse(bytes, false, false, false).expect("Unable to parse"); let evts_struct = get_events(&comp, get_structural_events); let evts_topo = get_events(&comp, get_topological_events); check_event_validity(&evts_struct, &evts_topo); diff --git a/src/ir/dwarf.rs b/src/ir/dwarf.rs index 426ff461..0e37be0c 100644 --- a/src/ir/dwarf.rs +++ b/src/ir/dwarf.rs @@ -1,8 +1,35 @@ -use gimli::read::{Dwarf}; - -/// The DWARF debug section in input WebAssembly binary. -#[derive(Debug, Default)] -pub struct ModuleDebugData { - /// DWARF debug data - pub dwarf: Dwarf> -} \ No newline at end of file +//! DWARF debug-info lifted aside from a Wasm module's custom sections. +//! +//! When `Module::parse` is called with `with_dwarf = true`, `.debug_*` custom +//! sections are diverted here instead of flowing through `custom_sections` +//! opaquely. Keeping them separate lets DWARF be handled coherently with an +//! instrumented module rather than passed through byte-for-byte. + +use crate::ir::types::CustomSection; + +/// Holds the `.debug_*` custom sections lifted out of a Wasm module's +/// `custom_sections` list at parse time. +#[derive(Clone, Debug, Default)] +pub struct ModuleDebugData<'a> { + /// `.debug_*` sections in the order they appeared in the input module. + /// Encode emits them in this order after the rest of `custom_sections`. + pub(crate) sections: Vec>, +} + +impl<'a> ModuleDebugData<'a> { + pub(crate) fn from_sections(sections: Vec>) -> Self { + Self { sections } + } + + /// Read-only view of the underlying `.debug_*` custom sections. + pub fn sections(&self) -> &[CustomSection<'a>] { + &self.sections + } + + /// Whether a custom-section name matches the DWARF convention. Includes + /// any `.debug_*` name so non-standard extensions still round-trip via + /// `ModuleDebugData` rather than escaping into `custom_sections`. + pub fn is_dwarf_section_name(name: &str) -> bool { + name.starts_with(".debug_") + } +} diff --git a/src/ir/mod.rs b/src/ir/mod.rs index e974b239..55b3dfcc 100644 --- a/src/ir/mod.rs +++ b/src/ir/mod.rs @@ -4,6 +4,7 @@ use std::ops::{Deref, Index, IndexMut}; use std::slice::SliceIndex; pub mod component; +pub mod dwarf; pub mod function; mod helpers; pub mod id; diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index f77c5ee5..c3799bce 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -5,6 +5,7 @@ use super::types::{ }; use crate::error::Error; use crate::error::Error::{InstrumentationError, IO}; +use crate::ir::dwarf::ModuleDebugData; use crate::ir::function::FunctionModifier; use crate::ir::id::{ CustomSectionID, DataSegmentID, FunctionID, GlobalID, ImportsID, LocalID, MemoryID, TypeID, @@ -87,6 +88,11 @@ pub struct Module<'a> { pub tags: Vec, /// Custom Sections pub custom_sections: CustomSections<'a>, + /// `.debug_*` custom sections lifted aside for DWARF rewriting. `Some` + /// when `Module::parse` was called with `with_dwarf = true`; `None` + /// otherwise. When `Some`, `.debug_*` sections do *not* appear in + /// `custom_sections` — they flow through `debug` instead. + pub debug: Option>, /// Number of local functions (not counting imported functions) pub(crate) num_local_functions: u32, /// Number of local globals (not counting imported globals) @@ -132,6 +138,10 @@ impl<'a> Module<'a> { /// Set enable_multi_memory to `true` to support parsing modules using multiple memories. /// Set with_offsets to `true` to save opcode pc offset metadata during parsing /// (can be used to determine the static pc offset inside a function body of the start of any opcode). + /// Set with_dwarf to `true` to lift `.debug_*` custom sections aside into + /// [`Module::debug`] so they can be rewritten coherently with the rest of the + /// module. When `false`, DWARF sections flow through `custom_sections` + /// opaquely (the default — addresses go stale after instrumentation). /// /// # Example /// @@ -140,15 +150,16 @@ impl<'a> Module<'a> { /// /// let file = "path_to_file"; /// let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - /// let module = Module::parse(&buff, false, false).unwrap(); + /// let module = Module::parse(&buff, false, false, false).unwrap(); /// ``` pub fn parse( wasm: &'a [u8], enable_multi_memory: bool, with_offsets: bool, + with_dwarf: bool, ) -> Result { let parser = Parser::new(0); - Module::parse_internal(wasm, enable_multi_memory, with_offsets, parser) + Module::parse_internal(wasm, enable_multi_memory, with_offsets, with_dwarf, parser) } fn parse_body( @@ -212,6 +223,7 @@ impl<'a> Module<'a> { wasm: &'a [u8], enable_multi_memory: bool, with_offsets: bool, + with_dwarf: bool, parser: Parser, ) -> Result { #[cfg(feature = "parallel")] @@ -231,6 +243,11 @@ impl<'a> Module<'a> { let mut start = None; let mut data_section_count = None; let mut custom_sections = vec![]; + // When `with_dwarf` is on, `.debug_*` custom sections divert into + // here instead of `custom_sections`. The Option distinguishes + // "opted in with no DWARF found" (Some(empty)) from "didn't opt in" + // (None). + let mut debug_sections: Option>> = with_dwarf.then(Vec::new); let mut tags: Vec = vec![]; let mut module_name: Option = None; @@ -428,6 +445,16 @@ impl<'a> Module<'a> { } } Payload::CustomSection(custom_section_reader) => { + let cs_name = custom_section_reader.name(); + if let Some(debug) = debug_sections.as_mut() { + if ModuleDebugData::is_dwarf_section_name(cs_name) { + debug.push(CustomSection { + name: cs_name, + data: Cow::Borrowed(custom_section_reader.data()), + }); + continue; + } + } match custom_section_reader.as_known() { wasmparser::KnownCustom::Name(name_section_reader) => { for subsection in name_section_reader { @@ -669,6 +696,7 @@ impl<'a> Module<'a> { data, tags, custom_sections: CustomSections::new(custom_sections), + debug: debug_sections.map(ModuleDebugData::from_sections), num_local_functions: code_sections_length as u32, num_local_globals: num_globals, num_local_tables: num_tables, @@ -718,7 +746,7 @@ impl<'a> Module<'a> { /// /// let file = "path_to_file"; /// let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - /// let mut module = Module::parse(&buff, false, false).unwrap(); + /// let mut module = Module::parse(&buff, false, false, false).unwrap(); /// let result = module.encode(); /// ``` pub fn encode(&self) -> types::Result> { @@ -1874,6 +1902,19 @@ impl<'a> Module<'a> { }); } + // Re-emit DWARF custom sections held aside in `debug`. They follow + // the rest of the custom sections; for inputs that already kept DWARF + // at the tail (the wasm-tools convention) this preserves byte + // ordering. + if let Some(debug) = tmp.debug.as_ref() { + for section in debug.sections() { + module.section(&wasm_encoder::CustomSection { + name: std::borrow::Cow::Borrowed(section.name), + data: section.data.clone(), + }); + } + } + Ok((module, side_effects)) } diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index b3cc7051..18d20286 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -19,7 +19,7 @@ mod validate; #[test] fn test_add_local_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add local func @@ -44,7 +44,7 @@ fn test_add_local_func() { #[test] fn test_add_import_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add imported func @@ -65,7 +65,7 @@ fn test_add_import_func() { #[test] fn test_add_local_then_imported_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add local function @@ -93,7 +93,7 @@ fn test_add_local_then_imported_func() { #[test] fn test_add_imported_then_local_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add imported func @@ -122,7 +122,7 @@ fn test_add_imported_then_local_func() { #[test] fn test_add_then_delete_local_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add local function @@ -149,7 +149,7 @@ fn test_add_then_delete_local_func() { #[test] fn test_delete_local_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // delete local function @@ -170,7 +170,7 @@ fn test_delete_local_func() { #[test] fn test_add_then_delete_imported_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add imported func @@ -195,7 +195,7 @@ fn test_add_then_delete_imported_func() { #[test] fn test_delete_imported_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // delete imported function @@ -214,7 +214,7 @@ fn test_delete_imported_func() { #[test] fn test_delete_local_and_imported_func() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // delete local function @@ -239,7 +239,7 @@ fn test_delete_local_and_imported_func() { #[test] fn test_convert_import_fn_to_local() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // convert the import to a function @@ -271,7 +271,7 @@ fn test_convert_import_fn_to_local() { #[test] fn test_convert_local_fn_to_import() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // convert local func to import @@ -295,7 +295,7 @@ fn test_convert_local_fn_to_import() { #[test] fn test_set_fn_name_import_through_import() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); let mut new_import_names = HashMap::new(); @@ -314,7 +314,7 @@ fn test_set_fn_name_import_through_import() { #[test] fn test_set_fn_name_import_through_module() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); let mut new_import_names = HashMap::new(); @@ -333,7 +333,7 @@ fn test_set_fn_name_import_through_module() { #[test] fn test_set_fn_name_local_through_functions() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); let fid = FunctionID(10); @@ -353,7 +353,7 @@ fn test_set_fn_name_local_through_functions() { #[test] fn test_set_fn_name_local_through_module() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); let fid = FunctionID(10); @@ -373,7 +373,7 @@ fn test_set_fn_name_local_through_module() { #[test] fn test_set_fn_name_local_through_func_builder() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); let mut new_func_names = HashMap::new(); @@ -417,7 +417,7 @@ fn test_set_fn_name_local_through_func_builder() { #[test] fn test_create_and_add_global() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add a local global @@ -451,7 +451,7 @@ fn test_create_and_add_global() { #[test] fn test_add_imported_global() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); // add an imported global @@ -487,7 +487,7 @@ fn test_add_imported_global() { #[test] fn test_delete_global() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); module.delete_global(GlobalID(2)); @@ -505,7 +505,7 @@ fn test_delete_global() { #[test] fn test_delete_imported_global() { let (buff, mut init_state) = setup(); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); state_assertions(&module, &init_state, false); module.delete_global(GlobalID(0)); @@ -670,7 +670,7 @@ fn is_valid( // reload from file let buff = std::fs::read(output_wasm_path).unwrap(); - let new_module = Module::parse(&buff, false, false).expect("Unable to parse"); + let new_module = Module::parse(&buff, false, false, false).expect("Unable to parse"); for (id, name) in new_import_names { assert_eq!( diff --git a/src/iterator/component_iterator.rs b/src/iterator/component_iterator.rs index 9ffb7d16..00a9cb26 100644 --- a/src/iterator/component_iterator.rs +++ b/src/iterator/component_iterator.rs @@ -108,7 +108,7 @@ impl<'b> Inject<'b> for ComponentIterator<'_, 'b> { /// /// let file = "path_to_file"; /// let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - /// let mut component = Component::parse(&buff, false, false).expect("Unable to parse"); + /// let mut component = Component::parse(&buff, false, false, false).expect("Unable to parse"); /// let mut comp_it = ComponentIterator::new(&mut component, HashMap::new()); /// /// // Everytime there is a `call 1` instruction we want to inject an `i32.const 0` diff --git a/src/iterator/module_iterator.rs b/src/iterator/module_iterator.rs index e7e55d4c..6ba536bf 100644 --- a/src/iterator/module_iterator.rs +++ b/src/iterator/module_iterator.rs @@ -71,7 +71,7 @@ impl<'b> Inject<'b> for ModuleIterator<'_, 'b> { /// let file = "path_to_file"; /// let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); /// // Must use `parse_only_module` here as we are only concerned about a Module and not a module that is inside a Component - /// let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + /// let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); /// let mut module_it = ModuleIterator::new(&mut module, &vec![]); /// /// // Everytime there is a `call 1` instruction we want to inject an `i32.const 0` diff --git a/src/lib.rs b/src/lib.rs index b2d613de..a150662b 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -42,7 +42,7 @@ //! use wirm::wasmparser::Operator; //! //! let bytes = std::fs::read("input.wasm").unwrap(); -//! let mut module = Module::parse(&bytes, false, false).unwrap(); +//! let mut module = Module::parse(&bytes, false, false, false).unwrap(); //! //! { //! // Iterator walks function bodies one instruction at a time. @@ -134,7 +134,7 @@ //! use wirm::ir::component::concrete::ConcreteType; //! //! let bytes = std::fs::read("example.wasm").unwrap(); -//! let comp = Component::parse(&bytes, false, false).unwrap(); +//! let comp = Component::parse(&bytes, false, false, false).unwrap(); //! //! if let Some(ConcreteType::Instance { funcs, .. }) = //! comp.concretize_import("wasi:http/handler@0.3.0-draft") @@ -212,7 +212,7 @@ //! } //! //! let bytes = std::fs::read("example.wasm").unwrap(); -//! let comp = Component::parse(&bytes, false, false).unwrap(); +//! let comp = Component::parse(&bytes, false, false, false).unwrap(); //! let mut finder = TypesFromImports::default(); //! walk_structural(&comp, &mut finder); //! ``` diff --git a/tests/common/mod.rs b/tests/common/mod.rs index 4cb77056..91fa537c 100644 --- a/tests/common/mod.rs +++ b/tests/common/mod.rs @@ -131,7 +131,7 @@ where F: for<'a, 'b> FnOnce(&mut ModuleIterator<'a, 'b>), { let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); { let mut mod_it = ModuleIterator::new(&mut module, &vec![]); instrument(&mut mod_it); @@ -148,7 +148,7 @@ where F: for<'a, 'b> FnOnce(&mut ComponentIterator<'a, 'b>), { let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut component = Component::parse(&buff, false, false).expect("Unable to parse"); + let mut component = Component::parse(&buff, false, false, false).expect("Unable to parse"); { let mut comp_it = ComponentIterator::new(&mut component, HashMap::new()); instrument(&mut comp_it); @@ -237,7 +237,7 @@ pub fn validate_module_instr( F: for<'a, 'b> FnOnce(&mut ModuleIterator<'a, 'b>), { let buff = wat::parse_str(wat_src).expect("couldn't parse WAT"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); { let mut mod_it = ModuleIterator::new(&mut module, &vec![]); instrument(&mut mod_it); diff --git a/tests/dwarf.rs b/tests/dwarf.rs new file mode 100644 index 00000000..b1d4447c --- /dev/null +++ b/tests/dwarf.rs @@ -0,0 +1,91 @@ +//! Tests for parse-aside DWARF handling (`with_dwarf = true`). +//! +//! When `with_dwarf` is on, `.debug_*` custom sections lift into +//! `Module::debug` instead of `custom_sections`, and encode re-emits the +//! section bytes verbatim. +//! +//! We assert on *DWARF section content* (not whole-module bytes) because +//! wirm regenerates other sections like `name` and isn't byte-identical +//! end-to-end even without instrumentation. + +use std::collections::BTreeMap; +use std::path::PathBuf; + +use wirm::ir::module::Module; + +const DWARF_SECTION_NAMES: &[&str] = &[".debug_abbrev", ".debug_str", ".debug_line", ".debug_info"]; + +fn input_path() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/test_inputs/handwritten/dwarf/add.wasm") +} + +/// Pulls every `.debug_*` custom section's bytes out, keyed by name. +/// `BTreeMap` so equality is order-independent — encode may legitimately +/// place DWARF sections in a different position relative to other custom +/// sections. +fn debug_section_bytes(wasm: &[u8]) -> BTreeMap> { + let mut out = BTreeMap::new(); + for payload in wasmparser::Parser::new(0).parse_all(wasm) { + if let wasmparser::Payload::CustomSection(cs) = payload.expect("valid wasm") { + if cs.name().starts_with(".debug_") { + out.insert(cs.name().to_string(), cs.data().to_vec()); + } + } + } + out +} + +/// `with_dwarf=false` leaves DWARF flowing through `custom_sections`. The +/// section bytes themselves still round-trip verbatim, since `custom_sections` +/// has always been opaque pass-through. +#[test] +fn opaque_round_trip_preserves_dwarf_section_bytes() { + let input = std::fs::read(input_path()).unwrap(); + let module = Module::parse(&input, false, false, false).unwrap(); + assert!( + module.debug.is_none(), + "with_dwarf=false should leave Module::debug as None", + ); + + let output = module.encode().unwrap(); + assert_eq!(debug_section_bytes(&input), debug_section_bytes(&output)); +} + +/// `with_dwarf=true` populates `Module::debug` with the `.debug_*` sections +/// in encounter order, removes them from `custom_sections`, and re-emits them +/// verbatim during encode. +#[test] +fn parse_aside_lifts_dwarf_into_debug_field() { + let input = std::fs::read(input_path()).unwrap(); + let module = Module::parse(&input, false, false, true).unwrap(); + + let debug = module.debug.as_ref().expect("debug present"); + let names: Vec<&str> = debug.sections().iter().map(|s| s.name).collect(); + assert_eq!(names, DWARF_SECTION_NAMES); + + for section in module.custom_sections.iter() { + assert!( + !section.name.starts_with(".debug_"), + "{} leaked into custom_sections", + section.name, + ); + } + + let output = module.encode().unwrap(); + assert_eq!(debug_section_bytes(&input), debug_section_bytes(&output)); +} + +/// `with_dwarf=true` on a module with no DWARF still yields `Some(empty)` — +/// distinguishes "user opted in, no DWARF was present" from "user didn't opt +/// in". +#[test] +fn parse_aside_empty_when_input_has_no_dwarf() { + let wat = "(module (func))"; + let wasm = wat::parse_str(wat).expect("wat compiles"); + let module = Module::parse(&wasm, false, false, true).unwrap(); + let debug = module + .debug + .as_ref() + .expect("debug present even when empty"); + assert!(debug.sections().is_empty()); +} diff --git a/tests/func_builder.rs b/tests/func_builder.rs index 0f9fed15..377ab627 100644 --- a/tests/func_builder.rs +++ b/tests/func_builder.rs @@ -33,7 +33,7 @@ fn func_builder_set_name_is_retained_after_round_trip() { // Round-trip the module and confirm the name section preserves the name. let bytes = module.encode().expect("encode failed"); - let reparsed = Module::parse(&bytes, false, false).expect("reparse failed"); + let reparsed = Module::parse(&bytes, false, false, false).expect("reparse failed"); assert_eq!( reparsed .functions @@ -51,7 +51,7 @@ fn func_builder_set_name_is_retained_after_round_trip() { fn func_builder_set_name_is_retained_when_appended_to_parsed_module() { let file_name = "tests/test_inputs/handwritten/modules/_start.wat"; let wasm = wat::parse_file(file_name).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&wasm, false, false).expect("Unable to parse"); + let mut module = Module::parse(&wasm, false, false, false).expect("Unable to parse"); let expected = "appended_via_builder"; let mut builder = FunctionBuilder::new(&[], &[]); @@ -61,7 +61,7 @@ fn func_builder_set_name_is_retained_when_appended_to_parsed_module() { let fid = builder.finish_module(&mut module); let bytes = module.encode().expect("encode failed"); - let reparsed = Module::parse(&bytes, false, false).expect("reparse failed"); + let reparsed = Module::parse(&bytes, false, false, false).expect("reparse failed"); assert_eq!( reparsed .functions @@ -99,7 +99,7 @@ fn run_fac_wirm() { fn run_start_wirm() { let file_name = "tests/test_inputs/handwritten/modules/start.wat"; let wasm = wat::parse_file(file_name).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&wasm, false, false).expect("Unable to parse"); + let mut module = Module::parse(&wasm, false, false, false).expect("Unable to parse"); let start_fun_id = module.start.unwrap(); let mut function_builder = module.functions.get_fn_modifier(start_fun_id).unwrap(); @@ -122,7 +122,7 @@ fn run_start_wirm() { fn run_start_wirm_default() { let file_name = "tests/test_inputs/handwritten/modules/start.wat"; let wasm = wat::parse_file(file_name).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&wasm, false, false).expect("Unable to parse"); + let mut module = Module::parse(&wasm, false, false, false).expect("Unable to parse"); let start_fun_id = module.start.unwrap(); let mut function_builder = module.functions.get_fn_modifier(start_fun_id).unwrap(); @@ -138,7 +138,7 @@ fn run_start_wirm_default() { fn add_import_and_local_fn_then_iterate() { let file_name = "tests/test_inputs/handwritten/modules/_start.wat"; let wasm = wat::parse_file(file_name).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&wasm, false, false).expect("Unable to parse"); + let mut module = Module::parse(&wasm, false, false, false).expect("Unable to parse"); // add an imported function AND THEN a new local function module.add_import_func("new".to_string(), "import".to_string(), TypeID(0)); assert_eq!(module.num_import_func(), 1); diff --git a/tests/instrumentation/component_level.rs b/tests/instrumentation/component_level.rs index b0996f8c..56959049 100644 --- a/tests/instrumentation/component_level.rs +++ b/tests/instrumentation/component_level.rs @@ -13,7 +13,7 @@ fn whamm_side_effects() { let file = "tests/test_inputs/spin/hello_world.wat"; let output_wasm_path = format!("{TEST_DEBUG_DIR}/whamm_side_effects.wasm"); let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut component = Component::parse(&buff, false, false).expect("Unable to parse"); + let mut component = Component::parse(&buff, false, false, false).expect("Unable to parse"); let lib_path = "tests/test_inputs/whamm/whamm_core.wasm"; let lib_buff = wat::parse_file(lib_path).expect("couldn't convert the input wat to Wasm"); @@ -68,7 +68,7 @@ pub fn configure_component_libraries<'a>( lib_bytes: &'a [u8], ) { let wasi_name = "wasi_snapshot_preview1"; - let lib_wasm = Component::parse(lib_bytes, false, true).unwrap(); + let lib_wasm = Component::parse(lib_bytes, false, true, false).unwrap(); // Create an instance type that defines the library let mut decls = vec![]; diff --git a/tests/instrumentation/instrumentation_test.rs b/tests/instrumentation/instrumentation_test.rs index 3d6a9a76..ae6ed3d4 100644 --- a/tests/instrumentation/instrumentation_test.rs +++ b/tests/instrumentation/instrumentation_test.rs @@ -22,7 +22,7 @@ use crate::common::{ fn no_injection() { let file = "tests/test_inputs/handwritten/components/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut component = Component::parse(&buff, false, false).expect("Unable to parse"); + let mut component = Component::parse(&buff, false, false, false).expect("Unable to parse"); let mut comp_it = ComponentIterator::new(&mut component, HashMap::new()); let interested = Operator::Call { function_index: 1 }; @@ -1385,7 +1385,7 @@ fn test_semantic_after_simple_2br_table() { fn add_imports_when_has_start_func() { let file = "tests/test_inputs/instr_testing/modules/add-imports-when-has-start-func.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); module.add_import_func("ima".to_string(), "new_import".to_string(), TypeID(0)); module.add_import_func("ya_dont".to_string(), "say".to_string(), TypeID(0)); let result = module.encode().expect("error"); diff --git a/tests/instrumentation/test_module.rs b/tests/instrumentation/test_module.rs index a025e31c..81f43baa 100644 --- a/tests/instrumentation/test_module.rs +++ b/tests/instrumentation/test_module.rs @@ -14,7 +14,7 @@ use crate::common::check_instrumentation_encoding; fn test_fn_types() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); assert_eq!( *module.functions.get_kind(FunctionID(0)), @@ -50,7 +50,7 @@ fn test_fn_types() { fn test_exports() { let file = "tests/test_inputs/instr_testing/modules/function_modification/export_deletion.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // Get func ID by name assert_eq!( @@ -86,7 +86,7 @@ fn test_exports() { fn test_import_delete() { let file = "tests/test_inputs/instr_testing/modules/function_modification/import_delete.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); let id = module.imports.find("bogus".to_string(), "hi".to_string()); let fid = module @@ -112,7 +112,7 @@ fn test_import_delete() { fn test_local_fn_delete() { let file = "tests/test_inputs/instr_testing/modules/function_modification/local_fn_delete.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.delete_func(FunctionID(2)); @@ -129,7 +129,7 @@ fn test_local_fn_delete() { fn test_panic_call_delete() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.delete_func(FunctionID(1)); @@ -142,7 +142,7 @@ fn test_renumber_fn_id() { let file = "tests/test_inputs/instr_testing/modules/function_modification/local_fn_renumber.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.delete_func(FunctionID(1)); @@ -159,7 +159,7 @@ fn test_middle_import_to_local() { let file = "tests/test_inputs/instr_testing/modules/function_modification/middle_import_to_local.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); let mut builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); builder.i32_const(1); @@ -182,7 +182,7 @@ fn test_first_import_to_local() { let file = "tests/test_inputs/instr_testing/modules/function_modification/first_import_to_local.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); let mut builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); builder.i32_const(1); @@ -205,7 +205,7 @@ fn test_last_import_to_local() { let file = "tests/test_inputs/instr_testing/modules/function_modification/last_import_to_local.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); let mut builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); builder.i32_const(1); @@ -228,7 +228,7 @@ fn test_all_import_to_local() { let file = "tests/test_inputs/instr_testing/modules/function_modification/all_import_to_local.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // Convert all to local let mut first_builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); @@ -265,7 +265,7 @@ fn test_some_import_to_local() { let file = "tests/test_inputs/instr_testing/modules/function_modification/some_import_to_local.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // Convert all to local let mut first_builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); @@ -295,7 +295,7 @@ fn test_middle_import_to_local_import_delete() { let file = "tests/test_inputs/instr_testing/modules/function_modification/middle_import_to_local_import_delete.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); let mut builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); builder.i32_const(1); @@ -320,7 +320,7 @@ fn test_middle_import_to_local_local_delete() { let file = "tests/test_inputs/instr_testing/modules/function_modification/middle_import_to_local_local_delete.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); let mut builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); builder.i32_const(1); @@ -345,7 +345,7 @@ fn test_middle_import_to_local_local_delete() { fn test_add_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/add_import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.add_import_func("wirm".to_string(), "better".to_string(), TypeID(2)); @@ -362,7 +362,7 @@ fn test_middle_local_to_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/middle_local_to_import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.convert_local_fn_to_import( FunctionID(2), @@ -384,7 +384,7 @@ fn test_first_local_to_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/first_local_to_import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.convert_local_fn_to_import( FunctionID(1), @@ -406,7 +406,7 @@ fn test_last_local_to_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/last_local_to_import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.convert_local_fn_to_import( FunctionID(3), @@ -428,7 +428,7 @@ fn test_all_local_to_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/all_local_to_import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.convert_local_fn_to_import( FunctionID(3), @@ -462,7 +462,7 @@ fn test_some_local_to_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/some_local_to_import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); module.convert_local_fn_to_import( FunctionID(3), @@ -489,7 +489,7 @@ fn test_some_local_to_import() { fn test_all_local_to_import_all_import_to_local() { let file = "tests/test_inputs/instr_testing/modules/function_modification/all_local_to_import_all_import_to_local.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // Convert all to local let mut first_builder = FunctionBuilder::new(&[DataType::I32, DataType::I32], &[]); @@ -544,7 +544,7 @@ fn test_all_local_to_import_all_import_to_local() { fn test_add_fns_init_exprs() { let file = "tests/test_inputs/instr_testing/modules/init-exprs.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // add first import func let (..) = module.add_import_func("test0".to_string(), "func0".to_string(), TypeID(4)); @@ -576,7 +576,7 @@ fn test_add_fns_init_exprs() { fn test_add_imports_and_local_fns() { let file = "tests/test_inputs/instr_testing/modules/function_modification/add_imported_and_local_funcs.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // add first import func let (fid, ..) = module.add_import_func("test0".to_string(), "func0".to_string(), TypeID(2)); @@ -610,7 +610,7 @@ fn add_global_with_import() { let file = "tests/test_inputs/instr_testing/modules/function_modification/add_global.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse module"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse module"); // add new global let gid = module.add_global( @@ -637,7 +637,7 @@ fn parse_start_module() -> Module<'static> { let file = "tests/test_inputs/handwritten/modules/_start.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); let buff: &'static [u8] = Box::leak(buff.into_boxed_slice()); - Module::parse(buff, false, false).expect("Unable to parse module") + Module::parse(buff, false, false, false).expect("Unable to parse module") } fn encoded_wat(module: &Module) -> String { @@ -790,7 +790,7 @@ fn test_elem_reindexing() { ) )"#; let buff = wat::parse_str(wat).unwrap(); - let mut module = Module::parse(&buff, false, false).unwrap(); + let mut module = Module::parse(&buff, false, false, false).unwrap(); // Add an import of a different type. Then the table will have entries of // the wrong type unless the element section is reindexed. diff --git a/tests/iterator_test.rs b/tests/iterator_test.rs index a102f05b..5d7c53c2 100644 --- a/tests/iterator_test.rs +++ b/tests/iterator_test.rs @@ -15,7 +15,7 @@ use wirm::Component; fn test_iterator_count() { let file = "tests/test_inputs/handwritten/components/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut component = Component::parse(&buff, false, false).expect("Unable to parse"); + let mut component = Component::parse(&buff, false, false, false).expect("Unable to parse"); let mut comp_it = ComponentIterator::new(&mut component, HashMap::new()); iterate_component_and_count(&mut comp_it, 10); } @@ -24,7 +24,7 @@ fn test_iterator_count() { fn test_iterator_count_mul_mod() { let file = "tests/test_inputs/handwritten/components/mul_mod.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut component = Component::parse(&buff, false, false).expect("Unable to parse"); + let mut component = Component::parse(&buff, false, false, false).expect("Unable to parse"); let mut comp_it = ComponentIterator::new(&mut component, HashMap::new()); iterate_component_and_count(&mut comp_it, 15); } @@ -33,7 +33,7 @@ fn test_iterator_count_mul_mod() { fn test_mod_iterator_count() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); let mut mod_it = ModuleIterator::new(&mut module, &vec![]); iterate_module_and_count(&mut mod_it, 1, 9); } @@ -43,7 +43,7 @@ fn test_mod_iterator_count() { fn test_blocks() { let file = "tests/test_inputs/handwritten/modules/block.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); assert_eq!(module.num_import_func(), 0); let mut mod_it = ModuleIterator::new(&mut module, &vec![]); @@ -56,7 +56,7 @@ fn test_blocks() { fn test_it_instr_at() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); let mut mod_it = ModuleIterator::new(&mut module, &vec![]); let loc = Location::Module { @@ -92,7 +92,7 @@ fn test_it_instr_at() { fn test_it_dup_instr() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); let mut mod_it = ModuleIterator::new(&mut module, &vec![]); loop { @@ -127,7 +127,7 @@ fn test_it_dup_instr() { fn test_it_add_local_diff_type() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); let mut mod_it = ModuleIterator::new(&mut module, &vec![]); mod_it.add_local(wirm::ir::types::DataType::I64); @@ -141,7 +141,7 @@ fn test_it_add_local_diff_type() { fn test_imports() { let file = "tests/test_inputs/handwritten/modules/import.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); assert_eq!(module.num_import_func(), 2); let mut mod_it = ModuleIterator::new(&mut module, &vec![]); @@ -157,7 +157,7 @@ fn test_function_skipping_module() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).expect("Unable to parse"); + let mut module = Module::parse(&buff, false, false, false).expect("Unable to parse"); let functions_skip = vec![FunctionID(1)]; let mut mod_it = ModuleIterator::new(&mut module, &functions_skip); @@ -187,7 +187,7 @@ fn test_function_skipping_component() { let file = "tests/test_inputs/handwritten/components/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let mut comp = Component::parse(&buff, false, false).expect("Unable to parse"); + let mut comp = Component::parse(&buff, false, false, false).expect("Unable to parse"); let functions_skip = vec![FunctionID(0)]; let mut mapping = HashMap::new(); mapping.insert(ModuleID(0), functions_skip); @@ -215,7 +215,7 @@ fn test_fn_name() { let file = "tests/test_inputs/handwritten/modules/add.wat"; let buff = wat::parse_file(file).expect("couldn't convert the input wat to Wasm"); - let module = Module::parse(&buff, false, false).expect("Unable to parse"); + let module = Module::parse(&buff, false, false, false).expect("Unable to parse"); assert_eq!( "add".to_string(), *module.functions.get_name(FunctionID(1)).as_ref().unwrap() diff --git a/tests/round_trip_component.rs b/tests/round_trip_component.rs index 4b6bd3fd..36822bb6 100644 --- a/tests/round_trip_component.rs +++ b/tests/round_trip_component.rs @@ -10,7 +10,7 @@ fn round_trip_component(testname: &str, folder: &str) { testname ); let buff = wat::parse_file(filename).expect("couldn't convert the input wat to Wasm"); - let component = Component::parse(&buff, false, false).expect("Unable to parse"); + let component = Component::parse(&buff, false, false, false).expect("Unable to parse"); let result = component.encode().expect("error"); write_to_file( &result, diff --git a/tests/round_trip_module.rs b/tests/round_trip_module.rs index 94e95c45..029e0695 100644 --- a/tests/round_trip_module.rs +++ b/tests/round_trip_module.rs @@ -15,7 +15,7 @@ fn round_trip_module(testname: &str, folder: &str) { let buff = wat::parse_file(filename).expect("couldn't convert the input wat to Wasm"); let original = wasmprinter::print_bytes(buff.clone()).expect("couldn't convert original Wasm to wat"); - let module = Module::parse(&buff, false, false).unwrap(); + let module = Module::parse(&buff, false, false, false).unwrap(); let result = module.encode().expect("error!"); let out = wasmprinter::print_bytes(result).expect("couldn't translated Wasm to wat"); @@ -65,7 +65,7 @@ mod round_trip { fn set_name() { let filename = "tests/test_inputs/handwritten/modules/func1.wat"; let buff = wat::parse_file(filename).expect("couldn't convert the input wat to Wasm"); - let mut module = Module::parse(&buff, false, false).unwrap(); + let mut module = Module::parse(&buff, false, false, false).unwrap(); module.set_fn_name(FunctionID(1), "test".to_string()); // println!("{:#?}", module); let result = module.encode().expect("error!"); diff --git a/tests/round_trip_wast.rs b/tests/round_trip_wast.rs index 5c9d102a..55fb9560 100644 --- a/tests/round_trip_wast.rs +++ b/tests/round_trip_wast.rs @@ -11,12 +11,12 @@ fn roundtrip(label: &str, bytes: &[u8], component: bool) { let original = wasmprinter::print_bytes(bytes).expect("couldn't convert original Wasm to wat"); println!("original: {:?}", original); if component { - let parser = Component::parse(bytes, false, false).expect("Unable to parse"); + let parser = Component::parse(bytes, false, false, false).expect("Unable to parse"); let result = parser.encode().expect("error"); let out = wasmprinter::print_bytes(result.clone()).expect("couldn't translate Wasm to wat"); assert_eq!(out, original); } else { - let parser = Module::parse(bytes, false, false).expect("Unable to parse"); + let parser = Module::parse(bytes, false, false, false).expect("Unable to parse"); let result = parser.encode().expect("error during parse"); let out = wasmprinter::print_bytes(result.clone()).expect("couldn't translate Wasm to wat"); assert_eq!(out, original); diff --git a/tests/test_inputs/handwritten/dwarf/add.wasm b/tests/test_inputs/handwritten/dwarf/add.wasm new file mode 100644 index 0000000000000000000000000000000000000000..4cd498f82bcb14efed46724c62a9bb2c1dce492f GIT binary patch literal 409 zcmYLF%TB{E5S+E0M;kSv?SUdL5o&KJDFq2ZC6KrwZu~%TNCH7^iW0Xu_EY#mzJLpM zDBy!Vo}C@-EM&bR0OZgU+wEw_8NqRg%QOYL9dZyNq&E*EQOZ?DHo??bhrmFRD8WN7 zO0#71vXDtqWn1dg#2p6QKlc0fz&3dQ*hc=Yzo%c9Kgw3`Ki ziHS$_<@e0fhHVMoKrr%IKw|@8>FheRfMuP;aj1Vu0%~LVN-r@gH$!2x Date: Fri, 29 May 2026 15:29:10 -0400 Subject: [PATCH 03/13] Capture new per-op PCs during encode --- DWARF-plan.md | 44 ++++++++++++++--------- src/ir/module/mod.rs | 82 ++++++++++++++++++++++++++++++++++--------- src/ir/module/test.rs | 49 ++++++++++++++++++++++++++ 3 files changed, 143 insertions(+), 32 deletions(-) diff --git a/DWARF-plan.md b/DWARF-plan.md index 9dad54b6..eda3e889 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -14,13 +14,13 @@ Work through these in order. Each box is roughly one PR-sized unit. - [x] **0. Validate the testing approach.** Spike confirmed the design is viable; findings recorded under "Spike findings (step 0)" below. -- [ ] **1. Parse-aside.** Add `gimli` dep. Add opt-in flag (`with_dwarf` +- [x] **1. Parse-aside.** Add `gimli` dep. Add opt-in flag (`with_dwarf` on `Module::parse`). When set, recognize `.debug_*` custom sections and pull them into `ModuleDebugData` (currently a stub at `src/ir/dwarf.rs`). All other sections continue to flow through `custom_sections` unchanged. No rewriting yet — encode should round-trip the parsed-aside DWARF byte-identically. -- [ ] **2. Capture new per-op PCs during encode.** Modify +- [x] **2. Capture new per-op PCs during encode.** Modify `encode_function` (`src/ir/module/mod.rs:1202`) to record the `wasm_encoder::Function::byte_len()` after each emitted op, producing a `Vec` of new in-function PCs aligned with the @@ -108,22 +108,34 @@ These are settled — recorded here so we don't relitigate. ## Current-state notes -- `src/ir/dwarf.rs` is a one-line stub (`use gimli::read::Dwarf`) and - `gimli` is not yet a dependency. `ir/mod.rs` doesn't pull it in. -- DWARF sections currently pass through opaquely. - `parse_internal` (`src/ir/module/mod.rs:430`) sticks unrecognized - custom sections into `custom_sections`; `encode_internal` - (`src/ir/module/mod.rs:1866`) re-emits them byte-for-byte. After - instrumentation, the output's `.debug_line` points at the wrong - addresses. -- The "original instruction → original PC" half is already done. - `Module::parse(_, _, with_offsets)` in `src/ir/module/mod.rs:148` - threads through to `Instructions::new` - (`src/ir/types.rs:1578`), which records `offset - locals_start` per - op. Exposed via `Instructions::lookup_pc_offset_for`. +- `src/ir/dwarf.rs` defines `ModuleDebugData`, which holds the + `.debug_*` custom sections lifted aside at parse time. `gimli` 0.33 + is a dependency; `ir/mod.rs` declares the module. +- DWARF sections still pass through opaquely by default. When + `Module::parse` is called with `with_dwarf = true`, the custom-section + dispatch in `parse_internal` + (`src/ir/module/mod.rs:447`) diverts `.debug_*` into `Module::debug` + instead of `custom_sections`, and `encode_internal` + (`src/ir/module/mod.rs:1365`) re-emits them byte-for-byte at the tail + of the custom-section run. No address rewriting yet — after + instrumentation the output's `.debug_line` still points at the wrong + addresses regardless of the opt-in. +- The "original instruction → original PC" half is done. + `Module::parse(_, _, with_offsets, _)` threads through to + `Instructions::new` (`src/ir/types.rs:1578`), which records + `offset - locals_start` per op. Empirically `locals_start` lands on + the first instruction's offset (the entire locals declaration is + excluded), so op 0 sits at PC 0. Exposed via + `Instructions::lookup_pc_offset_for`. +- The "emit-order → new PC" half is also done. `encode_internal` + captures per-local-function `Vec` of start offsets (gated on + `Module::debug.is_some()`) and returns it as the third tuple + element. Capture state lives in a `PcCapture` threaded through + `encode_function`'s helpers; PCs are rebased onto the first + instruction so they share the parse-side convention directly. - Special-mode resolution lives in `Module::resolve_special_instrumentation` - (`src/ir/module/mod.rs:730`). It expands + (`src/ir/module/mod.rs:758`). It expands `SemanticAfter`/`BlockEntry`/`BlockExit`/`BlockAlt` plus func-entry and func-exit into concrete `Before`/`After`/`Alternate` injections pinned to specific target instructions. The anchor walk runs after diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index c3799bce..48b5f474 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -731,7 +731,7 @@ impl<'a> Module<'a> { /// Emit the module into a wasm binary file. pub fn emit_wasm(&mut self, file_name: &str) -> types::Result<()> { - let (module, _) = self.encode_internal(false)?; + let (module, _, _) = self.encode_internal(false)?; let wasm = module.finish(); std::fs::write(file_name, wasm).map_err(IO)?; Ok(()) @@ -1232,7 +1232,8 @@ impl<'a> Module<'a> { func_mapping: &HashMap, global_mapping: &HashMap, memory_mapping: &HashMap, - ) -> types::Result { + capture_pcs: bool, + ) -> types::Result<(wasm_encoder::Function, Option>)> { let mut reencode = RoundtripReencoder; let Body { instructions, @@ -1244,17 +1245,26 @@ impl<'a> Module<'a> { converted_locals.push((*c, wasm_encoder::ValType::from(&*ty))); } let mut function = wasm_encoder::Function::new(converted_locals); + // Capture each emitted op's start offset when DWARF rewriting is on. + // `locals_base` rebases raw `byte_len()` (which counts from the locals + // declaration) onto the first instruction, matching the parse-side + // convention used by `Instructions::new`. + let locals_base = function.byte_len(); + let mut capture = capture_pcs.then(|| PcCapture { + offsets: Vec::new(), + locals_base, + }); let instr_len = instructions.len() - 1; let (ops, mut flags) = instructions.get_ops_flags_mut(); for (idx, op) in ops.iter_mut().enumerate() { fix_op_id_mapping(op, func_mapping, global_mapping, memory_mapping)?; if flags.is_none() { - encode(&op.clone(), &mut function, &mut reencode); + encode(&op.clone(), &mut function, &mut reencode, &mut capture); continue; } let instrument = &mut flags.as_mut().unwrap()[idx]; if !instrument.has_instr() { - encode(&op.clone(), &mut function, &mut reencode); + encode(&op.clone(), &mut function, &mut reencode, &mut capture); continue; } else { instrument.check_special_is_resolved(); @@ -1279,6 +1289,7 @@ impl<'a> Module<'a> { memory_mapping, &mut function, &mut reencode, + &mut capture, )?; // If there are any alternate, encode the alternate @@ -1291,10 +1302,11 @@ impl<'a> Module<'a> { memory_mapping, &mut function, &mut reencode, + &mut capture, )?; } } else { - encode(&op.clone(), &mut function, &mut reencode); + encode(&op.clone(), &mut function, &mut reencode, &mut capture); } // Now encode the after instructions @@ -1306,6 +1318,7 @@ impl<'a> Module<'a> { memory_mapping, &mut function, &mut reencode, + &mut capture, )?; } } @@ -1317,10 +1330,11 @@ impl<'a> Module<'a> { memory_mapping: &HashMap, function: &mut wasm_encoder::Function, reencode: &mut RoundtripReencoder, + capture: &mut Option, ) -> types::Result<()> { for instr in instrs { fix_op_id_mapping(instr, func_mapping, global_mapping, memory_mapping)?; - encode(instr, function, reencode); + encode(instr, function, reencode, capture); } Ok(()) } @@ -1328,7 +1342,13 @@ impl<'a> Module<'a> { instr: &Operator, function: &mut wasm_encoder::Function, reencode: &mut RoundtripReencoder, + capture: &mut Option, ) { + if let Some(capture) = capture.as_mut() { + capture + .offsets + .push(function.byte_len() - capture.locals_base); + } function.instruction( &reencode .instruction(instr.clone()) @@ -1337,7 +1357,7 @@ impl<'a> Module<'a> { } } - Ok(function) + Ok((function, capture.map(|c| c.offsets))) } /// Encodes an Wirm Module to a wasm_encoder Module. @@ -1345,15 +1365,18 @@ impl<'a> Module<'a> { pub(crate) fn encode_internal( &self, pull_side_effects: bool, - ) -> types::Result<( - wasm_encoder::Module, - HashMap>>, - )> { + ) -> types::Result<(wasm_encoder::Module, SideEffects<'a>, Option)> { #[cfg(feature = "parallel")] use rayon::prelude::*; let mut tmp = self.clone(); + // DWARF rewriting opt-in carries over from parse: `Module::debug` is + // `Some` iff `with_dwarf` was set. When on, capture each emitted op's + // new in-function PC so the DWARF rewriter can remap addresses. + let capture_pcs = tmp.debug.is_some(); + let mut new_pcs_by_func = NewPcMap::new(); + // First fix the ID mappings throughout the module let func_mapping = if tmp.functions.recalculate_ids { Self::recalculate_ids(&mut tmp.functions) @@ -1773,6 +1796,7 @@ impl<'a> Module<'a> { &func_mapping, &global_mapping, &memory_mapping, + capture_pcs, ); Some((idx, f, encoded_func)) }) @@ -1791,13 +1815,18 @@ impl<'a> Module<'a> { let f = f.unwrap_local_mut().expect( "Internal error: Should have found the local function successfully!", ); - let encoded_func = - Self::encode_function(f, &func_mapping, &global_mapping, &memory_mapping); + let encoded_func = Self::encode_function( + f, + &func_mapping, + &global_mapping, + &memory_mapping, + capture_pcs, + ); Some((idx, f, encoded_func)) }) .collect::>(); - for (idx, original_func, function) in functions { + for (idx, original_func, encoded) in functions { // at this point the IDs in all the function instrumentation opcodes have been corrected // add the probe side effects! if pull_side_effects { @@ -1806,7 +1835,11 @@ impl<'a> Module<'a> { if let Some(name) = &original_func.body.name { function_names.append(idx as u32, name.as_str()); } - code.function(&function?); + let (function, func_pcs) = encoded?; + if let Some(func_pcs) = func_pcs { + new_pcs_by_func.insert(idx as u32, func_pcs); + } + code.function(&function); } module.section(&code); } @@ -1915,7 +1948,7 @@ impl<'a> Module<'a> { } } - Ok((module, side_effects)) + Ok((module, side_effects, capture_pcs.then_some(new_pcs_by_func))) } // ============================== @@ -2368,6 +2401,23 @@ pub trait LocalOrImport { type BlockID = u32; type InstrBody<'a> = Vec>; + +/// Side effects pulled out of a module during encode, grouped by injection point. +type SideEffects<'a> = HashMap>>; +/// Per-local-function map: function index -> start offset of each emitted op in +/// emit order. Captured during encode when DWARF rewriting is on. +type NewPcMap = HashMap>; + +/// Per-op PC capture state threaded through `encode_function`. Present only when +/// DWARF rewriting is opted in. +struct PcCapture { + /// Start offset of each emitted op within the function body, in emit order, + /// using the parse-side convention (measured from the first instruction). + offsets: Vec, + /// Locals-declaration byte length, subtracted from raw `byte_len()` to + /// rebase offsets onto the first instruction. + locals_base: usize, +} struct InstrBodyFlagged<'a> { body: InstrBody<'a>, bool_flag: LocalID, diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 18d20286..03dab345 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -985,3 +985,52 @@ fn test_custom_section_constructors() { assert_eq!(section2.name, "test2"); assert_eq!(section2.data.as_ref(), &owned_data2); } + +// DWARF rewriting, step 2: encode captures each emitted op's new in-function +// PC when `with_dwarf` is on. For an uninstrumented module the emit order +// matches the original op order, so the captured start-PCs must equal the +// per-op offsets recovered by re-parsing the encoded output with offsets. +#[test] +fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { + let wat = r#"(module + (func (result i32) i32.const 1 i32.const 2 i32.add) + (func nop nop))"#; + let wasm = wat::parse_str(wat).expect("wat compiles"); + let module = Module::parse(&wasm, false, false, true).expect("parse"); + + let (encoded, _side_effects, new_pcs) = module.encode_internal(false).expect("encode"); + let new_pcs = new_pcs.expect("with_dwarf=true should capture per-op PCs"); + assert!(!new_pcs.is_empty(), "expected per-function PC maps"); + let out = encoded.finish(); + + let reparsed = Module::parse(&out, false, true, false).expect("reparse output"); + for (func_idx, captured) in &new_pcs { + let local = reparsed + .functions + .unwrap_local(FunctionID(*func_idx)) + .expect("local function"); + for (i, pc) in captured.iter().enumerate() { + assert_eq!( + Some(*pc), + local.lookup_pc_offset_for(i), + "func {func_idx} op {i}: captured PC disagrees with re-parsed offset", + ); + } + // No op past the captured range — captured length matches the body. + assert_eq!( + local.lookup_pc_offset_for(captured.len()), + None, + "func {func_idx}: re-parsed output has more ops than captured", + ); + } +} + +// `with_dwarf = false` must leave the capture off: no per-op PC map is +// produced, so encode pays nothing for modules that didn't opt in. +#[test] +fn without_dwarf_captures_no_per_op_pcs() { + let wasm = wat::parse_str("(module (func nop))").expect("wat compiles"); + let module = Module::parse(&wasm, false, false, false).expect("parse"); + let (_encoded, _side_effects, new_pcs) = module.encode_internal(false).expect("encode"); + assert!(new_pcs.is_none(), "with_dwarf=false should not capture PCs"); +} From 922b3de5b0235afbf9805c9264ebbea943671159 Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Fri, 29 May 2026 15:51:44 -0400 Subject: [PATCH 04/13] Track new code-section start --- DWARF-plan.md | 2 +- src/ir/module/mod.rs | 42 +++++++++++++++++++++++++++++------ src/ir/module/test.rs | 51 ++++++++++++++++++++++++++++++++++++------- 3 files changed, 79 insertions(+), 16 deletions(-) diff --git a/DWARF-plan.md b/DWARF-plan.md index eda3e889..f20d8785 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -25,7 +25,7 @@ Work through these in order. Each box is roughly one PR-sized unit. `wasm_encoder::Function::byte_len()` after each emitted op, producing a `Vec` of new in-function PCs aligned with the emitted op order. Gated on the same opt-in flag. -- [ ] **3. Track new code-section start.** During `encode_internal`, +- [x] **3. Track new code-section start.** During `encode_internal`, record the byte offset where the code section begins in the output (header bytes + sizes of preceding sections). DWARF PCs are relative to the code section start. diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 48b5f474..4651e490 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -1250,7 +1250,7 @@ impl<'a> Module<'a> { // declaration) onto the first instruction, matching the parse-side // convention used by `Instructions::new`. let locals_base = function.byte_len(); - let mut capture = capture_pcs.then(|| PcCapture { + let mut capture = capture_pcs.then_some(PcCapture { offsets: Vec::new(), locals_base, }); @@ -1365,7 +1365,11 @@ impl<'a> Module<'a> { pub(crate) fn encode_internal( &self, pull_side_effects: bool, - ) -> types::Result<(wasm_encoder::Module, SideEffects<'a>, Option)> { + ) -> types::Result<( + wasm_encoder::Module, + SideEffects<'a>, + Option, + )> { #[cfg(feature = "parallel")] use rayon::prelude::*; @@ -1375,7 +1379,11 @@ impl<'a> Module<'a> { // `Some` iff `with_dwarf` was set. When on, capture each emitted op's // new in-function PC so the DWARF rewriter can remap addresses. let capture_pcs = tmp.debug.is_some(); - let mut new_pcs_by_func = NewPcMap::new(); + let mut new_pcs_by_func = HashMap::new(); + // Snapshot of `module.len()` at the moment the code section ID byte + // gets pushed. The DWARF rewriter uses this to translate per-function + // PCs into module-absolute addresses. + let mut code_section_start: Option = None; // First fix the ID mappings throughout the module let func_mapping = if tmp.functions.recalculate_ids { @@ -1841,6 +1849,9 @@ impl<'a> Module<'a> { } code.function(&function); } + if capture_pcs { + code_section_start = Some(module.len()); + } module.section(&code); } @@ -1948,7 +1959,14 @@ impl<'a> Module<'a> { } } - Ok((module, side_effects, capture_pcs.then_some(new_pcs_by_func))) + Ok(( + module, + side_effects, + capture_pcs.then_some(DwarfEncodeMaps { + code_section_start, + per_func_pcs: new_pcs_by_func, + }), + )) } // ============================== @@ -2404,9 +2422,6 @@ type InstrBody<'a> = Vec>; /// Side effects pulled out of a module during encode, grouped by injection point. type SideEffects<'a> = HashMap>>; -/// Per-local-function map: function index -> start offset of each emitted op in -/// emit order. Captured during encode when DWARF rewriting is on. -type NewPcMap = HashMap>; /// Per-op PC capture state threaded through `encode_function`. Present only when /// DWARF rewriting is opted in. @@ -2418,6 +2433,19 @@ struct PcCapture { /// rebase offsets onto the first instruction. locals_base: usize, } + +/// DWARF rewriting state captured during encode. Present only when the input +/// module opted in via `with_dwarf` at parse time. +#[allow(dead_code)] // fields are consumed by the in-progress DWARF rewriter +pub(crate) struct DwarfEncodeMaps { + /// Module-absolute byte offset where the code section begins in the output + /// (the section ID byte). `None` when no code section was emitted. + pub(crate) code_section_start: Option, + /// Per-local-function map: function index -> start offset of each emitted + /// op in emit order. PCs share the parse-side convention (measured from the + /// first instruction of the function body). + pub(crate) per_func_pcs: HashMap>, +} struct InstrBodyFlagged<'a> { body: InstrBody<'a>, bool_flag: LocalID, diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 03dab345..49a61ab2 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -998,13 +998,16 @@ fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { let wasm = wat::parse_str(wat).expect("wat compiles"); let module = Module::parse(&wasm, false, false, true).expect("parse"); - let (encoded, _side_effects, new_pcs) = module.encode_internal(false).expect("encode"); - let new_pcs = new_pcs.expect("with_dwarf=true should capture per-op PCs"); - assert!(!new_pcs.is_empty(), "expected per-function PC maps"); + let (encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); + let maps = maps.expect("with_dwarf=true should capture DWARF encode maps"); + assert!( + !maps.per_func_pcs.is_empty(), + "expected per-function PC maps" + ); let out = encoded.finish(); let reparsed = Module::parse(&out, false, true, false).expect("reparse output"); - for (func_idx, captured) in &new_pcs { + for (func_idx, captured) in &maps.per_func_pcs { let local = reparsed .functions .unwrap_local(FunctionID(*func_idx)) @@ -1025,12 +1028,44 @@ fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { } } -// `with_dwarf = false` must leave the capture off: no per-op PC map is -// produced, so encode pays nothing for modules that didn't opt in. +// DWARF rewriting, step 3: encode records the byte offset where the code +// section begins in the output, so the rewriter can translate per-function +// PCs into module-absolute addresses. The captured offset must point at the +// code-section ID byte (0x0A) and lie past the wasm preamble. +#[test] +fn with_dwarf_captures_code_section_start_offset() { + let wasm = wat::parse_str("(module (func nop))").expect("wat compiles"); + let module = Module::parse(&wasm, false, false, true).expect("parse"); + + let (encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); + let maps = maps.expect("with_dwarf=true should capture DWARF encode maps"); + let css = maps + .code_section_start + .expect("code section is emitted for a module with a local function"); + + let out = encoded.finish(); + // Wasm magic+version is 8 bytes; the code section sits after at least the + // type and function sections that wirm always emits, so this must be past + // the preamble. The captured offset must point at the code-section ID. + assert!( + css > 8, + "code_section_start should be past the 8-byte preamble" + ); + assert_eq!( + out[css], 0x0a, + "code_section_start must point at the code-section ID byte (0x0A)", + ); +} + +// `with_dwarf = false` must leave capture off entirely: no DWARF encode maps +// are produced, so encode pays nothing for modules that didn't opt in. #[test] fn without_dwarf_captures_no_per_op_pcs() { let wasm = wat::parse_str("(module (func nop))").expect("wat compiles"); let module = Module::parse(&wasm, false, false, false).expect("parse"); - let (_encoded, _side_effects, new_pcs) = module.encode_internal(false).expect("encode"); - assert!(new_pcs.is_none(), "with_dwarf=false should not capture PCs"); + let (_encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); + assert!( + maps.is_none(), + "with_dwarf=false should not capture DWARF maps" + ); } From 06faee95ba16c90de858623be055443a32e0be3d Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Fri, 29 May 2026 15:54:55 -0400 Subject: [PATCH 05/13] fix broken condition checking if the module contains local funcs --- src/ir/module/mod.rs | 4 ++-- src/ir/module/test.rs | 15 +++++++++++++++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 4651e490..943818e5 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -763,7 +763,7 @@ impl<'a> Module<'a> { pull_side_effects: bool, side_effects: &mut HashMap>>, ) -> types::Result<()> { - if !self.num_local_functions > 0 { + if self.num_local_functions > 0 { for rel_func_idx in (self.imports.num_funcs - self.imports.num_funcs_added) as usize ..self.functions.as_vec().len() { @@ -1783,7 +1783,7 @@ impl<'a> Module<'a> { module.section(&data_count); } - if !tmp.num_local_functions > 0 { + if tmp.num_local_functions > 0 { let mut code = wasm_encoder::CodeSection::new(); #[cfg(feature = "parallel")] diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 49a61ab2..11b90836 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -1057,6 +1057,21 @@ fn with_dwarf_captures_code_section_start_offset() { ); } +// A module with no local functions emits no code section, so the rewriter +// has no code-section anchor to record even when DWARF rewriting is on. +#[test] +fn with_dwarf_no_code_section_leaves_code_section_start_none() { + let wasm = wat::parse_str("(module)").expect("wat compiles"); + let module = Module::parse(&wasm, false, false, true).expect("parse"); + let (_encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); + let maps = maps.expect("with_dwarf=true should capture DWARF encode maps"); + assert!( + maps.code_section_start.is_none(), + "no code section emitted => no captured start offset", + ); + assert!(maps.per_func_pcs.is_empty()); +} + // `with_dwarf = false` must leave capture off entirely: no DWARF encode maps // are produced, so encode pays nothing for modules that didn't opt in. #[test] From 2ce55de506a022b94e218174111b9217b22809ea Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Mon, 1 Jun 2026 09:04:54 -0400 Subject: [PATCH 06/13] calculate the debug anchors of origi+injected ops --- DWARF-plan.md | 2 +- src/ir/module/mod.rs | 71 ++++++++++++++++++++++++++++--------------- src/ir/module/test.rs | 28 +++++++++-------- 3 files changed, 63 insertions(+), 38 deletions(-) diff --git a/DWARF-plan.md b/DWARF-plan.md index f20d8785..4f72da66 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -29,7 +29,7 @@ Work through these in order. Each box is roughly one PR-sized unit. record the byte offset where the code section begins in the output (header bytes + sizes of preceding sections). DWARF PCs are relative to the code section start. -- [ ] **4. Post-resolve anchor-walk.** After +- [x] **4. Post-resolve anchor-walk.** After `resolve_special_instrumentation`, walk every function's `InstrumentationFlag.{before, after, alternate}` and produce `Vec<(emit_order_idx, anchor_instr_idx)>` per function. Original diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 943818e5..7bc8d332 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -1233,7 +1233,7 @@ impl<'a> Module<'a> { global_mapping: &HashMap, memory_mapping: &HashMap, capture_pcs: bool, - ) -> types::Result<(wasm_encoder::Function, Option>)> { + ) -> types::Result<(wasm_encoder::Function, Option)> { let mut reencode = RoundtripReencoder; let Body { instructions, @@ -1245,18 +1245,27 @@ impl<'a> Module<'a> { converted_locals.push((*c, wasm_encoder::ValType::from(&*ty))); } let mut function = wasm_encoder::Function::new(converted_locals); - // Capture each emitted op's start offset when DWARF rewriting is on. - // `locals_base` rebases raw `byte_len()` (which counts from the locals - // declaration) onto the first instruction, matching the parse-side - // convention used by `Instructions::new`. + // Capture each emitted op's start offset and anchor when DWARF + // rewriting is on. `locals_base` rebases raw `byte_len()` (which counts + // from the locals declaration) onto the first instruction, matching + // the parse-side convention used by `Instructions::new`. let locals_base = function.byte_len(); let mut capture = capture_pcs.then_some(PcCapture { - offsets: Vec::new(), + maps: FuncDwarfMaps { + pcs: Vec::new(), + anchors: Vec::new(), + }, locals_base, + current_anchor: 0, }); let instr_len = instructions.len() - 1; let (ops, mut flags) = instructions.get_ops_flags_mut(); for (idx, op) in ops.iter_mut().enumerate() { + // Every op emitted during this iteration anchors to `idx` (the original + // op and any before/after/alt injections attached to it). + if let Some(capture) = capture.as_mut() { + capture.current_anchor = idx; + } fix_op_id_mapping(op, func_mapping, global_mapping, memory_mapping)?; if flags.is_none() { encode(&op.clone(), &mut function, &mut reencode, &mut capture); @@ -1346,8 +1355,10 @@ impl<'a> Module<'a> { ) { if let Some(capture) = capture.as_mut() { capture - .offsets + .maps + .pcs .push(function.byte_len() - capture.locals_base); + capture.maps.anchors.push(capture.current_anchor); } function.instruction( &reencode @@ -1357,7 +1368,7 @@ impl<'a> Module<'a> { } } - Ok((function, capture.map(|c| c.offsets))) + Ok((function, capture.map(|c| c.maps))) } /// Encodes an Wirm Module to a wasm_encoder Module. @@ -1375,11 +1386,8 @@ impl<'a> Module<'a> { let mut tmp = self.clone(); - // DWARF rewriting opt-in carries over from parse: `Module::debug` is - // `Some` iff `with_dwarf` was set. When on, capture each emitted op's - // new in-function PC so the DWARF rewriter can remap addresses. let capture_pcs = tmp.debug.is_some(); - let mut new_pcs_by_func = HashMap::new(); + let mut per_func_maps: HashMap = HashMap::new(); // Snapshot of `module.len()` at the moment the code section ID byte // gets pushed. The DWARF rewriter uses this to translate per-function // PCs into module-absolute addresses. @@ -1843,9 +1851,9 @@ impl<'a> Module<'a> { if let Some(name) = &original_func.body.name { function_names.append(idx as u32, name.as_str()); } - let (function, func_pcs) = encoded?; - if let Some(func_pcs) = func_pcs { - new_pcs_by_func.insert(idx as u32, func_pcs); + let (function, func_maps) = encoded?; + if let Some(func_maps) = func_maps { + per_func_maps.insert(idx as u32, func_maps); } code.function(&function); } @@ -1964,7 +1972,7 @@ impl<'a> Module<'a> { side_effects, capture_pcs.then_some(DwarfEncodeMaps { code_section_start, - per_func_pcs: new_pcs_by_func, + per_func: per_func_maps, }), )) } @@ -2423,15 +2431,30 @@ type InstrBody<'a> = Vec>; /// Side effects pulled out of a module during encode, grouped by injection point. type SideEffects<'a> = HashMap>>; -/// Per-op PC capture state threaded through `encode_function`. Present only when -/// DWARF rewriting is opted in. +/// Encode-time scratch threaded through `encode_function`. struct PcCapture { - /// Start offset of each emitted op within the function body, in emit order, - /// using the parse-side convention (measured from the first instruction). - offsets: Vec, + /// PC and anchor arrays under construction, returned verbatim at the end. + maps: FuncDwarfMaps, /// Locals-declaration byte length, subtracted from raw `byte_len()` to /// rebase offsets onto the first instruction. locals_base: usize, + /// Anchor for the next emission, set once per outer-loop iteration. + current_anchor: usize, +} + +/// Per-function PC and anchor maps captured during encode. +/// +/// `pcs` and `anchors` are parallel arrays indexed by emit-order position: +/// for each emitted op `i`, `pcs[i]` is its byte offset within the function +/// body and `anchors[i]` is the original-instruction index it should inherit +/// debug info from. +pub(crate) struct FuncDwarfMaps { + /// Start offset of each emitted op within the function body. + pub(crate) pcs: Vec, + /// Original-instruction index each emitted op anchors to. Original ops + /// anchor to themselves; injected ops inherit their host op's idx so a + /// debugger never "stops inside" wirm-injected code. + pub(crate) anchors: Vec, } /// DWARF rewriting state captured during encode. Present only when the input @@ -2441,10 +2464,8 @@ pub(crate) struct DwarfEncodeMaps { /// Module-absolute byte offset where the code section begins in the output /// (the section ID byte). `None` when no code section was emitted. pub(crate) code_section_start: Option, - /// Per-local-function map: function index -> start offset of each emitted - /// op in emit order. PCs share the parse-side convention (measured from the - /// first instruction of the function body). - pub(crate) per_func_pcs: HashMap>, + /// Per-local-function map: function index -> emit-order PC and anchor maps. + pub(crate) per_func: HashMap, } struct InstrBodyFlagged<'a> { body: InstrBody<'a>, diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 11b90836..06abd095 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -986,10 +986,6 @@ fn test_custom_section_constructors() { assert_eq!(section2.data.as_ref(), &owned_data2); } -// DWARF rewriting, step 2: encode captures each emitted op's new in-function -// PC when `with_dwarf` is on. For an uninstrumented module the emit order -// matches the original op order, so the captured start-PCs must equal the -// per-op offsets recovered by re-parsing the encoded output with offsets. #[test] fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { let wat = r#"(module @@ -1000,28 +996,36 @@ fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { let (encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); let maps = maps.expect("with_dwarf=true should capture DWARF encode maps"); - assert!( - !maps.per_func_pcs.is_empty(), - "expected per-function PC maps" - ); + assert!(!maps.per_func.is_empty(), "expected per-function maps"); let out = encoded.finish(); let reparsed = Module::parse(&out, false, true, false).expect("reparse output"); - for (func_idx, captured) in &maps.per_func_pcs { + for (func_idx, captured) in &maps.per_func { let local = reparsed .functions .unwrap_local(FunctionID(*func_idx)) .expect("local function"); - for (i, pc) in captured.iter().enumerate() { + assert_eq!( + captured.pcs.len(), + captured.anchors.len(), + "func {func_idx}: pcs and anchors must be parallel arrays", + ); + for (i, pc) in captured.pcs.iter().enumerate() { assert_eq!( Some(*pc), local.lookup_pc_offset_for(i), "func {func_idx} op {i}: captured PC disagrees with re-parsed offset", ); + // Uninstrumented: every emitted op is its own original, so anchor + // is the identity. + assert_eq!( + captured.anchors[i], i, + "func {func_idx} op {i}: uninstrumented anchor must be identity", + ); } // No op past the captured range — captured length matches the body. assert_eq!( - local.lookup_pc_offset_for(captured.len()), + local.lookup_pc_offset_for(captured.pcs.len()), None, "func {func_idx}: re-parsed output has more ops than captured", ); @@ -1069,7 +1073,7 @@ fn with_dwarf_no_code_section_leaves_code_section_start_none() { maps.code_section_start.is_none(), "no code section emitted => no captured start offset", ); - assert!(maps.per_func_pcs.is_empty()); + assert!(maps.per_func.is_empty()); } // `with_dwarf = false` must leave capture off entirely: no DWARF encode maps From defa54003ca8be2b270bd85e45a93710a3ff1a25 Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Thu, 4 Jun 2026 16:42:54 -0400 Subject: [PATCH 07/13] actually do the rewriting of the dwarf section --- src/error.rs | 3 + src/ir/dwarf.rs | 236 ++++++++++++++++++++++++++++++++++++++++-- src/ir/module/mod.rs | 149 ++++++++++++++++++++++++-- src/ir/module/test.rs | 12 ++- src/ir/types.rs | 15 ++- tests/dwarf.rs | 91 ++++++++++++++-- 6 files changed, 470 insertions(+), 36 deletions(-) diff --git a/src/error.rs b/src/error.rs index 5731b7f8..c5576698 100644 --- a/src/error.rs +++ b/src/error.rs @@ -39,6 +39,8 @@ pub enum Error { InvalidMemoryReservedByte { func_range: Range, }, + /// Error raised while reading or rewriting DWARF sections. + DwarfError(String), } impl From for Error { @@ -122,6 +124,7 @@ impl std::fmt::Display for Error { Error::InvalidMemoryReservedByte { func_range } => { write!(f, "Found a `memory.*` instruction with an invalid reserved byte in function at {:?}", func_range) } + Error::DwarfError(msg) => write!(f, "DWARF rewriting error: {msg}"), } } } diff --git a/src/ir/dwarf.rs b/src/ir/dwarf.rs index 0e37be0c..6511609c 100644 --- a/src/ir/dwarf.rs +++ b/src/ir/dwarf.rs @@ -1,24 +1,40 @@ -//! DWARF debug-info lifted aside from a Wasm module's custom sections. -//! -//! When `Module::parse` is called with `with_dwarf = true`, `.debug_*` custom -//! sections are diverted here instead of flowing through `custom_sections` -//! opaquely. Keeping them separate lets DWARF be handled coherently with an -//! instrumented module rather than passed through byte-for-byte. +use std::collections::HashMap; +use crate::error::Error; use crate::ir::types::CustomSection; -/// Holds the `.debug_*` custom sections lifted out of a Wasm module's -/// `custom_sections` list at parse time. +/// Wasm DWARF uses 32-bit code addresses. +const WASM_DWARF_ADDRESS_SIZE: u8 = 4; +/// Byte length of the unit-length field at the start of a DWARF32 line program. +const DWARF32_LENGTH_FIELD_BYTES: usize = 4; +/// Byte length of the unit-length field at the start of a DWARF64 line program +/// (4-byte 0xffffffff magic + 8-byte length). +const DWARF64_LENGTH_FIELD_BYTES: usize = 12; + +/// Per-function input-side metadata the DWARF rewriter consults to translate +/// in-function PCs to/from DWARF row addresses. +#[derive(Clone, Debug)] +pub struct OrigFuncDebugData { + /// DWARF address of the function's first instruction (= bytes of size LEB + /// + locals declarations, counted from the body's size LEB byte). + pub first_instr_dwarf_offset: usize, +} + #[derive(Clone, Debug, Default)] pub struct ModuleDebugData<'a> { /// `.debug_*` sections in the order they appeared in the input module. /// Encode emits them in this order after the rest of `custom_sections`. pub(crate) sections: Vec>, + /// Per-local-function metadata, keyed by function index. + pub(crate) per_func: HashMap, } impl<'a> ModuleDebugData<'a> { - pub(crate) fn from_sections(sections: Vec>) -> Self { - Self { sections } + pub(crate) fn new( + sections: Vec>, + per_func: HashMap, + ) -> Self { + Self { sections, per_func } } /// Read-only view of the underlying `.debug_*` custom sections. @@ -33,3 +49,203 @@ impl<'a> ModuleDebugData<'a> { name.starts_with(".debug_") } } + +/// New per-function maps the rewriter consults to emit translated DWARF rows. +pub(crate) struct PerFuncEncodeMaps<'m> { + /// Start byte of each emitted op within the function body, emit order. + pub pcs: &'m [usize], + /// Original-instruction index each emitted op anchors to. + pub anchors: &'m [usize], + /// DWARF address of the function's first instruction in the new module. + pub first_instr_dwarf_offset: usize, + /// Total bytes of the new function body (size LEB + content). + pub body_total_size: usize, +} + +/// Rewrite `.debug_line` so row addresses match the new code layout. Each row +/// inherits the source location of the original op its emitted op anchors to. +/// Assumes one sequence per local function in code-section order +/// (`wasm-tools parse --generate-dwarf`'s convention). +pub(crate) fn rewrite_debug_line( + input_bytes: &[u8], + debug: &ModuleDebugData, + per_func_new: &HashMap>, + orig_per_instr_pcs: &HashMap>, +) -> Result, Error> { + use gimli::{ + write::{self, Address, ConvertLineRow, DebugLine, EndianVec}, + EndianSlice, LittleEndian, SectionId, + }; + + let endian = LittleEndian; + let dl_bytes = EndianSlice::new(input_bytes, endian); + let empty = EndianSlice::new(&[], endian); + + let section_lookup = |name: &str| -> Option> { + debug + .sections() + .iter() + .find(|s| s.name == name) + .map(|s| EndianSlice::new(s.data.as_ref(), endian)) + }; + let debug_str_bytes = section_lookup(".debug_str").unwrap_or(empty); + let debug_line_str_bytes = section_lookup(".debug_line_str").unwrap_or(empty); + let debug_str_offsets_bytes = section_lookup(".debug_str_offsets").unwrap_or(empty); + + let dwarf_read = gimli::read::Dwarf::load(|id| -> Result<_, gimli::Error> { + Ok(match id { + SectionId::DebugLine => dl_bytes, + SectionId::DebugStr => debug_str_bytes, + SectionId::DebugLineStr => debug_line_str_bytes, + SectionId::DebugStrOffsets => debug_str_offsets_bytes, + _ => empty, + }) + }) + .map_err(|e| Error::DwarfError(format!("loading .debug_line: {e}")))?; + + // Parse the line program at offset 0. Multi-program (multi-CU) inputs are + // refused below; the rewriter doesn't yet route per-CU. + let from_program = dwarf_read + .debug_line + .program( + gimli::DebugLineOffset(0), + WASM_DWARF_ADDRESS_SIZE, + None, + None, + ) + .map_err(|e| Error::DwarfError(format!("opening line program: {e}")))?; + + let header = from_program.header(); + let length_field_size = match header.encoding().format { + gimli::Format::Dwarf32 => DWARF32_LENGTH_FIELD_BYTES, + gimli::Format::Dwarf64 => DWARF64_LENGTH_FIELD_BYTES, + }; + let first_program_bytes = length_field_size + header.unit_length(); + if first_program_bytes < input_bytes.len() { + return Err(Error::DwarfError(format!( + "multi-program .debug_line not yet supported ({first_program_bytes}/{} bytes consumed by the first program)", + input_bytes.len() + ))); + } + + let mut dwarf_write = write::Dwarf::new(); + let mut convert = dwarf_write + .read_line_program(&dwarf_read, from_program, None, None) + .map_err(|e| Error::DwarfError(format!("converter init: {e}")))?; + + // Walk input rows, grouping into sequences. We capture each row's + // absolute address so we can rebuild the orig_pc → row lookup per + // function below. + struct CollectedRow { + address: u64, + row: write::LineRow, + } + let mut sequences: Vec> = Vec::new(); + let mut current_sequence: Vec = Vec::new(); + let mut current_base: u64 = 0; + loop { + let item = convert + .read_row() + .map_err(|e| Error::DwarfError(format!("reading row: {e}")))?; + let Some(item) = item else { + break; + }; + match item { + ConvertLineRow::SetAddress(addr) => { + current_base = addr; + } + ConvertLineRow::Row(row) => { + let address = current_base + row.address_offset; + current_sequence.push(CollectedRow { address, row }); + } + ConvertLineRow::EndSequence(_length) => { + sequences.push(std::mem::take(&mut current_sequence)); + current_base = 0; + } + } + } + + // sequence[i] is assigned to the i-th local function (sorted by index); + // refuse mismatches so we never silently assign wrong source locations. + if sequences.len() != per_func_new.len() { + return Err(Error::DwarfError(format!( + ".debug_line sequence/function count mismatch ({} vs {})", + sequences.len(), + per_func_new.len() + ))); + } + + // `convert` already translated read FileIds into the write program's FileId + // space, so we copy `row.file` verbatim and don't need the file mapping. + let (mut new_program, _file_mapping) = convert.program(); + + // Sorted so sequence index matches new function index deterministically. + let mut func_indices: Vec = per_func_new.keys().copied().collect(); + func_indices.sort_unstable(); + + for (seq_idx, func_idx) in func_indices.iter().enumerate() { + let Some(orig_dbg) = debug.per_func.get(func_idx) else { + // Should not happen if parse-side capture is complete. + continue; + }; + let Some(orig_pcs) = orig_per_instr_pcs.get(func_idx) else { + continue; + }; + let new_maps = per_func_new.get(func_idx).unwrap(); + + // Build orig_pc → row lookup for this function's input sequence. + let Some(input_seq) = sequences.get(seq_idx) else { + // Input had no sequence for this function: leave it without + // line-program coverage. + continue; + }; + let mut by_orig_pc: HashMap = HashMap::new(); + for cr in input_seq { + let addr = cr.address as usize; + if addr < orig_dbg.first_instr_dwarf_offset { + continue; + } + let orig_pc = addr - orig_dbg.first_instr_dwarf_offset; + by_orig_pc.insert(orig_pc, &cr.row); + } + + new_program.begin_sequence(Some(Address::Constant(0))); + for (emit_idx, &new_pc) in new_maps.pcs.iter().enumerate() { + let anchor = new_maps.anchors[emit_idx]; + let Some(&orig_pc) = orig_pcs.get(anchor) else { + continue; + }; + let Some(src_row) = by_orig_pc.get(&orig_pc) else { + continue; + }; + let new_addr = (new_maps.first_instr_dwarf_offset + new_pc) as u64; + let row = new_program.row(); + row.address_offset = new_addr; + row.file = src_row.file; + row.line = src_row.line; + row.column = src_row.column; + row.is_statement = src_row.is_statement; + row.basic_block = src_row.basic_block; + row.prologue_end = src_row.prologue_end; + row.epilogue_begin = src_row.epilogue_begin; + row.discriminator = src_row.discriminator; + new_program.generate_row(); + } + new_program.end_sequence(new_maps.body_total_size as u64); + } + + // Use `dwarf_write`'s string tables — the program's row strings were + // registered into them via `convert`, and gimli enforces matching BaseIds. + let mut writer = DebugLine::from(EndianVec::new(endian)); + let encoding = new_program.encoding(); + new_program + .write( + &mut writer, + encoding, + &mut dwarf_write.line_strings, + &mut dwarf_write.strings, + ) + .map_err(|e| Error::DwarfError(format!("writing line program: {e}")))?; + + Ok(writer.0.into_vec()) +} diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 7bc8d332..2b0fe53d 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -5,7 +5,7 @@ use super::types::{ }; use crate::error::Error; use crate::error::Error::{InstrumentationError, IO}; -use crate::ir::dwarf::ModuleDebugData; +use crate::ir::dwarf::{ModuleDebugData, OrigFuncDebugData}; use crate::ir::function::FunctionModifier; use crate::ir::id::{ CustomSectionID, DataSegmentID, FunctionID, GlobalID, ImportsID, LocalID, MemoryID, TypeID, @@ -132,6 +132,18 @@ impl<'a> Iterator for FlagsIter<'a> { } } +/// Number of bytes an unsigned LEB128 encoding of `value` occupies. Used by +/// the DWARF rewriter at both parse and encode time to account for function +/// body size-LEB lengths in per-function DWARF address composition. +fn unsigned_leb128_len(mut value: u64) -> usize { + let mut len = 1; + while value >= 0x80 { + value >>= 7; + len += 1; + } + len +} + impl<'a> Module<'a> { /// Parses a `Module` from a wasm binary. /// @@ -167,10 +179,6 @@ impl<'a> Module<'a> { enable_multi_memory: bool, with_offsets: bool, ) -> Result { - let locals_orig = body - .get_locals_reader()? - .get_binary_reader() - .original_position(); let locals = body .get_locals_reader()? .into_iter() @@ -184,13 +192,17 @@ impl<'a> Module<'a> { }) .collect(); + // PCs are measured from the first instruction (after the locals + // declaration), aligning the parse-side convention with the encode + // side so both can be composed in a single coordinate system. + let pc_origin = body.get_binary_reader_for_operators()?.original_position(); let op_reader = body.get_operators_reader()?; let ops = op_reader .into_iter_with_offsets() .collect::, _>>() .expect("ops"); - let instructions = Instructions::new(ops, locals_orig, with_offsets); + let instructions = Instructions::new(ops, pc_origin, with_offsets); if let Some(last) = instructions.get_ops().last() { if let Operator::End = last { } else { @@ -219,6 +231,24 @@ impl<'a> Module<'a> { }) } + /// Compute the DWARF rewriter's input-side metadata for a single function + /// body without consuming it. + fn orig_func_debug_from(body: &wasmparser::FunctionBody) -> Result { + // `body.range()` covers the body *content* (locals + instructions), + // excluding the leading size LEB consumed when the FunctionBody was + // read. DWARF wasm addresses are measured from that size LEB byte, + // so the first instruction's DWARF offset is `size_leb_len + (first + // instruction byte − content start)`. + let body_range = body.range(); + let body_content_size = body_range.end - body_range.start; + let size_leb_len = unsigned_leb128_len(body_content_size as u64); + let first_instr_byte = body.get_binary_reader_for_operators()?.original_position(); + let first_instr_offset_in_content = first_instr_byte - body_range.start; + Ok(OrigFuncDebugData { + first_instr_dwarf_offset: size_leb_len + first_instr_offset_in_content, + }) + } + pub(crate) fn parse_internal( wasm: &'a [u8], enable_multi_memory: bool, @@ -229,6 +259,11 @@ impl<'a> Module<'a> { #[cfg(feature = "parallel")] use rayon::prelude::*; + // The DWARF rewriter needs orig per-instruction PCs to translate + // input row addresses; force offset capture when the caller opted + // into DWARF rewriting even if they didn't ask for `with_offsets`. + let with_offsets = with_offsets || with_dwarf; + let mut imports: ModuleImports = ModuleImports::default(); let mut types: HashMap = HashMap::new(); let mut recgroups = vec![]; @@ -579,29 +614,34 @@ impl<'a> Module<'a> { } #[cfg(feature = "parallel")] - let code_sections = bodies_and_names + let bodies_with_dbg: Vec<(Body, OrigFuncDebugData)> = bodies_and_names .into_par_iter() .map(|(body, name)| { + let orig_dbg = Self::orig_func_debug_from(&body)?; let mut body = Self::parse_body(body, enable_multi_memory, with_offsets)?; if let Some(name) = name { body.name = Some(name); } - Ok::<_, Error>(body) + Ok::<_, Error>((body, orig_dbg)) }) .collect::, _>>()?; #[cfg(not(feature = "parallel"))] - let code_sections = bodies_and_names + let bodies_with_dbg: Vec<(Body, OrigFuncDebugData)> = bodies_and_names .into_iter() .map(|(body, name)| { + let orig_dbg = Self::orig_func_debug_from(&body)?; let mut body = Self::parse_body(body, enable_multi_memory, with_offsets)?; if let Some(name) = name { body.name = Some(name); } - Ok::<_, Error>(body) + Ok::<_, Error>((body, orig_dbg)) }) .collect::, _>>()?; + let (code_sections, per_func_dbg_vec): (Vec, Vec) = + bodies_with_dbg.into_iter().unzip(); + if code_section_count != code_sections.len() || code_section_count != functions.len() { return Err(Error::IncorrectCodeCounts { function_section_count: functions.len(), @@ -681,6 +721,15 @@ impl<'a> Module<'a> { let num_memories = memories.len() as u32; let num_tables = tables.len() as u32; let module_globals = ModuleGlobals::new(&imports, globals); + let num_imported_funcs = imports.num_funcs; + let debug = debug_sections.map(|s| { + let per_func: HashMap = per_func_dbg_vec + .into_iter() + .enumerate() + .map(|(i, dbg)| (num_imported_funcs + i as u32, dbg)) + .collect(); + ModuleDebugData::new(s, per_func) + }); Ok(Module { types: ModuleTypes::new(recgroups, types), imports, @@ -696,7 +745,7 @@ impl<'a> Module<'a> { data, tags, custom_sections: CustomSections::new(custom_sections), - debug: debug_sections.map(ModuleDebugData::from_sections), + debug, num_local_functions: code_sections_length as u32, num_local_globals: num_globals, num_local_tables: num_tables, @@ -1254,6 +1303,10 @@ impl<'a> Module<'a> { maps: FuncDwarfMaps { pcs: Vec::new(), anchors: Vec::new(), + // Filled in at the end of `encode_function` once the body's + // total content length is known. + first_instr_dwarf_offset: 0, + body_total_size: 0, }, locals_base, current_anchor: 0, @@ -1368,6 +1421,16 @@ impl<'a> Module<'a> { } } + if let Some(cap) = capture.as_mut() { + // The DWARF rewriter needs the function's first-instruction DWARF + // offset and total body size. Size-LEB length is recovered from + // the body content length (`function.byte_len()` covers locals + + // instructions; the LEB encodes that value when emitted later). + let body_content_len = function.byte_len(); + let size_leb_len = unsigned_leb128_len(body_content_len as u64); + cap.maps.first_instr_dwarf_offset = size_leb_len + cap.locals_base; + cap.maps.body_total_size = size_leb_len + body_content_len; + } Ok((function, capture.map(|c| c.maps))) } @@ -1954,6 +2017,63 @@ impl<'a> Module<'a> { }); } + // Rewrite the `.debug_line` section so its addresses match the + // new code layout. We then re-emit the DWARF sections below; the + // other `.debug_*` sections still pass through verbatim (step 6 + // covers `.debug_info`, location lists, etc.). + if capture_pcs { + if let Some(debug) = tmp.debug.as_ref() { + let line_idx = debug + .sections() + .iter() + .position(|s| s.name == ".debug_line"); + if let Some(line_idx) = line_idx { + // Build per-instr orig PCs for each local function with + // captured maps. `with_offsets` is auto-enabled at parse + // time when `with_dwarf` is on, so offsets are always + // present here — treat their absence as a bug. + let mut orig_per_instr_pcs: HashMap> = HashMap::new(); + for (func_idx, _) in per_func_maps.iter() { + let local = tmp + .functions + .unwrap_local(FunctionID(*func_idx)) + .expect("captured DWARF map should be for a local function"); + let offsets = local + .body + .instructions + .offsets() + .expect("with_dwarf opt-in must populate orig per-instr offsets"); + orig_per_instr_pcs.insert(*func_idx, offsets.to_vec()); + } + let per_func_new: HashMap = + per_func_maps + .iter() + .map(|(idx, m)| { + ( + *idx, + crate::ir::dwarf::PerFuncEncodeMaps { + pcs: &m.pcs, + anchors: &m.anchors, + first_instr_dwarf_offset: m.first_instr_dwarf_offset, + body_total_size: m.body_total_size, + }, + ) + }) + .collect(); + let input_bytes = debug.sections()[line_idx].data.as_ref(); + let new_bytes = crate::ir::dwarf::rewrite_debug_line( + input_bytes, + debug, + &per_func_new, + &orig_per_instr_pcs, + )?; + if let Some(debug_mut) = tmp.debug.as_mut() { + debug_mut.sections[line_idx].data = std::borrow::Cow::Owned(new_bytes); + } + } + } + } + // Re-emit DWARF custom sections held aside in `debug`. They follow // the rest of the custom sections; for inputs that already kept DWARF // at the tail (the wasm-tools convention) this preserves byte @@ -2455,6 +2575,13 @@ pub(crate) struct FuncDwarfMaps { /// anchor to themselves; injected ops inherit their host op's idx so a /// debugger never "stops inside" wirm-injected code. pub(crate) anchors: Vec, + /// DWARF address of the first instruction in the new function body + /// (= size-LEB bytes + locals-declaration bytes). Composes with `pcs[i]` + /// to give per-function DWARF row addresses: `addr = first_instr_dwarf_offset + pcs[i]`. + pub(crate) first_instr_dwarf_offset: usize, + /// Total bytes of the new function body including the size LEB. The line + /// program's end-of-sequence row marks address `body_total_size`. + pub(crate) body_total_size: usize, } /// DWARF rewriting state captured during encode. Present only when the input diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 06abd095..71193ca9 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -986,11 +986,21 @@ fn test_custom_section_constructors() { assert_eq!(section2.data.as_ref(), &owned_data2); } +// The third function declares two locals so this also acts as a regression +// guard against a parse↔encode PC convention drift: until the rewriter work +// aligned both sides on "PC measured from the first instruction", the parse +// side incidentally agreed with encode only when there were zero declared +// locals. #[test] fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { let wat = r#"(module (func (result i32) i32.const 1 i32.const 2 i32.add) - (func nop nop))"#; + (func nop nop) + (func (local i32) (local i64) + i32.const 0 + local.set 0 + i64.const 0 + local.set 1))"#; let wasm = wat::parse_str(wat).expect("wat compiles"); let module = Module::parse(&wasm, false, false, true).expect("parse"); diff --git a/src/ir/types.rs b/src/ir/types.rs index 33ed55a3..e372ef18 100644 --- a/src/ir/types.rs +++ b/src/ir/types.rs @@ -1577,15 +1577,18 @@ impl<'a> Instructions<'a> { pub fn new( instructions: Vec<(Operator<'a>, usize)>, - locals_start: usize, + pc_origin: usize, save_offsets: bool, ) -> Self { let mut instrs = vec![]; let mut pcs = vec![]; instructions.iter().for_each(|(operator, offset)| { instrs.push(operator.clone()); - // we want to store the offset inside a function body (including locals bytes)! not the overall module. - pcs.push(*offset - locals_start); + // Offsets are measured from the first instruction (after the locals + // declaration), matching the encode-side convention so the DWARF + // rewriter can compose orig and new PCs in a single coordinate + // system. + pcs.push(*offset - pc_origin); }); Self { instructions: instrs, @@ -1701,6 +1704,12 @@ impl<'a> Instructions<'a> { None } } + + /// All captured per-instruction PCs, or `None` if `with_offsets` wasn't set + /// at parse time. PCs are measured from the function's first instruction. + pub fn offsets(&self) -> Option<&[usize]> { + self.offsets.as_deref() + } } #[derive(Debug, Default, Clone)] diff --git a/tests/dwarf.rs b/tests/dwarf.rs index b1d4447c..03000178 100644 --- a/tests/dwarf.rs +++ b/tests/dwarf.rs @@ -1,12 +1,11 @@ -//! Tests for parse-aside DWARF handling (`with_dwarf = true`). +//! Tests for parse-aside DWARF handling and the `.debug_line` rewriter. //! //! When `with_dwarf` is on, `.debug_*` custom sections lift into -//! `Module::debug` instead of `custom_sections`, and encode re-emits the -//! section bytes verbatim. -//! -//! We assert on *DWARF section content* (not whole-module bytes) because -//! wirm regenerates other sections like `name` and isn't byte-identical -//! end-to-end even without instrumentation. +//! `Module::debug` instead of `custom_sections`. The other DWARF sections +//! still round-trip byte-identically (no rewriter for them yet); `.debug_line` +//! is rewritten so its row addresses match the new code layout. We verify +//! `.debug_line` semantically (rows) rather than by bytes because gimli's +//! writer may choose more compact opcodes than the input. use std::collections::BTreeMap; use std::path::PathBuf; @@ -35,6 +34,35 @@ fn debug_section_bytes(wasm: &[u8]) -> BTreeMap> { out } +/// Parses the input's `.debug_line` via gimli and collects each non-end-of- +/// sequence row as `(address, line, column)`. Used for semantic comparison +/// between input and rewritten outputs. +fn debug_line_rows(wasm: &[u8]) -> Vec<(u64, u64, u64)> { + let dl_bytes = debug_section_bytes(wasm) + .remove(".debug_line") + .expect("input has .debug_line"); + let endian = gimli::LittleEndian; + let slice = gimli::EndianSlice::new(&dl_bytes, endian); + let dl = gimli::read::DebugLine::new(slice.slice(), endian); + let program = dl + .program(gimli::DebugLineOffset(0), 4, None, None) + .expect("line program parses"); + let mut rows = program.rows(); + let mut out = Vec::new(); + while let Some((_header, row)) = rows.next_row().expect("row reads") { + if row.end_sequence() { + continue; + } + let line = row.line().map(|n| n.get()).unwrap_or(0); + let column = match row.column() { + gimli::ColumnType::LeftEdge => 0, + gimli::ColumnType::Column(c) => c.get(), + }; + out.push((row.address(), line, column)); + } + out +} + /// `with_dwarf=false` leaves DWARF flowing through `custom_sections`. The /// section bytes themselves still round-trip verbatim, since `custom_sections` /// has always been opaque pass-through. @@ -51,9 +79,10 @@ fn opaque_round_trip_preserves_dwarf_section_bytes() { assert_eq!(debug_section_bytes(&input), debug_section_bytes(&output)); } -/// `with_dwarf=true` populates `Module::debug` with the `.debug_*` sections -/// in encounter order, removes them from `custom_sections`, and re-emits them -/// verbatim during encode. +/// `with_dwarf=true` lifts `.debug_*` aside and re-emits them. The non-line +/// sections round-trip byte-identically (no rewriter for them yet); +/// `.debug_line` is rewritten so the bytes may differ but the logical rows +/// must match for an uninstrumented module. #[test] fn parse_aside_lifts_dwarf_into_debug_field() { let input = std::fs::read(input_path()).unwrap(); @@ -72,7 +101,23 @@ fn parse_aside_lifts_dwarf_into_debug_field() { } let output = module.encode().unwrap(); - assert_eq!(debug_section_bytes(&input), debug_section_bytes(&output)); + let in_sections = debug_section_bytes(&input); + let out_sections = debug_section_bytes(&output); + // Non-line sections still pass through byte-for-byte. + for name in DWARF_SECTION_NAMES.iter().filter(|n| **n != ".debug_line") { + assert_eq!( + in_sections.get(*name), + out_sections.get(*name), + "{name} should round-trip byte-identically (no rewriter for it)", + ); + } + // `.debug_line` must be semantically equivalent: same rows for an + // uninstrumented module. + assert_eq!( + debug_line_rows(&input), + debug_line_rows(&output), + ".debug_line rows must match for an uninstrumented round-trip", + ); } /// `with_dwarf=true` on a module with no DWARF still yields `Some(empty)` — @@ -89,3 +134,27 @@ fn parse_aside_empty_when_input_has_no_dwarf() { .expect("debug present even when empty"); assert!(debug.sections().is_empty()); } + +/// Explicit address-translation invariant for the rewriter: for an +/// uninstrumented module the rewritten rows must use the same addresses as +/// the input, because new layout equals orig layout. +#[test] +fn rewriter_preserves_row_addresses_uninstrumented() { + let input = std::fs::read(input_path()).unwrap(); + let in_rows = debug_line_rows(&input); + + let module = Module::parse(&input, false, false, true).unwrap(); + let output = module.encode().unwrap(); + let out_rows = debug_line_rows(&output); + + let in_addrs: Vec = in_rows.iter().map(|r| r.0).collect(); + let out_addrs: Vec = out_rows.iter().map(|r| r.0).collect(); + assert_eq!( + in_addrs, out_addrs, + "uninstrumented round-trip must preserve row addresses", + ); + + // Spot-check the test data so a future refactor that silently loses rows + // is caught: add.wasm has rows at addrs 2, 4, 6, 7. + assert_eq!(in_addrs, vec![2, 4, 6, 7]); +} From 18b5d720091fb551976c17bb4701a0c939028945 Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Thu, 4 Jun 2026 20:02:53 -0400 Subject: [PATCH 08/13] rewrite PCs in dwarf .debug_info --- DWARF-plan.md | 4 +- src/ir/dwarf.rs | 386 ++++++++++++++++++++++++++++++++++++++++++- src/ir/module/mod.rs | 141 ++++++++++------ tests/dwarf.rs | 186 ++++++++++++++++++--- 4 files changed, 644 insertions(+), 73 deletions(-) diff --git a/DWARF-plan.md b/DWARF-plan.md index 4f72da66..9f6681ac 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -35,11 +35,11 @@ Work through these in order. Each box is roughly one PR-sized unit. `Vec<(emit_order_idx, anchor_instr_idx)>` per function. Original ops map to themselves; injected ops map to their host instruction. -- [ ] **5. `.debug_line` rewriter.** Walk the input's line program; +- [x] **5. `.debug_line` rewriter.** Walk the input's line program; emit a new program where each original PC is translated via the maps from steps 2–4. For every injected PC, emit a row with the anchor's `(file, line, col, is_stmt, ...)`. -- [ ] **6. `.debug_info` and friends.** Use +- [x] **6. `.debug_info` and friends.** Use `gimli::write::Dwarf::from(&read, &addr_translator)` with an address translator that consults the same maps. This handles `DW_AT_low_pc`/`high_pc`, `DW_AT_ranges`, location lists, diff --git a/src/ir/dwarf.rs b/src/ir/dwarf.rs index 6511609c..5938f9ba 100644 --- a/src/ir/dwarf.rs +++ b/src/ir/dwarf.rs @@ -13,11 +13,22 @@ const DWARF64_LENGTH_FIELD_BYTES: usize = 12; /// Per-function input-side metadata the DWARF rewriter consults to translate /// in-function PCs to/from DWARF row addresses. +/// +/// Assumes inputs generated by `wasm-tools parse --generate-dwarf` (one CU +/// per local function in code-section order, per-function address spaces); +/// other DWARF emitters may produce inputs the rewriter currently rejects. #[derive(Clone, Debug)] pub struct OrigFuncDebugData { - /// DWARF address of the function's first instruction (= bytes of size LEB - /// + locals declarations, counted from the body's size LEB byte). + /// Byte length of the function-body size LEB. The body-content start (just + /// past the size LEB) sits at this DWARF address — wasm-tools' typical + /// `subprogram` `low_pc` lands here. + pub size_leb_len: usize, + /// DWARF address of the function's first instruction (= size LEB bytes + + /// locals-declaration bytes, counted from the body's size LEB byte). pub first_instr_dwarf_offset: usize, + /// Total bytes of the function body including the size LEB. `high_pc` + /// stored as `DW_FORM_addr` typically equals this. + pub body_total_size: usize, } #[derive(Clone, Debug, Default)] @@ -56,6 +67,13 @@ pub(crate) struct PerFuncEncodeMaps<'m> { pub pcs: &'m [usize], /// Original-instruction index each emitted op anchors to. pub anchors: &'m [usize], + /// For each orig op index, the emit position of the orig op itself (vs + /// `anchors.position(|a| a == idx)`, which would point at a leading + /// before-injection). When the orig op is replaced by an alt, this is + /// the first emit position in the alt sequence. + pub self_emit_for_orig: &'m [usize], + /// Byte length of the new function-body size LEB. + pub size_leb_len: usize, /// DWARF address of the function's first instruction in the new module. pub first_instr_dwarf_offset: usize, /// Total bytes of the new function body (size LEB + content). @@ -70,7 +88,7 @@ pub(crate) fn rewrite_debug_line( input_bytes: &[u8], debug: &ModuleDebugData, per_func_new: &HashMap>, - orig_per_instr_pcs: &HashMap>, + orig_per_instr_pcs: &HashMap, ) -> Result, Error> { use gimli::{ write::{self, Address, ConvertLineRow, DebugLine, EndianVec}, @@ -249,3 +267,365 @@ pub(crate) fn rewrite_debug_line( Ok(writer.0.into_vec()) } + +/// Translate a per-function orig DWARF address to its new equivalent. Handles +/// these zones inside the function body: +/// +/// - **End-of-function** (`addr >= body_total_size_orig`): clamp to +/// `body_total_size_new` so `high_pc` / sentinel addresses land at the new +/// one-past-end. +/// - **Function start** (`addr == 0`): identity. The size-LEB byte sits at +/// address 0 in both orig and new. +/// - **Body-content start** (`addr == size_leb_len_orig`): map to +/// `size_leb_len_new`. wasm-tools' `subprogram` `low_pc` typically lands +/// here. +/// - **First instruction** (`addr == first_instr_dwarf_offset_orig`): map to +/// `first_instr_dwarf_offset_new`. +/// - **Header zone, other addresses** (size-LEB body / locals interior): +/// DWARF rarely names these. We pass through identity and `debug_assert!` +/// so a regression surfaces during development. +/// - **Instruction zone**: find the orig op whose range contains `addr` via +/// `partition_point` on the sorted `orig_pcs`, then look up the orig op's +/// own emit position (so `low_pc` lands on the orig op rather than a +/// leading before-injection). Preserve any byte offset inside the op. +fn translate_func_addr( + orig_addr: u64, + orig_dbg: &OrigFuncDebugData, + new_maps: &PerFuncEncodeMaps<'_>, + orig_pcs: &[usize], +) -> Option { + debug_assert!( + orig_pcs.windows(2).all(|w| w[0] <= w[1]), + "orig_pcs must be sorted ascending (partition_point assumption)", + ); + + let addr = orig_addr as usize; + + if addr >= orig_dbg.body_total_size { + return Some(new_maps.body_total_size as u64); + } + if addr == 0 { + return Some(0); + } + if addr == orig_dbg.size_leb_len { + return Some(new_maps.size_leb_len as u64); + } + if addr == orig_dbg.first_instr_dwarf_offset { + return Some(new_maps.first_instr_dwarf_offset as u64); + } + if addr < orig_dbg.first_instr_dwarf_offset { + debug_assert!( + false, + "DWARF addr in function header but not at a known boundary: {addr}" + ); + return Some(orig_addr); + } + + let pc_in_func = addr - orig_dbg.first_instr_dwarf_offset; + // `orig_pcs` is sorted ascending; `partition_point` finds the first index + // whose PC exceeds `pc_in_func`, so the preceding index is the orig op + // containing the address. + let after = orig_pcs.partition_point(|&p| p <= pc_in_func); + if after == 0 { + // Shouldn't happen — we already handled addr <= first_instr above. + return None; + } + let orig_idx = after - 1; + let emit_idx = new_maps.self_emit_for_orig[orig_idx]; + if emit_idx == usize::MAX { + // Orig op was dropped during encode (e.g., a future deletion API). + // Surface as InvalidAddress through the closure rather than panicking. + return None; + } + let offset_in_op = pc_in_func - orig_pcs[orig_idx]; + let new_pc = new_maps.pcs[emit_idx]; + Some((new_maps.first_instr_dwarf_offset + new_pc + offset_in_op) as u64) +} + +/// Rewrite the PC-bearing DWARF sections other than `.debug_line` (handled by +/// `rewrite_debug_line`). Uses `gimli::write::Dwarf::convert` with a manual +/// per-DIE attribute walk so `DW_AT_low_pc` / `DW_AT_high_pc` stored as +/// constants (wasm-tools' convention) — which gimli's auto-conversion would +/// pass through verbatim — get translated alongside `DW_FORM_addr` values. +/// +/// Single-function inputs only: with multiple local functions the per-function +/// DWARF address spaces overlap (each starts at 0) so the translator cannot +/// disambiguate. +pub(crate) fn rewrite_other_dwarf_sections( + debug: &ModuleDebugData, + per_func_new: &HashMap>, + orig_per_instr_pcs: &HashMap, +) -> Result>, Error> { + use gimli::{ + write::{self, Address, EndianVec, Sections}, + EndianSlice, LittleEndian, SectionId, + }; + + if !debug.sections().iter().any(|s| s.name == ".debug_info") { + return Ok(HashMap::new()); + } + if per_func_new.len() > 1 { + return Err(Error::DwarfError( + "multi-function .debug_info rewriting not yet supported".to_string(), + )); + } + + let endian = LittleEndian; + let empty = EndianSlice::new(&[], endian); + let section_lookup = |name: &str| -> EndianSlice<'_, LittleEndian> { + debug + .sections() + .iter() + .find(|s| s.name == name) + .map(|s| EndianSlice::new(s.data.as_ref(), endian)) + .unwrap_or(empty) + }; + + let dwarf_read = gimli::read::Dwarf::load(|id| -> Result<_, gimli::Error> { + Ok(match id { + SectionId::DebugInfo => section_lookup(".debug_info"), + SectionId::DebugAbbrev => section_lookup(".debug_abbrev"), + SectionId::DebugLine => section_lookup(".debug_line"), + SectionId::DebugStr => section_lookup(".debug_str"), + SectionId::DebugLineStr => section_lookup(".debug_line_str"), + SectionId::DebugStrOffsets => section_lookup(".debug_str_offsets"), + SectionId::DebugLoc => section_lookup(".debug_loc"), + SectionId::DebugLocLists => section_lookup(".debug_loclists"), + SectionId::DebugRanges => section_lookup(".debug_ranges"), + SectionId::DebugRngLists => section_lookup(".debug_rnglists"), + SectionId::DebugAranges => section_lookup(".debug_aranges"), + SectionId::DebugFrame => section_lookup(".debug_frame"), + _ => empty, + }) + }) + .map_err(|e| Error::DwarfError(format!("loading DWARF: {e}")))?; + + // Cache the single captured func entry outside the per-address closure. + let single = per_func_new.iter().next().and_then(|(idx, new_maps)| { + let orig_dbg = debug.per_func.get(idx)?; + let orig_pcs = orig_per_instr_pcs.get(idx)?; + Some((orig_dbg, new_maps, orig_pcs)) + }); + let translator = move |orig_addr: u64| -> Option
{ + let Some((orig_dbg, new_maps, orig_pcs)) = single else { + return Some(Address::Constant(orig_addr)); + }; + translate_func_addr(orig_addr, orig_dbg, new_maps, orig_pcs).map(Address::Constant) + }; + + let mut dwarf_write = write::Dwarf::default(); + convert_units_with_low_high_pc(&mut dwarf_write, &dwarf_read, &translator) + .map_err(|e| Error::DwarfError(format!("converting DWARF: {e}")))?; + + let mut sections = Sections::new(EndianVec::new(endian)); + dwarf_write + .write(&mut sections) + .map_err(|e| Error::DwarfError(format!("writing DWARF: {e}")))?; + + let mut out: HashMap<&'static str, Vec> = HashMap::new(); + let mut take = |name: &'static str, bytes: Vec| { + if !bytes.is_empty() { + out.insert(name, bytes); + } + }; + // `gimli::write::Sections` does not include `.debug_aranges` or + // `.debug_str_offsets`: gimli doesn't generate those, so they currently + // pass through verbatim and may go stale after instrumentation. Worth + // a follow-up for inputs that carry an aranges table. + take(".debug_info", sections.debug_info.0.into_vec()); + take(".debug_abbrev", sections.debug_abbrev.0.into_vec()); + take(".debug_str", sections.debug_str.0.into_vec()); + take(".debug_line_str", sections.debug_line_str.0.into_vec()); + take(".debug_loc", sections.debug_loc.0.into_vec()); + take(".debug_loclists", sections.debug_loclists.0.into_vec()); + take(".debug_ranges", sections.debug_ranges.0.into_vec()); + take(".debug_rnglists", sections.debug_rnglists.0.into_vec()); + take(".debug_frame", sections.debug_frame.0.into_vec()); + Ok(out) +} + +/// Drive `gimli::write::Dwarf::convert` per-unit, walking each DIE manually so +/// we can intercept `DW_AT_low_pc` / `DW_AT_high_pc` stored as constants and +/// translate them. Per gimli's own contract, the default `convert_attribute_value` +/// passes `Data*` / `Udata` through verbatim — which would leave function +/// length attributes stale after instrumentation. We sidestep that here. +/// +/// `.debug_line` writing is left to the dedicated step-5 rewriter: gimli's +/// auto-converted line program is still attached to each unit via +/// `set_line_program`, but `.debug_line` bytes for the output module come +/// from `rewrite_debug_line`. The caller writes our line bytes after this +/// function's output. +fn convert_units_with_low_high_pc( + dwarf_write: &mut gimli::write::Dwarf, + dwarf_read: &gimli::read::Dwarf>, + translator: &dyn Fn(u64) -> Option, +) -> gimli::write::ConvertResult<()> { + let mut conv_section = dwarf_write.convert(dwarf_read)?; + while let Some((mut conv_unit, root_entry)) = conv_section.read_unit()? { + if let Some(convert_program) = conv_unit.read_line_program(None, None)? { + let (program, files) = convert_program.convert(translator)?; + conv_unit.set_line_program(program, files); + } + + let root_id = conv_unit.unit.root(); + convert_die(&mut conv_unit, root_id, &root_entry, translator)?; + + let mut entry = root_entry; + while let Some(id_opt) = conv_unit.read_entry(&mut entry)? { + if id_opt.is_none() { + continue; + } + let id = conv_unit.add_entry(id_opt, &entry); + convert_die(&mut conv_unit, id, &entry, translator)?; + } + } + Ok(()) +} + +/// Convert one DIE's attributes, intercepting `DW_AT_low_pc` (any form) and +/// `DW_AT_high_pc` (`Data*`/`Udata` = offset length per the DWARF spec) so +/// instrumented modules get the right addresses. Other attributes go through +/// `convert_attribute_value` unchanged. +/// +/// One pre-scan finds `low_pc` and translates it once; the main loop reuses +/// the cached translation when writing both `low_pc` and `high_pc`. Mirrors +/// gimli's own skip of `DW_AT_GNU_locviews` (an unsupported GNU extension). +fn convert_die>( + conv_unit: &mut gimli::write::ConvertUnit<'_, R>, + id: gimli::write::UnitEntryId, + entry: &gimli::write::ConvertUnitEntry<'_, R>, + translator: &dyn Fn(u64) -> Option, +) -> gimli::write::ConvertResult<()> { + use gimli::{constants::*, write::ConvertError}; + + // Pre-scan for low_pc and translate it once. DWARF doesn't strictly order + // low_pc before high_pc, so this can't be folded into the main loop + // without buffering. + let orig_low_pc: Option = entry + .attrs + .iter() + .find(|attr| attr.name() == DW_AT_low_pc) + .and_then(|attr| read_pc_uint(&attr.value())); + let new_low_pc_abs: Option = match orig_low_pc { + Some(v) => match translator(v) { + Some(gimli::write::Address::Constant(a)) => Some(a), + Some(_) | None => return Err(ConvertError::InvalidAddress), + }, + None => None, + }; + + for attr in &entry.attrs { + let name = attr.name(); + if name == DW_AT_GNU_locviews { + continue; + } + let value = if name == DW_AT_low_pc { + write_low_pc_attr( + attr, + new_low_pc_abs.expect("we set it from this attr above"), + )? + } else if name == DW_AT_high_pc { + translate_high_pc_attr(attr, translator, orig_low_pc, new_low_pc_abs)? + } else { + conv_unit.convert_attribute_value(entry.read_unit, attr, translator)? + }; + conv_unit.unit.get_mut(id).set(name, value); + } + Ok(()) +} + +/// Decode a `DW_AT_low_pc` / `DW_AT_high_pc` raw value into the absolute u64 +/// the rewriter operates on. Used by both passes through `convert_die` so the +/// form recognition can't drift between them. +fn read_pc_uint>( + value: &gimli::read::AttributeValue, +) -> Option { + use gimli::read::AttributeValue as RA; + match *value { + RA::Addr(a) => Some(a), + RA::Data1(d) => Some(d as u64), + RA::Data2(d) => Some(d as u64), + RA::Data4(d) => Some(d as u64), + RA::Data8(d) => Some(d), + RA::Udata(d) => Some(d), + _ => None, + } +} + +/// Write a translated `low_pc` value back in the same form it was read in. +/// Unexpected forms (e.g. `Sdata`, string refs) are an error rather than a +/// silent zero — the wast convention only uses `Addr`/`Data4`. +fn write_low_pc_attr>( + attr: &gimli::read::Attribute, + new_abs: u64, +) -> gimli::write::ConvertResult { + use gimli::{read::AttributeValue as RA, write::AttributeValue as WA, write::ConvertError}; + Ok(match attr.value() { + RA::Addr(_) => WA::Address(gimli::write::Address::Constant(new_abs)), + RA::Data1(_) => WA::Data1(u8::try_from(new_abs).map_err(|_| ConvertError::InvalidAddress)?), + RA::Data2(_) => { + WA::Data2(u16::try_from(new_abs).map_err(|_| ConvertError::InvalidAddress)?) + } + RA::Data4(_) => { + WA::Data4(u32::try_from(new_abs).map_err(|_| ConvertError::InvalidAddress)?) + } + RA::Data8(_) => WA::Data8(new_abs), + RA::Udata(_) => WA::Udata(new_abs), + _ => return Err(ConvertError::InvalidAttributeValue), + }) +} + +fn translate_high_pc_attr>( + attr: &gimli::read::Attribute, + translator: &dyn Fn(u64) -> Option, + orig_low_pc: Option, + new_low_pc_abs: Option, +) -> gimli::write::ConvertResult { + use gimli::{read::AttributeValue as RA, write::AttributeValue as WA, write::ConvertError}; + + // For Addr form, high_pc is an absolute address: translate directly. + if let RA::Addr(a) = attr.value() { + return match translator(a) { + Some(addr) => Ok(WA::Address(addr)), + None => Err(ConvertError::InvalidAddress), + }; + } + + // For Data*/Udata forms, high_pc is a length (offset from low_pc). Recover + // the orig absolute via orig_low + length, translate it, subtract new low. + let length = match attr.value() { + RA::Data1(d) => d as u64, + RA::Data2(d) => d as u64, + RA::Data4(d) => d as u64, + RA::Data8(d) => d, + RA::Udata(d) => d, + _ => return Err(ConvertError::InvalidAttributeValue), + }; + let (Some(orig_low), Some(new_low)) = (orig_low_pc, new_low_pc_abs) else { + return Err(ConvertError::InvalidAttributeValue); + }; + let orig_high_abs = orig_low + .checked_add(length) + .ok_or(ConvertError::InvalidAttributeValue)?; + let new_high_abs = match translator(orig_high_abs) { + Some(gimli::write::Address::Constant(a)) => a, + _ => return Err(ConvertError::InvalidAddress), + }; + let new_length = new_high_abs + .checked_sub(new_low) + .ok_or(ConvertError::InvalidAttributeValue)?; + Ok(match attr.value() { + RA::Data1(_) => { + WA::Data1(u8::try_from(new_length).map_err(|_| ConvertError::InvalidAttributeValue)?) + } + RA::Data2(_) => { + WA::Data2(u16::try_from(new_length).map_err(|_| ConvertError::InvalidAttributeValue)?) + } + RA::Data4(_) => { + WA::Data4(u32::try_from(new_length).map_err(|_| ConvertError::InvalidAttributeValue)?) + } + RA::Data8(_) => WA::Data8(new_length), + RA::Udata(_) => WA::Udata(new_length), + _ => unreachable!("filtered above"), + }) +} diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 2b0fe53d..e9095dd7 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -245,7 +245,9 @@ impl<'a> Module<'a> { let first_instr_byte = body.get_binary_reader_for_operators()?.original_position(); let first_instr_offset_in_content = first_instr_byte - body_range.start; Ok(OrigFuncDebugData { + size_leb_len, first_instr_dwarf_offset: size_leb_len + first_instr_offset_in_content, + body_total_size: size_leb_len + body_content_size, }) } @@ -1299,19 +1301,25 @@ impl<'a> Module<'a> { // from the locals declaration) onto the first instruction, matching // the parse-side convention used by `Instructions::new`. let locals_base = function.byte_len(); + let n_ops = instructions.len(); let mut capture = capture_pcs.then_some(PcCapture { maps: FuncDwarfMaps { pcs: Vec::new(), anchors: Vec::new(), + // Sentinel: any unwritten slot means the orig op was never + // emitted (e.g., a deleted op via a future API). Step 6 + // refuses translation if an address resolves to such a slot. + self_emit_for_orig: vec![usize::MAX; n_ops], // Filled in at the end of `encode_function` once the body's // total content length is known. + size_leb_len: 0, first_instr_dwarf_offset: 0, body_total_size: 0, }, locals_base, current_anchor: 0, }); - let instr_len = instructions.len() - 1; + let instr_len = n_ops - 1; let (ops, mut flags) = instructions.get_ops_flags_mut(); for (idx, op) in ops.iter_mut().enumerate() { // Every op emitted during this iteration anchors to `idx` (the original @@ -1321,11 +1329,13 @@ impl<'a> Module<'a> { } fix_op_id_mapping(op, func_mapping, global_mapping, memory_mapping)?; if flags.is_none() { + record_self_emit(&mut capture, idx); encode(&op.clone(), &mut function, &mut reencode, &mut capture); continue; } let instrument = &mut flags.as_mut().unwrap()[idx]; if !instrument.has_instr() { + record_self_emit(&mut capture, idx); encode(&op.clone(), &mut function, &mut reencode, &mut capture); continue; } else { @@ -1357,6 +1367,9 @@ impl<'a> Module<'a> { // If there are any alternate, encode the alternate if !at_end && !alternate.is_none() { if let Some(alt) = alternate { + // The alt replaces the orig op; its first emit is the + // best analog of "where this orig op landed". + record_self_emit(&mut capture, idx); update_ids_and_encode( &mut alt.instrs, func_mapping, @@ -1368,6 +1381,7 @@ impl<'a> Module<'a> { )?; } } else { + record_self_emit(&mut capture, idx); encode(&op.clone(), &mut function, &mut reencode, &mut capture); } @@ -1385,6 +1399,12 @@ impl<'a> Module<'a> { } } + fn record_self_emit(capture: &mut Option, idx: usize) { + if let Some(cap) = capture.as_mut() { + cap.maps.self_emit_for_orig[idx] = cap.maps.pcs.len(); + } + } + fn update_ids_and_encode( instrs: &mut Vec, func_mapping: &HashMap, @@ -1428,6 +1448,7 @@ impl<'a> Module<'a> { // instructions; the LEB encodes that value when emitted later). let body_content_len = function.byte_len(); let size_leb_len = unsigned_leb128_len(body_content_len as u64); + cap.maps.size_leb_len = size_leb_len; cap.maps.first_instr_dwarf_offset = size_leb_len + cap.locals_base; cap.maps.body_total_size = size_leb_len + body_content_len; } @@ -2017,58 +2038,78 @@ impl<'a> Module<'a> { }); } - // Rewrite the `.debug_line` section so its addresses match the - // new code layout. We then re-emit the DWARF sections below; the - // other `.debug_*` sections still pass through verbatim (step 6 - // covers `.debug_info`, location lists, etc.). + // Rewrite `.debug_line` (anchor-aware) and the other PC-bearing DWARF + // sections (address-translated via gimli::write::Dwarf::from) so the + // re-emit loop below sees coherent bytes. if capture_pcs { if let Some(debug) = tmp.debug.as_ref() { + // `with_offsets` is auto-enabled with `with_dwarf`, so offsets + // are always present here — treat their absence as a bug. We + // borrow them out of `tmp.functions` for the duration of the + // rewriter calls; no clone needed. + let mut orig_per_instr_pcs: HashMap = HashMap::new(); + for (func_idx, _) in per_func_maps.iter() { + let local = tmp + .functions + .unwrap_local(FunctionID(*func_idx)) + .expect("captured DWARF map should be for a local function"); + let offsets = local + .body + .instructions + .offsets() + .expect("with_dwarf opt-in must populate orig per-instr offsets"); + orig_per_instr_pcs.insert(*func_idx, offsets); + } + let per_func_new: HashMap = per_func_maps + .iter() + .map(|(idx, m)| { + ( + *idx, + crate::ir::dwarf::PerFuncEncodeMaps { + pcs: &m.pcs, + anchors: &m.anchors, + self_emit_for_orig: &m.self_emit_for_orig, + size_leb_len: m.size_leb_len, + first_instr_dwarf_offset: m.first_instr_dwarf_offset, + body_total_size: m.body_total_size, + }, + ) + }) + .collect(); + let line_idx = debug .sections() .iter() .position(|s| s.name == ".debug_line"); - if let Some(line_idx) = line_idx { - // Build per-instr orig PCs for each local function with - // captured maps. `with_offsets` is auto-enabled at parse - // time when `with_dwarf` is on, so offsets are always - // present here — treat their absence as a bug. - let mut orig_per_instr_pcs: HashMap> = HashMap::new(); - for (func_idx, _) in per_func_maps.iter() { - let local = tmp - .functions - .unwrap_local(FunctionID(*func_idx)) - .expect("captured DWARF map should be for a local function"); - let offsets = local - .body - .instructions - .offsets() - .expect("with_dwarf opt-in must populate orig per-instr offsets"); - orig_per_instr_pcs.insert(*func_idx, offsets.to_vec()); - } - let per_func_new: HashMap = - per_func_maps - .iter() - .map(|(idx, m)| { - ( - *idx, - crate::ir::dwarf::PerFuncEncodeMaps { - pcs: &m.pcs, - anchors: &m.anchors, - first_instr_dwarf_offset: m.first_instr_dwarf_offset, - body_total_size: m.body_total_size, - }, - ) - }) - .collect(); + let new_line_bytes = if let Some(line_idx) = line_idx { let input_bytes = debug.sections()[line_idx].data.as_ref(); - let new_bytes = crate::ir::dwarf::rewrite_debug_line( - input_bytes, - debug, - &per_func_new, - &orig_per_instr_pcs, - )?; - if let Some(debug_mut) = tmp.debug.as_mut() { - debug_mut.sections[line_idx].data = std::borrow::Cow::Owned(new_bytes); + Some(( + line_idx, + crate::ir::dwarf::rewrite_debug_line( + input_bytes, + debug, + &per_func_new, + &orig_per_instr_pcs, + )?, + )) + } else { + None + }; + let new_other_sections = crate::ir::dwarf::rewrite_other_dwarf_sections( + debug, + &per_func_new, + &orig_per_instr_pcs, + )?; + + if let Some(debug_mut) = tmp.debug.as_mut() { + if let Some((line_idx, bytes)) = new_line_bytes { + debug_mut.sections[line_idx].data = std::borrow::Cow::Owned(bytes); + } + let mut new_other_sections = new_other_sections; + for section in debug_mut.sections.iter_mut() { + if let Some(new_bytes) = new_other_sections.remove(section.name) { + section.data = std::borrow::Cow::Owned(new_bytes); + } } } } @@ -2575,6 +2616,14 @@ pub(crate) struct FuncDwarfMaps { /// anchor to themselves; injected ops inherit their host op's idx so a /// debugger never "stops inside" wirm-injected code. pub(crate) anchors: Vec, + /// For each orig op index, the emit position of the orig op itself (the + /// point where the orig op's encoding starts, vs the first emit anchored + /// to it which may be a before-injection). Step 6 uses this to translate + /// `low_pc`/`high_pc` that name an instruction byte rather than a + /// debugger-stop point. + pub(crate) self_emit_for_orig: Vec, + /// Byte length of the new function-body size LEB. + pub(crate) size_leb_len: usize, /// DWARF address of the first instruction in the new function body /// (= size-LEB bytes + locals-declaration bytes). Composes with `pcs[i]` /// to give per-function DWARF row addresses: `addr = first_instr_dwarf_offset + pcs[i]`. diff --git a/tests/dwarf.rs b/tests/dwarf.rs index 03000178..a3f07d54 100644 --- a/tests/dwarf.rs +++ b/tests/dwarf.rs @@ -1,11 +1,12 @@ -//! Tests for parse-aside DWARF handling and the `.debug_line` rewriter. +//! Tests for parse-aside DWARF handling, the `.debug_line` rewriter (step 5), +//! and the address-translated rewriter for the other DWARF sections (step 6). //! //! When `with_dwarf` is on, `.debug_*` custom sections lift into -//! `Module::debug` instead of `custom_sections`. The other DWARF sections -//! still round-trip byte-identically (no rewriter for them yet); `.debug_line` -//! is rewritten so its row addresses match the new code layout. We verify -//! `.debug_line` semantically (rows) rather than by bytes because gimli's -//! writer may choose more compact opcodes than the input. +//! `Module::debug` instead of `custom_sections`. `.debug_line` is rewritten +//! with anchor-aware row inheritance; `.debug_info` (+ `.debug_abbrev`, +//! `.debug_str` etc.) is rewritten via `gimli::write::Dwarf::from` with an +//! address translator. Both rewriters preserve semantics, not byte content, +//! so the tests check logical equivalence rather than byte equality. use std::collections::BTreeMap; use std::path::PathBuf; @@ -34,6 +35,66 @@ fn debug_section_bytes(wasm: &[u8]) -> BTreeMap> { out } +/// Walks `.debug_info` and collects each DIE's `(low_pc, high_pc)` pair. Used +/// for semantic comparison since the rewriter may emit different bytes (e.g. +/// different abbreviation codes) but must preserve address ranges. +fn debug_info_pcs(wasm: &[u8]) -> Vec<(u64, u64)> { + let sections = debug_section_bytes(wasm); + let endian = gimli::LittleEndian; + let lookup = |name: &str| -> gimli::EndianSlice<'_, gimli::LittleEndian> { + gimli::EndianSlice::new( + sections.get(name).map(|v| v.as_slice()).unwrap_or(&[]), + endian, + ) + }; + let dwarf = gimli::read::Dwarf::load(|id| -> Result<_, gimli::Error> { + Ok(match id { + gimli::SectionId::DebugInfo => lookup(".debug_info"), + gimli::SectionId::DebugAbbrev => lookup(".debug_abbrev"), + gimli::SectionId::DebugStr => lookup(".debug_str"), + gimli::SectionId::DebugLine => lookup(".debug_line"), + gimli::SectionId::DebugLineStr => lookup(".debug_line_str"), + _ => gimli::EndianSlice::new(&[], endian), + }) + }) + .expect("load DWARF"); + + let mut out = Vec::new(); + let mut units = dwarf.units(); + while let Some(header) = units.next().expect("unit header") { + let unit = dwarf.unit(header).expect("unit"); + let mut entries = unit.entries(); + while let Some(entry) = entries.next_dfs().expect("dfs") { + // wasm-tools' CU stores low_pc/high_pc as DW_FORM_data4 (low_pc = + // absolute, high_pc = length per the DWARF spec); the subprogram + // uses DW_FORM_addr for low_pc and data4 for high_pc. Accept both. + let read_uint = |v: gimli::read::AttributeValue<_>| -> Option { + match v { + gimli::read::AttributeValue::Addr(a) => Some(a), + gimli::read::AttributeValue::Data1(d) => Some(d as u64), + gimli::read::AttributeValue::Data2(d) => Some(d as u64), + gimli::read::AttributeValue::Data4(d) => Some(d as u64), + gimli::read::AttributeValue::Data8(d) => Some(d), + gimli::read::AttributeValue::Udata(d) => Some(d), + _ => None, + } + }; + let low = entry.attr_value(gimli::DW_AT_low_pc).and_then(read_uint); + let high_raw = entry.attr_value(gimli::DW_AT_high_pc).and_then(read_uint); + // For Addr form high_pc is absolute; for Data*/Udata it's a length. + let high = match entry.attr_value(gimli::DW_AT_high_pc) { + Some(gimli::read::AttributeValue::Addr(_)) => high_raw, + Some(_) => high_raw.zip(low).map(|(l_len, l_addr)| l_addr + l_len), + None => None, + }; + if let (Some(l), Some(h)) = (low, high) { + out.push((l, h)); + } + } + } + out +} + /// Parses the input's `.debug_line` via gimli and collects each non-end-of- /// sequence row as `(address, line, column)`. Used for semantic comparison /// between input and rewritten outputs. @@ -79,10 +140,11 @@ fn opaque_round_trip_preserves_dwarf_section_bytes() { assert_eq!(debug_section_bytes(&input), debug_section_bytes(&output)); } -/// `with_dwarf=true` lifts `.debug_*` aside and re-emits them. The non-line -/// sections round-trip byte-identically (no rewriter for them yet); -/// `.debug_line` is rewritten so the bytes may differ but the logical rows -/// must match for an uninstrumented module. +/// `with_dwarf=true` lifts `.debug_*` aside and re-emits them. For an +/// uninstrumented round-trip the rewriters preserve DWARF semantics: every +/// `.debug_line` row and every `.debug_info` `(low_pc, high_pc)` pair must +/// match input. Byte equality is not required since gimli's encoder is free +/// to pick different abbreviation codes, opcode sequences, etc. #[test] fn parse_aside_lifts_dwarf_into_debug_field() { let input = std::fs::read(input_path()).unwrap(); @@ -101,23 +163,16 @@ fn parse_aside_lifts_dwarf_into_debug_field() { } let output = module.encode().unwrap(); - let in_sections = debug_section_bytes(&input); - let out_sections = debug_section_bytes(&output); - // Non-line sections still pass through byte-for-byte. - for name in DWARF_SECTION_NAMES.iter().filter(|n| **n != ".debug_line") { - assert_eq!( - in_sections.get(*name), - out_sections.get(*name), - "{name} should round-trip byte-identically (no rewriter for it)", - ); - } - // `.debug_line` must be semantically equivalent: same rows for an - // uninstrumented module. assert_eq!( debug_line_rows(&input), debug_line_rows(&output), ".debug_line rows must match for an uninstrumented round-trip", ); + assert_eq!( + debug_info_pcs(&input), + debug_info_pcs(&output), + ".debug_info DIE address ranges must match for an uninstrumented round-trip", + ); } /// `with_dwarf=true` on a module with no DWARF still yields `Some(empty)` — @@ -135,6 +190,93 @@ fn parse_aside_empty_when_input_has_no_dwarf() { assert!(debug.sections().is_empty()); } +/// Instrumented round-trip: injecting 4 nops before `i32.add` grows the +/// function body by 4 bytes; `.debug_info`'s `(low_pc, high_pc)` must +/// reflect the new size, not the original. This is the regression case the +/// `Dwarf::convert` fix in step 6 was added for. +#[test] +fn rewriter_translates_high_pc_after_body_growth() { + use wasmparser::Operator; + use wirm::iterator::iterator_trait::{IteratingInstrumenter, Iterator}; + use wirm::Opcode; + + let input = std::fs::read(input_path()).unwrap(); + let in_pcs = debug_info_pcs(&input); + assert_eq!(in_pcs, vec![(0, 8), (1, 8)], "fixture sanity check"); + + let mut module = Module::parse(&input, false, false, true).unwrap(); + { + let mut it = wirm::iterator::module_iterator::ModuleIterator::new(&mut module, &Vec::new()); + loop { + if matches!(it.curr_op(), Some(Operator::I32Add)) { + it.before().nop().nop().nop().nop(); + break; + } + if it.next().is_none() { + break; + } + } + } + + let output = module.encode().unwrap(); + let out_pcs = debug_info_pcs(&output); + + // Body grew by 4 bytes. CU spans the whole body: (0, 8+4)=(0,12). + // Subprogram starts after the size LEB at the locals byte: (1, 12). + assert_eq!( + out_pcs, + vec![(0, 12), (1, 12)], + ".debug_info DIE ranges must reflect the post-injection body size", + ); +} + +/// Alternate-path coverage: replacing `i32.add` (1 byte) with `nop nop nop` +/// (3 bytes) grows the body by 2 bytes. This exercises `self_emit_for_orig` +/// on the alt branch — the orig op `i32.add` is *not* emitted, but its +/// self-emit slot must still point at the first alt instruction so any DIE +/// addressing the orig op lands on the alt's start byte. +#[test] +fn rewriter_handles_alternate_replacement() { + use wasmparser::Operator; + use wirm::iterator::iterator_trait::{IteratingInstrumenter, Iterator}; + use wirm::Opcode; + + let input = std::fs::read(input_path()).unwrap(); + let mut module = Module::parse(&input, false, false, true).unwrap(); + { + let mut it = wirm::iterator::module_iterator::ModuleIterator::new(&mut module, &Vec::new()); + loop { + if matches!(it.curr_op(), Some(Operator::I32Add)) { + it.alternate().nop().nop().nop(); + break; + } + if it.next().is_none() { + break; + } + } + } + + let output = module.encode().unwrap(); + let out_pcs = debug_info_pcs(&output); + + // Body grew by 2 bytes (3 nops − 1 i32.add). CU/subprogram extents track. + assert_eq!( + out_pcs, + vec![(0, 10), (1, 10)], + ".debug_info DIE ranges must reflect the alt-induced body size delta", + ); + + // Sanity-check `.debug_line`: the i32.add's row must be addressable in + // the rewritten output. With the alt path, `self_emit_for_orig[i32.add]` + // points at the first nop's emit position, so the row stays present at + // the alt's start byte (offset_in_op = 0). + let out_rows = debug_line_rows(&output); + assert!( + !out_rows.is_empty(), + ".debug_line must still cover the function after alt replacement", + ); +} + /// Explicit address-translation invariant for the rewriter: for an /// uninstrumented module the rewritten rows must use the same addresses as /// the input, because new layout equals orig layout. From eed716be48d6d7034136f2e7fa7fc1beb592fcfa Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Thu, 4 Jun 2026 20:14:13 -0400 Subject: [PATCH 09/13] add rewritten dwarf as custom sections --- DWARF-plan.md | 2 +- src/ir/module/mod.rs | 14 ++++++++++++++ tests/dwarf.rs | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+), 1 deletion(-) diff --git a/DWARF-plan.md b/DWARF-plan.md index 9f6681ac..4701d2b6 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -44,7 +44,7 @@ Work through these in order. Each box is roughly one PR-sized unit. address translator that consults the same maps. This handles `DW_AT_low_pc`/`high_pc`, `DW_AT_ranges`, location lists, aranges, frame info — anything that holds a PC. -- [ ] **7. Re-emit DWARF as custom sections.** Encoded DWARF goes into +- [x] **7. Re-emit DWARF as custom sections.** Encoded DWARF goes into the output module's custom sections in the conventional order. - [ ] **8. Unit/regression tests.** Five or six hand-written `.wat` inputs run through `wasm-tools parse --generate-dwarf full`, diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index e9095dd7..525031cb 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -2111,6 +2111,20 @@ impl<'a> Module<'a> { section.data = std::borrow::Cow::Owned(new_bytes); } } + // Append any DWARF sections gimli produced that the input + // didn't carry (e.g. `.debug_line_str` for inputs whose + // strings only become inlined after rewrite). They land at + // the tail in deterministic name order. + let mut new_names: Vec<&'static str> = + new_other_sections.keys().copied().collect(); + new_names.sort_unstable(); + for name in new_names { + let bytes = new_other_sections.remove(name).unwrap(); + debug_mut.sections.push(CustomSection { + name, + data: std::borrow::Cow::Owned(bytes), + }); + } } } } diff --git a/tests/dwarf.rs b/tests/dwarf.rs index a3f07d54..731f6b00 100644 --- a/tests/dwarf.rs +++ b/tests/dwarf.rs @@ -277,6 +277,42 @@ fn rewriter_handles_alternate_replacement() { ); } +/// Rewritten DWARF lands in the output module's custom sections in +/// input order — re-emit is order-preserving so downstream tooling that +/// scans `.debug_*` payloads doesn't have to deal with reshuffled section +/// layouts. +#[test] +fn rewriter_preserves_dwarf_section_order_in_output() { + let input = std::fs::read(input_path()).unwrap(); + let module = Module::parse(&input, false, false, true).unwrap(); + let output = module.encode().unwrap(); + + let mut out_names: Vec = Vec::new(); + for payload in wasmparser::Parser::new(0).parse_all(&output) { + if let wasmparser::Payload::CustomSection(cs) = payload.expect("valid wasm") { + if cs.name().starts_with(".debug_") { + out_names.push(cs.name().to_string()); + } + } + } + let input_prefix: Vec = DWARF_SECTION_NAMES + .iter() + .map(|s| (*s).to_string()) + .collect(); + assert!( + out_names.starts_with(&input_prefix), + "input `.debug_*` sections must appear in input order at the head of \ + the output: expected prefix {input_prefix:?}, got {out_names:?}", + ); + // For add.wasm, gimli additionally materializes `.debug_line_str` (the v5 + // path table) during the .debug_line rewrite. Pin the exact tail so a + // future change that drops or reorders gimli additions is caught. + assert_eq!( + &out_names[input_prefix.len()..], + &[".debug_line_str".to_string()], + ); +} + /// Explicit address-translation invariant for the rewriter: for an /// uninstrumented module the rewritten rows must use the same addresses as /// the input, because new layout equals orig layout. From 41e9cb4e1c3aa4c440904010ebea993856301aae Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Thu, 4 Jun 2026 21:05:38 -0400 Subject: [PATCH 10/13] lots of unit and property tests --- Cargo.lock | 156 ++++++++++ Cargo.toml | 1 + DWARF-plan.md | 8 +- src/ir/module/test.rs | 289 ++++++++++++++++++ tests/common/dwarf.rs | 138 +++++++++ tests/common/mod.rs | 1 + tests/dwarf.rs | 169 +++++----- tests/test_inputs/handwritten/dwarf/README.md | 33 ++ .../handwritten/dwarf/two_funcs.wasm | Bin 0 -> 441 bytes .../handwritten/dwarf/two_funcs.wat | 14 + 10 files changed, 716 insertions(+), 93 deletions(-) create mode 100644 tests/common/dwarf.rs create mode 100644 tests/test_inputs/handwritten/dwarf/README.md create mode 100644 tests/test_inputs/handwritten/dwarf/two_funcs.wasm create mode 100644 tests/test_inputs/handwritten/dwarf/two_funcs.wat diff --git a/Cargo.lock b/Cargo.lock index ca02043f..61f4d89a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -40,12 +40,33 @@ dependencies = [ "syn", ] +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + [[package]] name = "base64" version = "0.22.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" +[[package]] +name = "bit-set" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3" +dependencies = [ + "bit-vec", +] + +[[package]] +name = "bit-vec" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" + [[package]] name = "bitflags" version = "2.10.0" @@ -702,6 +723,15 @@ dependencies = [ "rustix", ] +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + [[package]] name = "object" version = "0.39.1" @@ -760,6 +790,15 @@ dependencies = [ "serde", ] +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + [[package]] name = "proc-macro2" version = "1.0.103" @@ -769,6 +808,25 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "proptest" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744" +dependencies = [ + "bit-set", + "bit-vec", + "bitflags", + "num-traits", + "rand", + "rand_chacha", + "rand_xorshift", + "regex-syntax", + "rusty-fork", + "tempfile", + "unarray", +] + [[package]] name = "pulley-interpreter" version = "44.0.0" @@ -792,6 +850,12 @@ dependencies = [ "syn", ] +[[package]] +name = "quick-error" +version = "1.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0" + [[package]] name = "quote" version = "1.0.42" @@ -807,6 +871,44 @@ version = "5.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" +[[package]] +name = "rand" +version = "0.9.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "44c5af06bb1b7d3216d91932aed5265164bf384dc89cd6ba05cf59a35f5f76ea" +dependencies = [ + "rand_chacha", + "rand_core", +] + +[[package]] +name = "rand_chacha" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb" +dependencies = [ + "ppv-lite86", + "rand_core", +] + +[[package]] +name = "rand_core" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c" +dependencies = [ + "getrandom 0.3.4", +] + +[[package]] +name = "rand_xorshift" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "513962919efc330f829edb2535844d1b912b0fbe2ca165d613e4e8788bb05a5a" +dependencies = [ + "rand_core", +] + [[package]] name = "rayon" version = "1.11.0" @@ -852,6 +954,12 @@ dependencies = [ "smallvec", ] +[[package]] +name = "regex-syntax" +version = "0.8.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a" + [[package]] name = "rustc-demangle" version = "0.1.26" @@ -883,6 +991,18 @@ version = "1.0.22" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" +[[package]] +name = "rusty-fork" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc6bf79ff24e648f6da1f8d1f011e9cac26491b619e6b9280f2b47f1774e6ee2" +dependencies = [ + "fnv", + "quick-error", + "tempfile", + "wait-timeout", +] + [[package]] name = "ryu" version = "1.0.20" @@ -1113,6 +1233,12 @@ version = "1.19.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb" +[[package]] +name = "unarray" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eaea85b334db583fe3274d12b4cd1880032beab409c0d774be044d4480ab9a94" + [[package]] name = "unicode-ident" version = "1.0.22" @@ -1147,6 +1273,15 @@ version = "0.9.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" +[[package]] +name = "wait-timeout" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ac3b126d3914f9849036f826e054cbabdc8519970b8998ddaf3b5bd3c65f11" +dependencies = [ + "libc", +] + [[package]] name = "wasi" version = "0.11.1+wasi-snapshot-preview1" @@ -1649,6 +1784,7 @@ dependencies = [ "gimli", "log", "paste", + "proptest", "rayon", "wasm-encoder 0.247.0", "wasmparser 0.247.0", @@ -1683,6 +1819,26 @@ dependencies = [ "wasmparser 0.246.2", ] +[[package]] +name = "zerocopy" +version = "0.8.50" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b065d4f0e55f82fae73202e189638116a87c55ab6b8e6c2721e13dd9d854ad1" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.50" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b631b19d36a892ab55420c92dbc83ccd79274f25be714855d3074aa71cab639" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + [[package]] name = "zstd" version = "0.13.3" diff --git a/Cargo.toml b/Cargo.toml index 14333052..b7dcbb42 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,6 +25,7 @@ wast = "247.0.0" wasmprinter = "0.247.0" wat = "1.247.0" wasmtime = "44.0.0" +proptest = "1" [features] default = [] diff --git a/DWARF-plan.md b/DWARF-plan.md index 4701d2b6..35e3f711 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -46,7 +46,7 @@ Work through these in order. Each box is roughly one PR-sized unit. aranges, frame info — anything that holds a PC. - [x] **7. Re-emit DWARF as custom sections.** Encoded DWARF goes into the output module's custom sections in the conventional order. -- [ ] **8. Unit/regression tests.** Five or six hand-written `.wat` +- [x] **8. Unit/regression tests.** Five or six hand-written `.wat` inputs run through `wasm-tools parse --generate-dwarf full`, with hand-checked expectations. Cover: locals added, nop injected before every op, replacement, block_alt, multi-function @@ -55,15 +55,13 @@ Work through these in order. Each box is roughly one PR-sized unit. prefer one row per opcode, distinct (line, col) per opcode, and either multi-byte injections or enough cumulative shift that no original op stays inside its old line-program range. -- [ ] **9. Differential test.** Two offset-discovery paths +- [x] **9. Differential test.** Two offset-discovery paths (in-encode capture vs. re-parse-after-encode), assert byte-identical maps. Catches off-by-ones in step 2. -- [ ] **10. Property test.** `proptest` over (small dwarf-bearing +- [x] **10. Property test.** `proptest` over (small dwarf-bearing corpus) × (random instrumentation plans). Single invariant: `lookup(new_pc)` in output equals `lookup(anchor_orig_pc)` in input, for every emitted op. -- [ ] **11. Real-binary tests.** Small Rust + C debug builds checked - in, end-to-end DWARF round-trip. - [ ] **12. libfuzzer target (corpus-based).** Reuse the parse-aside seeds from steps 8/11; let libfuzzer generate the instrumentation plan via `Arbitrary`. diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 71193ca9..16280b66 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -12,6 +12,8 @@ use std::path::PathBuf; // Shared with integration tests. See tests/common/wast_iter.rs for the same // pattern — keeps the "validate with all features" choice in one place. +#[path = "../../../tests/common/dwarf.rs"] +mod dwarf_helpers; #[path = "../../../tests/common/validate.rs"] mod validate; @@ -1042,6 +1044,293 @@ fn with_dwarf_captures_per_op_pcs_matching_reparsed_offsets() { } } +// DWARF rewriting, step 10: property test for the source-location invariant. +// +// For every emitted op, `lookup(new_pc)` in the output's `.debug_line` must +// equal `lookup(anchor_orig_pc)` in the input's. That is, the debugger sees +// the same source location for an emitted op as it would for the orig op +// that emit anchors to (whether the emit is the orig op itself, an injected +// before/after, or an alt's first instruction). This is the strong +// correctness invariant for the rewriter. +// +// The proptest generates random before-injection plans (a Vec of nop counts) +// over `add.wasm` and asserts the invariant for every captured emit. Failed +// plans get shrunk automatically. + +use dwarf_helpers::{debug_info_pcs, line_rows, lookup_src_at}; + +/// The strong source-location invariant the rewriter must preserve: for every +/// emitted op `i`, the output line program's source location at +/// `new_first + pcs[i]` equals the input line program's source location at +/// `orig_first + orig_pcs[anchors[i]]`. Encodes the module via +/// `encode_internal`, returns the encoded bytes so callers can run extra +/// sanity checks (e.g. range expansion) without re-encoding. +fn assert_source_location_invariant(input_bytes: &[u8], module: &Module) -> Vec { + let (encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); + let maps = maps.expect("with_dwarf=true captures DWARF maps"); + let out_bytes = encoded.finish(); + + let in_rows = line_rows(input_bytes); + let out_rows = line_rows(&out_bytes); + + let debug = module.debug.as_ref().expect("orig debug data"); + for (func_idx, fmap) in &maps.per_func { + let orig_dbg = debug + .per_func + .get(func_idx) + .expect("orig per-func data for captured function"); + let local = module + .functions + .unwrap_local(FunctionID(*func_idx)) + .expect("local function for captured map"); + let orig_pcs = local + .body + .instructions + .offsets() + .expect("with_dwarf opt-in must populate offsets"); + let orig_first = orig_dbg.first_instr_dwarf_offset as u64; + let new_first = fmap.first_instr_dwarf_offset as u64; + + for (emit_idx, &new_pc) in fmap.pcs.iter().enumerate() { + let anchor = fmap.anchors[emit_idx]; + let orig_pc = orig_pcs[anchor] as u64; + let new_dwarf_addr = new_first + new_pc as u64; + let orig_dwarf_addr = orig_first + orig_pc; + let out_src = lookup_src_at(&out_rows, new_dwarf_addr); + let in_src = lookup_src_at(&in_rows, orig_dwarf_addr); + assert_eq!( + out_src, in_src, + "func {func_idx} emit {emit_idx} (anchor orig {anchor}): \ + src mismatch — orig_addr={orig_dwarf_addr}, new_addr={new_dwarf_addr}", + ); + } + } + + out_bytes +} + +fn dwarf_fixture_path(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/test_inputs/handwritten/dwarf") + .join(name) +} + +// `proptest::strategy::Strategy` provides the `.prop_map` adapter we use in +// the generator's `prop_oneof![]` arms. The macro expands its strategy +// expressions at module scope, so the trait import has to be here, not inside +// the test body. +use proptest::strategy::Strategy as _; + +/// Per-op random instrumentation: one of these is generated for every visited +/// op. `Skip` leaves the op alone; the other variants inject N nops in the +/// corresponding mode. Alt with N=0 is treated as a no-op so we never produce +/// a "replace orig with nothing" plan. +#[derive(Debug, Clone)] +enum DwarfFuzzAction { + Skip, + BeforeNops(u8), + AfterNops(u8), + AltNops(u8), +} + +proptest::proptest! { + /// For every emitted op, the rewritten `.debug_line`'s source location at + /// the new PC must equal the input's source location at the anchor's orig + /// PC. Generates per-op random instrumentation plans (mix of + /// before/after/alt at random counts) over `add.wasm`. + #[test] + fn rewriter_preserves_source_location_under_random_injection( + plan in proptest::collection::vec( + proptest::prop_oneof![ + proptest::strategy::Just(DwarfFuzzAction::Skip), + (0u8..=3).prop_map(DwarfFuzzAction::BeforeNops), + (0u8..=3).prop_map(DwarfFuzzAction::AfterNops), + (1u8..=3).prop_map(DwarfFuzzAction::AltNops), + ], + 0..=8usize, + ) + ) { + use crate::iterator::iterator_trait::{IteratingInstrumenter, Iterator}; + use crate::iterator::module_iterator::ModuleIterator; + use crate::Opcode; + + let input = std::fs::read(dwarf_fixture_path("add.wasm")).expect("read fixture"); + let mut module = Module::parse(&input, false, false, true).expect("parse"); + { + let mut it = ModuleIterator::new(&mut module, &Vec::new()); + let mut op_idx = 0usize; + loop { + if it.curr_op().is_some() { + let action = plan + .get(op_idx) + .cloned() + .unwrap_or(DwarfFuzzAction::Skip); + match action { + DwarfFuzzAction::Skip => {} + DwarfFuzzAction::BeforeNops(n) => { + for _ in 0..n { + it.before().nop(); + } + } + DwarfFuzzAction::AfterNops(n) => { + for _ in 0..n { + it.after().nop(); + } + } + DwarfFuzzAction::AltNops(n) => { + for _ in 0..n { + it.alternate().nop(); + } + } + } + op_idx += 1; + } + if it.next().is_none() { + break; + } + } + } + let _ = assert_source_location_invariant(&input, &module); + } +} + +// Step 8 regression: nop injected before every op. Asserts the strong source- +// location invariant for every emit position, then sanity-checks that +// instrumentation actually took effect (DIE ranges expanded, max addr shifted). +// `lookup(new_pc) == lookup(anchor_orig_pc)` would trivially hold for an +// uninstrumented round-trip, so the post-helper checks confirm the test isn't +// silently a no-op. +#[test] +fn rewriter_anchors_nop_before_every_op_to_host_source_strong() { + use crate::iterator::iterator_trait::{IteratingInstrumenter, Iterator}; + use crate::iterator::module_iterator::ModuleIterator; + use crate::Opcode; + + let input = std::fs::read(dwarf_fixture_path("add.wasm")).expect("read fixture"); + let mut module = Module::parse(&input, false, false, true).expect("parse"); + { + let mut it = ModuleIterator::new(&mut module, &Vec::new()); + loop { + if it.curr_op().is_some() { + it.before().nop(); + } + if it.next().is_none() { + break; + } + } + } + let out = assert_source_location_invariant(&input, &module); + + // Sanity: every DIE range must have expanded to cover the injected bytes. + let in_pcs = debug_info_pcs(&input); + let out_pcs = debug_info_pcs(&out); + assert_eq!(in_pcs.len(), out_pcs.len(), "DIE count must be preserved"); + for ((il, ih), (ol, oh)) in in_pcs.iter().zip(out_pcs.iter()) { + assert!( + *oh - *ol > *ih - *il, + "DIE range must expand for injected bytes ({il}..{ih} → {ol}..{oh})", + ); + } +} + +// Step 8 regression: func_exit injection. Drives the special-mode resolution +// path that fans `func_exit` out to every exit slot (here a single end op). +// Strong source-location invariant + DIE range expansion confirm both the +// fan-out wiring and the address translation behave. +#[test] +fn rewriter_handles_func_exit_injection_strong() { + use crate::opcode::Instrumenter; + use crate::Opcode; + + let input = std::fs::read(dwarf_fixture_path("add.wasm")).expect("read fixture"); + let mut module = Module::parse(&input, false, false, true).expect("parse"); + { + let mut it = + crate::iterator::module_iterator::ModuleIterator::new(&mut module, &Vec::new()); + it.func_exit().nop().nop(); + } + let out = assert_source_location_invariant(&input, &module); + + let in_pcs = debug_info_pcs(&input); + let out_pcs = debug_info_pcs(&out); + assert_eq!(in_pcs.len(), out_pcs.len()); + for ((il, ih), (ol, oh)) in in_pcs.iter().zip(out_pcs.iter()) { + assert!( + *oh - *ol > *ih - *il, + "DIE range must expand for injected exit bytes ({il}..{ih} → {ol}..{oh})", + ); + } +} + +// DWARF rewriting, step 9: differential test for an INSTRUMENTED module. +// Two paths produce per-emit byte offsets — the in-encode capture (each emit +// op's `function.byte_len()` rebased onto the first instruction) and a +// re-parse of the encoded output with `with_offsets=true`. They must agree +// byte-for-byte, including across injected ops, because parsing the output +// observes every byte the encoder emitted. +// +// Injecting a nop before every visited op exercises the cumulative shift the +// step-0 spike findings flagged as the primary source of off-by-one bugs. +#[test] +fn with_dwarf_capture_matches_reparsed_offsets_with_injection() { + use crate::iterator::iterator_trait::{IteratingInstrumenter, Iterator}; + use crate::iterator::module_iterator::ModuleIterator; + use crate::Opcode; + + let wat = r#"(module + (func (result i32) i32.const 1 i32.const 2 i32.add) + (func (local i32) (local i64) + i32.const 0 + local.set 0 + i64.const 0 + local.set 1))"#; + let wasm = wat::parse_str(wat).expect("wat compiles"); + let mut module = Module::parse(&wasm, false, false, true).expect("parse"); + { + // Inject a nop before every visited op across every local function. + let mut it = ModuleIterator::new(&mut module, &Vec::new()); + loop { + if it.curr_op().is_some() { + it.before().nop(); + } + if it.next().is_none() { + break; + } + } + } + + let (encoded, _side_effects, maps) = module.encode_internal(false).expect("encode"); + let maps = maps.expect("with_dwarf=true should capture DWARF encode maps"); + let out = encoded.finish(); + let reparsed = Module::parse(&out, false, true, false).expect("reparse output"); + + assert!( + !maps.per_func.is_empty(), + "expected per-function maps for the instrumented test wat", + ); + for (func_idx, captured) in &maps.per_func { + let local = reparsed + .functions + .unwrap_local(FunctionID(*func_idx)) + .expect("local function in re-parsed output"); + for (i, pc) in captured.pcs.iter().enumerate() { + assert_eq!( + Some(*pc), + local.lookup_pc_offset_for(i), + "func {func_idx} emit {i}: in-encode capture vs re-parse mismatch", + ); + } + // Both paths must cover the same number of ops (no missing or extra + // emits on either side). `lookup_pc_offset_for(captured.len())` past + // the end returns `None` in re-parsed output. + assert_eq!( + local.lookup_pc_offset_for(captured.pcs.len()), + None, + "func {func_idx}: re-parsed output has more ops than captured", + ); + } +} + // DWARF rewriting, step 3: encode records the byte offset where the code // section begins in the output, so the rewriter can translate per-function // PCs into module-absolute addresses. The captured offset must point at the diff --git a/tests/common/dwarf.rs b/tests/common/dwarf.rs new file mode 100644 index 00000000..b883d4e1 --- /dev/null +++ b/tests/common/dwarf.rs @@ -0,0 +1,138 @@ +//! Shared DWARF inspection helpers for the rewriter tests. +//! +//! Loaded both by the integration tests in `tests/dwarf.rs` (via the normal +//! `mod common::dwarf` path) and by the in-crate unit tests in +//! `src/ir/module/test.rs` (via `#[path]` include). Keeping the two sides on +//! the same parse path avoids drift on details like `address_size` and the +//! sort-order assumption baked into `lookup_src_at`. + +#![allow(dead_code)] + +/// Parse a wasm's `.debug_line` and return `(address, line, column)` for each +/// non-end-of-sequence row, in the order gimli emits them. +/// +/// Assumes a single line program per `.debug_line` section (the rewriter +/// refuses multi-program inputs, so the helper does too). `address_size = 4` +/// is the wasm DWARF convention. +pub fn line_rows(wasm: &[u8]) -> Vec<(u64, u64, u64)> { + use wasmparser::{Parser, Payload}; + let mut line_bytes: Vec = Vec::new(); + for payload in Parser::new(0).parse_all(wasm) { + if let Payload::CustomSection(cs) = payload.expect("valid wasm") { + if cs.name() == ".debug_line" { + line_bytes = cs.data().to_vec(); + break; + } + } + } + let endian = gimli::LittleEndian; + let dl = gimli::read::DebugLine::new(&line_bytes, endian); + let program = dl + .program(gimli::DebugLineOffset(0), 4, None, None) + .expect("line program parses"); + let mut rows = program.rows(); + let mut out = Vec::new(); + while let Some((_header, row)) = rows.next_row().expect("row reads") { + if row.end_sequence() { + continue; + } + let line = row.line().map(|n| n.get()).unwrap_or(0); + let column = match row.column() { + gimli::ColumnType::LeftEdge => 0, + gimli::ColumnType::Column(c) => c.get(), + }; + out.push((row.address(), line, column)); + } + out +} + +/// Source-location lookup: find the row whose address is the largest +/// `≤ addr`, return its `(line, column)`. +/// +/// Requires `rows` to be sorted ascending by address (the rewriter's +/// invariant on single-program inputs). Asserts in debug builds. +pub fn lookup_src_at(rows: &[(u64, u64, u64)], addr: u64) -> Option<(u64, u64)> { + debug_assert!( + rows.windows(2).all(|w| w[0].0 <= w[1].0), + "lookup_src_at expects rows sorted by address; got {rows:?}", + ); + rows.iter() + .rev() + .find(|(a, _, _)| *a <= addr) + .map(|(_, l, c)| (*l, *c)) +} + +/// Walks `.debug_info` and collects each DIE's `(low_pc, high_pc)` pair. +/// Handles both `DW_FORM_addr` (absolute high_pc) and `DW_FORM_data*` / +/// `DW_FORM_udata` (high_pc as length, per the DWARF spec). +pub fn debug_info_pcs(wasm: &[u8]) -> Vec<(u64, u64)> { + use std::collections::BTreeMap; + use wasmparser::{Parser, Payload}; + let mut sections: BTreeMap<&'static str, Vec> = BTreeMap::new(); + let known: &[&'static str] = &[ + ".debug_info", + ".debug_abbrev", + ".debug_str", + ".debug_line", + ".debug_line_str", + ]; + for payload in Parser::new(0).parse_all(wasm) { + if let Payload::CustomSection(cs) = payload.expect("valid wasm") { + for name in known { + if cs.name() == *name { + sections.insert(name, cs.data().to_vec()); + } + } + } + } + let endian = gimli::LittleEndian; + let lookup = |name: &str| -> gimli::EndianSlice<'_, gimli::LittleEndian> { + gimli::EndianSlice::new( + sections.get(name).map(|v| v.as_slice()).unwrap_or(&[]), + endian, + ) + }; + let dwarf = gimli::read::Dwarf::load(|id| -> Result<_, gimli::Error> { + Ok(match id { + gimli::SectionId::DebugInfo => lookup(".debug_info"), + gimli::SectionId::DebugAbbrev => lookup(".debug_abbrev"), + gimli::SectionId::DebugStr => lookup(".debug_str"), + gimli::SectionId::DebugLine => lookup(".debug_line"), + gimli::SectionId::DebugLineStr => lookup(".debug_line_str"), + _ => gimli::EndianSlice::new(&[], endian), + }) + }) + .expect("load DWARF"); + + let read_uint = |v: gimli::read::AttributeValue<_>| -> Option { + match v { + gimli::read::AttributeValue::Addr(a) => Some(a), + gimli::read::AttributeValue::Data1(d) => Some(d as u64), + gimli::read::AttributeValue::Data2(d) => Some(d as u64), + gimli::read::AttributeValue::Data4(d) => Some(d as u64), + gimli::read::AttributeValue::Data8(d) => Some(d), + gimli::read::AttributeValue::Udata(d) => Some(d), + _ => None, + } + }; + + let mut out = Vec::new(); + let mut units = dwarf.units(); + while let Some(header) = units.next().expect("unit header") { + let unit = dwarf.unit(header).expect("unit"); + let mut entries = unit.entries(); + while let Some(entry) = entries.next_dfs().expect("dfs") { + let low = entry.attr_value(gimli::DW_AT_low_pc).and_then(read_uint); + let high_raw = entry.attr_value(gimli::DW_AT_high_pc).and_then(read_uint); + let high = match entry.attr_value(gimli::DW_AT_high_pc) { + Some(gimli::read::AttributeValue::Addr(_)) => high_raw, + Some(_) => high_raw.zip(low).map(|(l_len, l_addr)| l_addr + l_len), + None => None, + }; + if let (Some(l), Some(h)) = (low, high) { + out.push((l, h)); + } + } + } + out +} diff --git a/tests/common/mod.rs b/tests/common/mod.rs index 91fa537c..b434e728 100644 --- a/tests/common/mod.rs +++ b/tests/common/mod.rs @@ -8,6 +8,7 @@ use std::io::{BufRead, BufReader}; use std::path::Path; use wasmparser::Operator; +pub mod dwarf; pub mod validate; pub mod wast_iter; use wirm::ir::types::InstrumentationMode; diff --git a/tests/dwarf.rs b/tests/dwarf.rs index 731f6b00..f1e88d49 100644 --- a/tests/dwarf.rs +++ b/tests/dwarf.rs @@ -13,12 +13,22 @@ use std::path::PathBuf; use wirm::ir::module::Module; +#[path = "common/dwarf.rs"] +mod dwarf_helpers; + +use dwarf_helpers::line_rows as debug_line_rows; + const DWARF_SECTION_NAMES: &[&str] = &[".debug_abbrev", ".debug_str", ".debug_line", ".debug_info"]; fn input_path() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/test_inputs/handwritten/dwarf/add.wasm") } +fn multi_func_input_path() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests/test_inputs/handwritten/dwarf/two_funcs.wasm") +} + /// Pulls every `.debug_*` custom section's bytes out, keyed by name. /// `BTreeMap` so equality is order-independent — encode may legitimately /// place DWARF sections in a different position relative to other custom @@ -35,94 +45,7 @@ fn debug_section_bytes(wasm: &[u8]) -> BTreeMap> { out } -/// Walks `.debug_info` and collects each DIE's `(low_pc, high_pc)` pair. Used -/// for semantic comparison since the rewriter may emit different bytes (e.g. -/// different abbreviation codes) but must preserve address ranges. -fn debug_info_pcs(wasm: &[u8]) -> Vec<(u64, u64)> { - let sections = debug_section_bytes(wasm); - let endian = gimli::LittleEndian; - let lookup = |name: &str| -> gimli::EndianSlice<'_, gimli::LittleEndian> { - gimli::EndianSlice::new( - sections.get(name).map(|v| v.as_slice()).unwrap_or(&[]), - endian, - ) - }; - let dwarf = gimli::read::Dwarf::load(|id| -> Result<_, gimli::Error> { - Ok(match id { - gimli::SectionId::DebugInfo => lookup(".debug_info"), - gimli::SectionId::DebugAbbrev => lookup(".debug_abbrev"), - gimli::SectionId::DebugStr => lookup(".debug_str"), - gimli::SectionId::DebugLine => lookup(".debug_line"), - gimli::SectionId::DebugLineStr => lookup(".debug_line_str"), - _ => gimli::EndianSlice::new(&[], endian), - }) - }) - .expect("load DWARF"); - - let mut out = Vec::new(); - let mut units = dwarf.units(); - while let Some(header) = units.next().expect("unit header") { - let unit = dwarf.unit(header).expect("unit"); - let mut entries = unit.entries(); - while let Some(entry) = entries.next_dfs().expect("dfs") { - // wasm-tools' CU stores low_pc/high_pc as DW_FORM_data4 (low_pc = - // absolute, high_pc = length per the DWARF spec); the subprogram - // uses DW_FORM_addr for low_pc and data4 for high_pc. Accept both. - let read_uint = |v: gimli::read::AttributeValue<_>| -> Option { - match v { - gimli::read::AttributeValue::Addr(a) => Some(a), - gimli::read::AttributeValue::Data1(d) => Some(d as u64), - gimli::read::AttributeValue::Data2(d) => Some(d as u64), - gimli::read::AttributeValue::Data4(d) => Some(d as u64), - gimli::read::AttributeValue::Data8(d) => Some(d), - gimli::read::AttributeValue::Udata(d) => Some(d), - _ => None, - } - }; - let low = entry.attr_value(gimli::DW_AT_low_pc).and_then(read_uint); - let high_raw = entry.attr_value(gimli::DW_AT_high_pc).and_then(read_uint); - // For Addr form high_pc is absolute; for Data*/Udata it's a length. - let high = match entry.attr_value(gimli::DW_AT_high_pc) { - Some(gimli::read::AttributeValue::Addr(_)) => high_raw, - Some(_) => high_raw.zip(low).map(|(l_len, l_addr)| l_addr + l_len), - None => None, - }; - if let (Some(l), Some(h)) = (low, high) { - out.push((l, h)); - } - } - } - out -} - -/// Parses the input's `.debug_line` via gimli and collects each non-end-of- -/// sequence row as `(address, line, column)`. Used for semantic comparison -/// between input and rewritten outputs. -fn debug_line_rows(wasm: &[u8]) -> Vec<(u64, u64, u64)> { - let dl_bytes = debug_section_bytes(wasm) - .remove(".debug_line") - .expect("input has .debug_line"); - let endian = gimli::LittleEndian; - let slice = gimli::EndianSlice::new(&dl_bytes, endian); - let dl = gimli::read::DebugLine::new(slice.slice(), endian); - let program = dl - .program(gimli::DebugLineOffset(0), 4, None, None) - .expect("line program parses"); - let mut rows = program.rows(); - let mut out = Vec::new(); - while let Some((_header, row)) = rows.next_row().expect("row reads") { - if row.end_sequence() { - continue; - } - let line = row.line().map(|n| n.get()).unwrap_or(0); - let column = match row.column() { - gimli::ColumnType::LeftEdge => 0, - gimli::ColumnType::Column(c) => c.get(), - }; - out.push((row.address(), line, column)); - } - out -} +use dwarf_helpers::debug_info_pcs; /// `with_dwarf=false` leaves DWARF flowing through `custom_sections`. The /// section bytes themselves still round-trip verbatim, since `custom_sections` @@ -277,6 +200,11 @@ fn rewriter_handles_alternate_replacement() { ); } +// Note: the heavy-injection regression (nop before every op) was moved to +// `src/ir/module/test.rs::rewriter_anchors_nop_before_every_op_to_host_source_strong` +// where it can apply the full `lookup(new_pc) == lookup(anchor_orig_pc)` +// invariant via `pub(crate) DwarfEncodeMaps` access. + /// Rewritten DWARF lands in the output module's custom sections in /// input order — re-emit is order-preserving so downstream tooling that /// scans `.debug_*` payloads doesn't have to deal with reshuffled section @@ -313,6 +241,71 @@ fn rewriter_preserves_dwarf_section_order_in_output() { ); } +/// Adding a local declaration grows the locals-vec encoding, shifting the +/// first-instruction DWARF offset. The translator must map orig addresses +/// for instruction bytes onto the shifted positions and preserve the +/// boundary addresses (`addr == size_leb_len → size_leb_len`, +/// `addr == first_instr_dwarf_offset → first_instr_dwarf_offset_new`). +#[test] +fn rewriter_handles_added_local() { + use wirm::module_builder::AddLocal; + use wirm::DataType; + + let input = std::fs::read(input_path()).unwrap(); + let mut module = Module::parse(&input, false, false, true).unwrap(); + { + let mut it = wirm::iterator::module_iterator::ModuleIterator::new(&mut module, &Vec::new()); + // Cursor is on the first op; add a local to the current function. + let _local = it.add_local(DataType::I32); + } + + let output = module.encode().unwrap(); + + // Body content grew by 2 bytes (1 run-count + 1 type byte for the i32), + // so DIE ranges expand by exactly 2 and low_pc boundaries are preserved. + // Input ranges: (0, 8) and (1, 8). Expected output: (0, 10) and (1, 10). + let out_pcs = debug_info_pcs(&output); + assert_eq!(out_pcs, vec![(0, 10), (1, 10)]); + + // Line-program rows must shift right by the same 2 bytes (the locals + // grew but instruction encodings didn't change). + let in_rows = debug_line_rows(&input); + let out_rows = debug_line_rows(&output); + assert_eq!(in_rows.len(), out_rows.len()); + for ((ia, il, ic), (oa, ol, oc)) in in_rows.iter().zip(out_rows.iter()) { + assert_eq!(*oa, *ia + 2, "row address must shift by added locals bytes"); + assert_eq!((*il, *ic), (*ol, *oc), "(line, col) must be preserved"); + } +} + +/// Multi-function inputs currently exercise the rewriter's defensive gate: +/// per-function DWARF address spaces overlap when each starts at 0, so step +/// 6 refuses to translate until per-CU routing lands. This test pins the +/// error so a future change that softens the gate is intentional. +#[test] +fn rewriter_refuses_multi_function_input() { + let input = std::fs::read(multi_func_input_path()).unwrap(); + let module = Module::parse(&input, false, false, true).unwrap(); + let err = module + .encode() + .expect_err("multi-function .debug_info rewriting should refuse"); + // Structural match: the gate must produce a `DwarfError`, and the + // message must reference multi-function (so the test fails loudly if a + // future refactor produces the right variant but the wrong reason). + let msg = match &err { + wirm::error::Error::DwarfError(m) => m, + other => panic!("expected DwarfError variant, got {other:?}"), + }; + assert!( + msg.contains("multi-function"), + "DwarfError message should mention multi-function, got: {msg}", + ); +} + +// Note: the `func_exit` injection test was moved to +// `src/ir/module/test.rs::rewriter_handles_func_exit_injection_strong` so it +// can apply the full `lookup(new_pc) == lookup(anchor_orig_pc)` invariant. + /// Explicit address-translation invariant for the rewriter: for an /// uninstrumented module the rewritten rows must use the same addresses as /// the input, because new layout equals orig layout. diff --git a/tests/test_inputs/handwritten/dwarf/README.md b/tests/test_inputs/handwritten/dwarf/README.md new file mode 100644 index 00000000..57fa9e37 --- /dev/null +++ b/tests/test_inputs/handwritten/dwarf/README.md @@ -0,0 +1,33 @@ +# DWARF test fixtures + +Wasm modules carrying hand-authored DWARF debug info, used by the rewriter +tests in `tests/dwarf.rs` and the proptest in `src/ir/module/test.rs`. + +## Regenerating + +The `.wasm` files are produced from their sibling `.wat` files by +`wasm-tools parse --generate-dwarf full`, which synthesizes line-number +information from the `.wat`'s source positions. Re-run the command after +editing a `.wat` so the bundled `.wasm` stays in sync — otherwise tests will +either fail in surprising ways or, worse, keep passing against stale +expectations. + +```sh +# From this directory: +wasm-tools parse --generate-dwarf full add.wat -o add.wasm +wasm-tools parse --generate-dwarf full two_funcs.wat -o two_funcs.wasm +``` + +`wasm-tools 1.247.0` was used to produce the currently checked-in `.wasm` +files; other versions are likely fine but the byte layout of the emitted +DWARF may shift in ways that move pinned addresses in the tests. + +## Inspecting + +`llvm-dwarfdump` is the quickest way to confirm the line program / DIE +addresses match what the tests assert: + +```sh +llvm-dwarfdump --debug-line add.wasm +llvm-dwarfdump --debug-info add.wasm +``` diff --git a/tests/test_inputs/handwritten/dwarf/two_funcs.wasm b/tests/test_inputs/handwritten/dwarf/two_funcs.wasm new file mode 100644 index 0000000000000000000000000000000000000000..62e46b4001f80c3213ce4d61d3f5a4c3aa0a4243 GIT binary patch literal 441 zcmZut%TB^j5Ir;Z-hwo=6c%b+K#as)A!uUMMB~!<0WPt{LNK*WXtBCt>1X(bet{eB z@Nnf_+%t3L%o9=ZN&v{ACtA@;DF$jyR?|EOR2(fpjeu+D;f`M1HFD~rtF!1eA(tUa z5+PfMK<{L~4?Hx(=OkXv#!(zEk~a#dU+?(JJMsdj?X>m*K7sOF_r|wO4;B1Bo`0l)cgU tMK}zH;7wr(0Idyx#n5JRTh`LECQO=7qI3xT4SWqmE9bN^Ed%_oz%So3J8b{} literal 0 HcmV?d00001 diff --git a/tests/test_inputs/handwritten/dwarf/two_funcs.wat b/tests/test_inputs/handwritten/dwarf/two_funcs.wat new file mode 100644 index 00000000..50be106b --- /dev/null +++ b/tests/test_inputs/handwritten/dwarf/two_funcs.wat @@ -0,0 +1,14 @@ +(module + (func $foo (param $x i32) (result i32) + local.get $x + i32.const 1 + i32.add + ) + (func $bar (param $y i32) (result i32) + local.get $y + i32.const 2 + i32.mul + ) + (export "foo" (func $foo)) + (export "bar" (func $bar)) +) From e5ae79f696ce354736def0d73166d4462ba28501 Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Fri, 5 Jun 2026 08:59:00 -0400 Subject: [PATCH 11/13] add fuzz testing --- .github/workflows/fuzz.yml | 1 + fuzz/Cargo.lock | 37 +++++- fuzz/Cargo.toml | 11 ++ fuzz/README.md | 17 +-- fuzz/fuzz_targets/dwarf_invariant.rs | 161 +++++++++++++++++++++++++++ 5 files changed, 217 insertions(+), 10 deletions(-) create mode 100644 fuzz/fuzz_targets/dwarf_invariant.rs diff --git a/.github/workflows/fuzz.yml b/.github/workflows/fuzz.yml index 368c14dc..071ab45a 100644 --- a/.github/workflows/fuzz.yml +++ b/.github/workflows/fuzz.yml @@ -29,6 +29,7 @@ jobs: - component_instrument - component_concretize - component_walks + - dwarf_invariant steps: - uses: actions/checkout@v4 diff --git a/fuzz/Cargo.lock b/fuzz/Cargo.lock index 725986b1..8f274b86 100644 --- a/fuzz/Cargo.lock +++ b/fuzz/Cargo.lock @@ -70,6 +70,12 @@ version = "0.4.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b7ac824320a75a52197e8f2d787f6a38b6718bb6897a35142d749af3c0e8f4fe" +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + [[package]] name = "foldhash" version = "0.2.0" @@ -88,6 +94,24 @@ dependencies = [ "wasip2", ] +[[package]] +name = "gimli" +version = "0.33.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bf7f043f89559805f8c7cacc432749b2fa0d0a0a9ee46ce47164ed5ba7f126c" +dependencies = [ + "fnv", + "hashbrown 0.16.1", + "indexmap", + "stable_deref_trait", +] + +[[package]] +name = "hashbrown" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" + [[package]] name = "hashbrown" version = "0.17.0" @@ -106,7 +130,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" dependencies = [ "equivalent", - "hashbrown", + "hashbrown 0.17.0", "serde", "serde_core", ] @@ -220,6 +244,12 @@ version = "1.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + [[package]] name = "syn" version = "2.0.117" @@ -275,7 +305,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8e6fb4c2bee46c5ea4d40f8cdb5c131725cd976718ec56f1c8e82fbde5fa2a80" dependencies = [ "bitflags", - "hashbrown", + "hashbrown 0.17.0", "indexmap", "semver", "serde", @@ -285,6 +315,7 @@ dependencies = [ name = "wirm" version = "5.0.0" dependencies = [ + "gimli", "log", "paste", "wasm-encoder", @@ -295,6 +326,8 @@ dependencies = [ name = "wirm-fuzz" version = "0.0.0" dependencies = [ + "arbitrary", + "gimli", "libfuzzer-sys", "wasm-encoder", "wasm-smith", diff --git a/fuzz/Cargo.toml b/fuzz/Cargo.toml index fa5301aa..c31c8e91 100644 --- a/fuzz/Cargo.toml +++ b/fuzz/Cargo.toml @@ -8,12 +8,16 @@ edition = "2021" cargo-fuzz = true [dependencies] +arbitrary = { version = "1", features = ["derive"] } libfuzzer-sys = "0.4" # wasm-smith releases in lockstep with wasmparser/wasm-encoder. When # bumping wirm's wasmparser dep, bump this to match. See fuzz/DECISIONS.md. wasm-smith = { version = "0.247.0", features = ["component-model"] } wasm-encoder = "0.247.0" wasmparser = "0.247.0" +# Used by `dwarf_invariant` to re-parse the encoded `.debug_line`. Tracks +# the wirm-level pin in the parent crate. +gimli = "0.33" [dependencies.wirm] path = ".." @@ -59,3 +63,10 @@ path = "fuzz_targets/component_walks.rs" test = false doc = false bench = false + +[[bin]] +name = "dwarf_invariant" +path = "fuzz_targets/dwarf_invariant.rs" +test = false +doc = false +bench = false diff --git a/fuzz/README.md b/fuzz/README.md index aca4ff48..c5d995f8 100644 --- a/fuzz/README.md +++ b/fuzz/README.md @@ -36,14 +36,15 @@ cargo +nightly fuzz run module_roundtrip -- -max_total_time=300 ## Current targets -| Target | Exercises | -|-------------------------|------------------------------------------------------------------------------------------------------------| -| `module_roundtrip` | `Module::parse` → `Module::encode` → `wasmparser::Validator` | -| `module_instrument` | `module_roundtrip` + iterate and inject `nop` before every op | -| `component_roundtrip` | `Component::parse` → `Component::encode` → `wasmparser::Validator` | -| `component_instrument` | `component_roundtrip` + component-level forward-ref grafting onto an existing core/component `Instantiate` | -| `component_concretize` | `Component::concretize_import` / `concretize_export` on every named import/export | -| `component_walks` | `walk_structural` ≡ `walk_topological`; root `section_idx` in range + monotonic vs. wasmparser | +| Target | Exercises | +|------------------------|------------------------------------------------------------------------------------------------------------| +| `module_roundtrip` | `Module::parse` → `Module::encode` → `wasmparser::Validator` | +| `module_instrument` | `module_roundtrip` + iterate and inject `nop` before every op | +| `component_roundtrip` | `Component::parse` → `Component::encode` → `wasmparser::Validator` | +| `component_instrument` | `component_roundtrip` + component-level forward-ref grafting onto an existing core/component `Instantiate` | +| `component_concretize` | `Component::concretize_import` / `concretize_export` on every named import/export | +| `component_walks` | `walk_structural` ≡ `walk_topological`; root `section_idx` in range + monotonic vs. wasmparser | +| `dwarf_invariant` | DWARF rewriter for `Arbitrary`-generated instrumentation plans; weak validity + `(line, col)` subset check | More targets planned — see `DECISIONS.md`. diff --git a/fuzz/fuzz_targets/dwarf_invariant.rs b/fuzz/fuzz_targets/dwarf_invariant.rs new file mode 100644 index 00000000..d4b653ea --- /dev/null +++ b/fuzz/fuzz_targets/dwarf_invariant.rs @@ -0,0 +1,161 @@ +//! Coverage-guided fuzz of the DWARF rewriter against `add.wasm`. +//! +//! Companion to the in-crate proptest at +//! `src/ir/module/test.rs::rewriter_preserves_source_location_under_random_injection`. +//! The proptest does the *strong* invariant check +//! (`lookup(new_pc) == lookup(anchor_orig_pc)`) on 256 random plans per +//! `cargo test`; that check needs `pub(crate)` access to `DwarfEncodeMaps` +//! which the fuzz crate can't reach across the crate boundary. +//! +//! This target trades the strong check for *broader* exploration. It pins +//! `add.wasm` as the corpus seed and lets libfuzzer generate random +//! instrumentation plans via `Arbitrary`. Each plan is applied, the module +//! is re-encoded, and the output is checked against three weaker invariants +//! that still catch real regressions: +//! +//! 1. The encoded output validates under all wasmparser features. +//! 2. Gimli successfully re-parses the rewritten `.debug_line`. +//! 3. Every `(line, column)` pair the input carried still appears in the +//! output's line program (no source mapping was silently dropped). +//! +//! Plans that exercise paths the proptest hasn't seen accumulate in the +//! libfuzzer corpus and stay around across runs. + +#![no_main] + +use arbitrary::Arbitrary; +use libfuzzer_sys::fuzz_target; +use std::collections::BTreeSet; +use wirm::iterator::iterator_trait::{IteratingInstrumenter, Iterator}; +use wirm::iterator::module_iterator::ModuleIterator; +use wirm::Opcode; + +/// Per-op instrumentation action. `BeforeNops` / `AfterNops` accept zero (a +/// no-op); `AltNops(0)` would replace the orig op with nothing, which the +/// applier promotes to one nop so we never produce an empty alternate. +#[derive(Debug, Clone, Arbitrary)] +enum Action { + Skip, + BeforeNops(u8), + AfterNops(u8), + AltNops(u8), +} + +#[derive(Debug, Arbitrary)] +struct Plan { + actions: Vec, +} + +const FIXTURE: &[u8] = + include_bytes!("../../tests/test_inputs/handwritten/dwarf/add.wasm"); + +fuzz_target!(|plan: Plan| { + let mut module = match wirm::Module::parse(FIXTURE, false, false, true) { + Ok(m) => m, + // Fixture is committed and known-parseable, but keep this defensive + // in case a future wasmparser bump rejects it. + Err(_) => return, + }; + + { + let mut it = ModuleIterator::new(&mut module, &vec![]); + let mut idx = 0usize; + loop { + if it.curr_op().is_some() { + let action = plan.actions.get(idx).cloned().unwrap_or(Action::Skip); + apply_action(&mut it, action); + idx += 1; + } + if it.next().is_none() { + break; + } + } + } + + let encoded = module + .encode() + .expect("encode of instrumented add.wasm should succeed"); + + wasmparser::Validator::new_with_features(wasmparser::WasmFeatures::all()) + .validate_all(&encoded) + .expect("rewritten DWARF module failed wasmparser validation"); + + let in_locs = collect_line_col_pairs(FIXTURE); + let out_locs = collect_line_col_pairs(&encoded); + + for loc in &in_locs { + assert!( + out_locs.contains(loc), + "input (line, col) {loc:?} missing from output's .debug_line \ + after instrumentation", + ); + } +}); + +/// Apply a single action. Clamps counts to a small upper bound so a wildly +/// large plan doesn't blow up the body size (we want lots of distinct cases +/// per second, not one giant case per minute). +fn apply_action(it: &mut ModuleIterator<'_, '_>, action: Action) { + const MAX_PER_OP: u8 = 8; + match action { + Action::Skip => {} + Action::BeforeNops(n) => { + for _ in 0..n.min(MAX_PER_OP) { + it.before().nop(); + } + } + Action::AfterNops(n) => { + for _ in 0..n.min(MAX_PER_OP) { + it.after().nop(); + } + } + Action::AltNops(n) => { + // An empty alternate would delete the orig op; promote 0 → 1. + let n = n.max(1).min(MAX_PER_OP); + for _ in 0..n { + it.alternate().nop(); + } + } + } +} + +/// Pulls `.debug_line` out of `wasm`, parses it with gimli, and returns the +/// set of `(line, column)` pairs from non-end-of-sequence rows. Used to +/// check the weak "no source mapping dropped" invariant. Errors during +/// parse return an empty set, which would cause the input-subset check to +/// fail loudly for any non-trivial input. +fn collect_line_col_pairs(wasm: &[u8]) -> BTreeSet<(u64, u64)> { + use wasmparser::{Parser, Payload}; + + let mut line_bytes: Vec = Vec::new(); + for payload in Parser::new(0).parse_all(wasm) { + if let Ok(Payload::CustomSection(cs)) = payload { + if cs.name() == ".debug_line" { + line_bytes = cs.data().to_vec(); + break; + } + } + } + + let endian = gimli::LittleEndian; + let dl = gimli::read::DebugLine::new(&line_bytes, endian); + let program = match dl.program(gimli::DebugLineOffset(0), 4, None, None) { + Ok(p) => p, + Err(_) => return BTreeSet::new(), + }; + + let mut rows = program.rows(); + let mut out = BTreeSet::new(); + while let Ok(Some((_header, row))) = rows.next_row() { + if row.end_sequence() { + continue; + } + let line = row.line().map(|n| n.get()).unwrap_or(0); + let col = match row.column() { + gimli::ColumnType::LeftEdge => 0, + gimli::ColumnType::Column(c) => c.get(), + }; + out.insert((line, col)); + } + out +} From 10badbe01b3c0ada563e0a02c2f17d5d80dbdcc9 Mon Sep 17 00:00:00 2001 From: Elizabeth Gilbert Date: Fri, 5 Jun 2026 09:16:13 -0400 Subject: [PATCH 12/13] warn on unsupported section types --- DWARF-plan.md | 12 +++-- src/ir/module/mod.rs | 21 ++++++++ src/ir/module/test.rs | 121 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 151 insertions(+), 3 deletions(-) diff --git a/DWARF-plan.md b/DWARF-plan.md index 35e3f711..fd45ee6d 100644 --- a/DWARF-plan.md +++ b/DWARF-plan.md @@ -62,14 +62,20 @@ Work through these in order. Each box is roughly one PR-sized unit. corpus) × (random instrumentation plans). Single invariant: `lookup(new_pc)` in output equals `lookup(anchor_orig_pc)` in input, for every emitted op. -- [ ] **12. libfuzzer target (corpus-based).** Reuse the parse-aside +- [x] **12. libfuzzer target (corpus-based).** Reuse the parse-aside seeds from steps 8/11; let libfuzzer generate the instrumentation plan via `Arbitrary`. - [ ] **13. libfuzzer target (synthesized DWARF).** Helper that emits a stub `.debug_line` on top of `wasm-smith` output so we get unbounded variation. Optional, deferred until earlier layers - are stable. -- [ ] **14. Warn on adjacent debug sections.** When DWARF rewriting is + are stable. *Deferred alongside step 11: wasm-smith outputs are + multi-function, which step 6 refuses, so most cases would hit + the refusal path. Interesting features (loc lists, ranges, + inlined subroutines) are blocked behind multi-function support. + Action-space variation is already covered by step 10 proptest + + step 12 libfuzzer. Revisit alongside step 11 once multi-CU + routing lands.* +- [x] **14. Warn on adjacent debug sections.** When DWARF rewriting is opted in, detect `external_debug_info` and `sourceMappingURL` custom sections during parse and emit a `log::warn!` (the `log` crate is already a dep). The user opted into DWARF rewriting, diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 525031cb..36af7afa 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -483,6 +483,27 @@ impl<'a> Module<'a> { } Payload::CustomSection(custom_section_reader) => { let cs_name = custom_section_reader.name(); + if with_dwarf { + // Warn on rewriting gaps. Both of these are truly not wirm's + // responsibility. + match cs_name { + // wirm can't pull debug info from a URL and rewrite it... + "external_debug_info" => warn!( + "DWARF rewriting opted in but module also carries an \ + `external_debug_info` custom section. Its referenced \ + side-file's addresses go stale after instrumentation \ + and wirm does not rewrite it.", + ), + // totally different format (JSON) + "sourceMappingURL" => warn!( + "DWARF rewriting opted in but module also carries a \ + `sourceMappingURL` custom section. The referenced \ + source map's byte offsets go stale after \ + instrumentation and wirm does not rewrite it.", + ), + _ => {} + } + } if let Some(debug) = debug_sections.as_mut() { if ModuleDebugData::is_dwarf_section_name(cs_name) { debug.push(CustomSection { diff --git a/src/ir/module/test.rs b/src/ir/module/test.rs index 16280b66..81bab6a0 100644 --- a/src/ir/module/test.rs +++ b/src/ir/module/test.rs @@ -1262,6 +1262,127 @@ fn rewriter_handles_func_exit_injection_strong() { } } +// Step 14: when `with_dwarf` is on and the module also carries an adjacent +// debug section (`external_debug_info` or `sourceMappingURL`), parsing must +// emit a `log::warn!` so the user knows their adjacent debug-info goes stale +// after instrumentation — wirm passes those bytes through unchanged. + +/// One-shot global logger that captures every log record into a shared +/// `Vec`. The first test to install it wins; subsequent tests reuse +/// the same buffer and snapshot-diff to isolate their own records. +mod warn_capture { + use std::sync::{Mutex, OnceLock}; + + static BUFFER: OnceLock>> = OnceLock::new(); + static INSTALLED: OnceLock<()> = OnceLock::new(); + static LOGGER: CaptureLogger = CaptureLogger; + + struct CaptureLogger; + impl log::Log for CaptureLogger { + fn enabled(&self, _: &log::Metadata) -> bool { + true + } + fn log(&self, record: &log::Record) { + if let Some(buf) = BUFFER.get() { + if let Ok(mut v) = buf.lock() { + v.push(format!("{}: {}", record.level(), record.args())); + } + } + } + fn flush(&self) {} + } + + pub fn install() -> &'static Mutex> { + let buf = BUFFER.get_or_init(|| Mutex::new(Vec::new())); + INSTALLED.get_or_init(|| { + // Best-effort: if some other test in the same process already set + // a logger we silently fall back to whatever it's doing (the + // buffer just stays empty for our tests). + let _ = log::set_logger(&LOGGER); + log::set_max_level(log::LevelFilter::Warn); + }); + buf + } + + /// Snapshot the current buffer length so a test can read just the + /// records it produced. + pub fn snapshot(buf: &Mutex>) -> usize { + buf.lock().map(|v| v.len()).unwrap_or(0) + } + + pub fn records_since(buf: &Mutex>, start: usize) -> Vec { + buf.lock().map(|v| v[start..].to_vec()).unwrap_or_default() + } +} + +fn module_with_custom_section(name: &str, data: &[u8]) -> Vec { + let mut m = wasm_encoder::Module::new(); + m.section(&wasm_encoder::CustomSection { + name: std::borrow::Cow::Borrowed(name), + data: std::borrow::Cow::Borrowed(data), + }); + m.finish() +} + +#[test] +fn with_dwarf_warns_on_external_debug_info_section() { + let buf = warn_capture::install(); + let start = warn_capture::snapshot(buf); + + let bytes = module_with_custom_section("external_debug_info", b"https://example/dwarf"); + let module = Module::parse(&bytes, false, false, true).expect("parse"); + // The section passes through opaquely — it's NOT diverted into + // Module::debug because it isn't a `.debug_*` section. + assert!( + module + .debug + .as_ref() + .is_some_and(|d| d.sections().is_empty()), + "Module::debug should be Some(empty); adjacent debug sections \ + do not divert into the DWARF rewriter", + ); + + let records = warn_capture::records_since(buf, start); + assert!( + records + .iter() + .any(|r| r.contains("external_debug_info") && r.contains("does not rewrite")), + "expected an external_debug_info warning, got: {records:?}", + ); +} + +#[test] +fn with_dwarf_warns_on_source_mapping_url_section() { + let buf = warn_capture::install(); + let start = warn_capture::snapshot(buf); + + let bytes = module_with_custom_section("sourceMappingURL", b"./map.json"); + let _module = Module::parse(&bytes, false, false, true).expect("parse"); + + let records = warn_capture::records_since(buf, start); + assert!( + records + .iter() + .any(|r| r.contains("sourceMappingURL") && r.contains("does not rewrite")), + "expected a sourceMappingURL warning, got: {records:?}", + ); +} + +#[test] +fn without_dwarf_does_not_warn_on_adjacent_debug_sections() { + let buf = warn_capture::install(); + let start = warn_capture::snapshot(buf); + + let bytes = module_with_custom_section("external_debug_info", b"https://example/dwarf"); + let _module = Module::parse(&bytes, false, false, false).expect("parse"); + + let records = warn_capture::records_since(buf, start); + assert!( + !records.iter().any(|r| r.contains("external_debug_info")), + "no warning expected when with_dwarf=false, got: {records:?}", + ); +} + // DWARF rewriting, step 9: differential test for an INSTRUMENTED module. // Two paths produce per-emit byte offsets — the in-encode capture (each emit // op's `function.byte_len()` rebased onto the first instruction) and a From 00cee883d648426f8afe7d465ff475434a9752d8 Mon Sep 17 00:00:00 2001 From: evilg Date: Fri, 5 Jun 2026 09:53:27 -0400 Subject: [PATCH 13/13] Add 'deleted' field to CustomSection instances --- src/ir/module/mod.rs | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/ir/module/mod.rs b/src/ir/module/mod.rs index 36af7afa..465b6e0e 100644 --- a/src/ir/module/mod.rs +++ b/src/ir/module/mod.rs @@ -509,6 +509,7 @@ impl<'a> Module<'a> { debug.push(CustomSection { name: cs_name, data: Cow::Borrowed(custom_section_reader.data()), + deleted: false }); continue; } @@ -2144,6 +2145,7 @@ impl<'a> Module<'a> { debug_mut.sections.push(CustomSection { name, data: std::borrow::Cow::Owned(bytes), + deleted: false }); } }