fuse: drop vestigial cabi_realloc / canonical-ABI adapter once the inter-component boundary is internalized (unblocks --memory shared)

[avrabe]

Observation

After meld fuse of two components whose interface is fully resolved internally (app imports gale:kernel, host exports it → fused core has 0 remaining gale:kernel imports), the fused core still exports cabi_realloc (×2, one per input) and still contains memory.grow (×2):

(export "cabi_realloc" (func 13))
(export "cabi_realloc$1" (func 32))
memory.grow occurrences: 2

Those cabi_realloc functions (wit-bindgen canonical-ABI realloc, backed by Rust dlmalloc → sbrk → memory.grow) exist to marshal data across the component boundary — which fusion has just removed. They're vestigial. Because they're exports, loom can't DCE them, so the allocator + memory.grow survive.

Consequence

This is what blocks the lean MCU path: meld fuse --memory shared --address-rebase fails with "memory.grow not supported with address rebasing" — solely because of the dead canonical-ABI allocator. (The user's logic: meld fusion should make the per-component adapter unnecessary; right now it keeps it.)

Ask

When fusion internalizes the inter-component boundary and the remaining external surface is scalar/flat (no realloc-requiring lifts), meld should de-export / drop the now-internal cabi_realloc and canonical-ABI adapter functions (or mark them internal so loom DCEs them). Then dlmalloc/sbrk/memory.grow DCE away → --memory shared becomes viable → one lean single-address-space core for synth (target the wasm-dist 544 B class, not tens of KB).

Kill-criterion

"After meld fuse of an all-scalar-boundary component pair, cabi_realloc is still exported / memory.grow still present in the fused core" → not fixed.

Repro: crates/gale-app-demo/dissolve.sh on gale@feat/kiln-dissolve-libos. Relates: gale#89 (the gale-side tracking; the grow is NOT gale code — it's the canonical-ABI adapter meld should drop on fusion).

[avrabe]

Forwarded the root cause upstream to pulseengine/wit-bindgen#6: wit-bindgen emits the grow-able allocator behind cabi_realloc even for scalar-only worlds. If the fork lands a no-grow/elided cabi_realloc for flat worlds, the components carry no memory.grow and this meld-side drop becomes belt-and-suspenders (still nice to have for mixed cases). Two complementary fixes; wit-bindgen is the root.

[avrabe]

Grounding before implementation — a safety constraint that relocates the fix

Traced the cabi_realloc export machinery to design this. One finding changes the approach: the drop must NOT be done at the merger level.

• The plain cabi_realloc flows through the generic export-copy loop (merger.rs~2004); the multi-memory $N exports at ~2150; realloc_map is built ~2116-2135.
• But the downstream P2 component-wrap aliases the fused module's cabi_realloc for canon lower (component_wrap.rs:1280-1300). The merger has no signal for whether the wrap (or the external host surface) will still need it. A merger-level skip would silently break P2-component output where the host must allocate to pass non-scalar args — i.e. over-drop = silent marshalling corruption, exactly the failure to avoid. (test_cabi_realloc_suffixed_exports_generated expects cabi_realloc kept from the same local signals, confirming the merger can't distinguish keep-vs-drop.)

Safe approach (decided)

Do the de-export at the encode/output stage (lib.rs::encode_output), where the output format and the final external surface are known. De-export (→ dead → loom DCEs the allocator + memory.grow) iff all hold, else keep:

1. output is a core module (not a P2 wrap that aliases realloc for a non-scalar export), AND
2. the entire external export/import surface is scalar/flat (no realloc-requiring lift/lower), AND
3. no adapter site passes pointers and no resource path needs realloc.
   Fail-safe: keep cabi_realloc if any condition is unmet or undetermined. Under-drop = missed optimization (correct); over-drop = corruption.

Oracle

Two tiers: (a) an lib.rs-level unit test on a scalar-only-surface fused core asserting no cabi_realloc export; (b) ideally a golden_e2e Tier-B fixture with a real scalar-boundary component pair asserting 0 cabi_realloc exports and 0 memory.grow post-fuse (the true kill-criterion; needs a scalar-boundary fixture). Tier-5 → Mythos-gated.

Status

Designed + located; not implemented. The safe version is a focused encode-stage change with over-drop stakes (not the one-line merger skip it first appeared to be), so it deserves its own careful pass rather than a budget-pressured partial. Recording here; can pick up as a dedicated effort.

[avrabe]

Refinement (from wit-bindgen#6 cross-tool resolution) — meld owns this, and it's smaller than first scoped

avrabe confirmed the source-side elision (wit-bindgen disable_cabi_realloc) is the wrong layer: wasmtime needs cabi_realloc present to wire canonical-ABI linkage, so eliding at generation breaks the composed-on-wasmtime path. meld is the correct home (and the divergence point): the wasmtime path runs the original components (realloc present); the gale path runs the meld-fused --memory shared core (realloc removed). One wit-bindgen build serves both.

Two refinements to this issue's plan:

1. meld's job is the conditional UN-EXPORT, not a full strip. "Un-DCE-able" was only "it's exported" (exports are GC roots). Since the gale pipeline is meld → loom → synth, meld just needs to not emit the cabi_realloc export under the fail-safe condition; loom then DCEs the dead cabi_realloc + $sbrk + memory.grow. (Direct strip stays optional, only if loom is ever out of the path.) This is a smaller, lower-risk change than a wasm-level function strip.
2. The fail-safe condition is unchanged and remains the crux: un-export only for core-module output, all-scalar external surface, no adapter/resource/wrap consumer (the P2 wrap aliases realloc — component_wrap.rs:1280-1300). Keep otherwise.

Net: #298 = "conditionally suppress the cabi_realloc export (+ $N) in the fused core when provably vestigial; let loom DCE the rest." wit-bindgen#6 parks; gale#89 clears when this lands. Still Tier-5 + oracle-gated (oracle: scalar-boundary fuse → no cabi_realloc export; e2e kill-criterion memory.grow gone verified after loom).

[avrabe]

Decisive finding (from attempting the impl) — un-export alone is INSUFFICIENT; #298 ⟺ #299 are coupled

Verified the failure origin: the rejection is in meld at rewriter.rs:395-398 — when the address-rebase rewrite hits a MemoryGrow operator it hard-errors "memory.grow not supported with address rebasing". Crucially, that memory.grow lives in $sbrk (the allocator cabi_realloc → __rust_alloc → dlmalloc → $sbrk → memory.grow), not in cabi_realloc directly.

Consequence: un-exporting cabi_realloc (the refined plan) does not unblock the gale case. $sbrk is still in the module and still gets rewritten → the rebase pass still chokes on its memory.grow before loom ever runs to DCE anything. So "un-export and let loom DCE" can't work — meld's own pass fails first. #298 and #299 are therefore one problem, not two.

What it actually takes (two viable paths):

1. meld-side dead-allocator DCE (preferred, real fix): in --memory shared --address-rebase mode, before the rewrite, compute reachability from the non-cabi_realloc exports + start, identify the dead allocator chain (cabi_realloc, $sbrk, dlmalloc, …) when the boundary is internalized + scalar, and exclude those functions from the merged module (with re-index). The dead memory.grow never reaches the rebase check. This is a bounded DCE pass + re-index.
2. Reachability-gated rewriter relaxation: keep the hard-fail for memory.grow that is live, but when it's provably dead (only reachable from a vestigial cabi_realloc being dropped), skip/emit-without-rebase instead of failing. Same reachability requirement, smaller surgery.

Both need a call-graph reachability analysis meld doesn't currently have at that stage; both must stay fail-safe (a live memory.grow must still hard-fail — never silently mis-rebase). The current hard-fail is the correct conservative default; the fix is to carve out the provably-dead case.

Status: the "conditional un-export" framing is superseded — this is a real meld-side DCE/reachability feature (path 1 recommended), coupled with #299. Fully characterized now; not rushed. Oracle stays: scalar-boundary fuse with --memory shared --address-rebase → succeeds, 0 cabi_realloc export, 0 memory.grow.

[avrabe]

In-repo oracles landed + two findings that fix the seam for the implementation

Branch fix/298-dead-allocator-dce now carries the executable spec (no code-behavior change yet):

• Kill-criterion oracle (merger.rs::test_298_vestigial_grow_blocks_shared_rebase_fusion): fuses a scalar-boundary pair in the real MCU config (SharedMemory ⟹ address_rebasing, as lib.rs forces) where the exported cabi_realloc body carries memory.grow. Asserts today's hard-fail. This moves the repro from gale's external dissolve.sh into meld (also closes the meld-repo-local-smoke gap noted on Close the meld→kiln end-to-end seam: run a meld-fused core module on kiln and assert behavioural equivalence #297). When the fix lands this goes red → flip to: merge succeeds, 0 cabi_realloc export, 0 memory.grow.
• Positive control (test_298_no_grow_realloc_fuses_under_shared_rebase_today): same harness, memory.grow removed. Asserts a grow-free cabi_realloc already fuses under shared+rebase today, with no fuse: drop vestigial cabi_realloc / canonical-ABI adapter once the inter-component boundary is internalized (unblocks --memory shared) #298 fix. This validates the fork's integration/embedded-rt-no-grow arena approach from the meld side without building the fork toolchain. ⟹ the two fixes are complementary: fork no-grow = belt, fuse: drop vestigial cabi_realloc / canonical-ABI adapter once the inter-component boundary is internalized (unblocks --memory shared) #298 meld-side drop = suspenders (mixed/un-elided case).

Finding 1 — the reject fires during merge, so "post-merge DCE" is impossible as-is

The hard-fail is reached at merger.rs:1923 (extract_function_body → rewriter.rs MemoryGrow arm) while building each MergedFunction.body — i.e. before MergedModule is complete. So any reachability/DCE pass that wants to run after the merge first requires making the rewrite tolerant (defer the grow-reject, record the grow-bearing merged-func indices), then deciding post-merge. This is why path-1 and path-2 both need the same two-phase restructure; it is not a localized patch.

Finding 2 — merge-time reachability is blind to the realloc consumers → over-drop trap

cabi_realloc's consumers — the generated adapters and the P2 component-wrap (component_wrap.rs:1280-1300, which aliases the fused realloc for canon lower) — are produced by the Fuser, after merge. A reachability/DCE computed inside Merger cannot see them, so it would happily drop a realloc the wrap still needs = the silent marshalling corruption this issue is explicitly trying to avoid.

Consequence for the design: the keep-vs-drop decision must be computed in the Fuser (where output-format, adapter-sites, and the external scalar-surface are all known) and threaded down as an explicit signal; Merger/rewriter must default to keep + hard-fail (today's safe behavior) and only relax when the Fuser proves the allocator vestigial. This matches the earlier "encode-stage / upstream signal" conclusion and explains why merge-time-only is unsafe.

Net: the safe implementation is (a) Fuser computes a vestigial_allocator verdict from output-format + adapter-sites + scalar-surface (fail-safe: unsure ⟹ keep); (b) rewriter gains a tolerant mode enabled only under that verdict (defer grow-reject for the dead chain, hard-fail otherwise); (c) post-merge, stub the provably-dead functions (no re-index → all index maps stay valid) + suppress the cabi_realloc export; loom DCEs the rest. Still Tier-5 → Mythos clean-room + mythos-pass-done before merge.

[avrabe]

Reclassified SECONDARY per #300

Per the #300 decision, meld commits to model 2 (multi-memory structural isolation) as the dissolved-MCU path; --memory shared (this issue) is now a secondary option, not the committed isolation path.

What landed on fix/298-dead-allocator-dce stays (it's sound and guards the secondary shared path): in-repo kill-criterion + real-artifact blocker oracles, proven compatibility with the fork's arena-backed no-grow cabi_realloc, the inert fail-safe vestigial-allocator verdict (cabi_realloc_drop_provably_safe), and the inert tolerant-rewriter flag. Pausing the corruption-critical drop wiring (parts c) — it's no longer on the priority path. Reopen/resume if the shared mode is needed for a specific target.