LLVM backend: O(1) dispatcher lookup, register-allocator-sized chunks, IR-level PGO, and a ps1 correctness fix by dougchansan · Pull Request #14 · ExpansionPak/DolRecomp

dougchansan · 2026-08-09T21:35:16Z

The LLVM backend was leaving most of its performance on the floor next to the C backend. This is the general half of the work I did on that — four changes, none of them specific to any one title.

Measured on Mario Kart: Double Dash!! (GM4E01), the backend went from 0.49x to ~0.957x of a PGO'd C build, and ends up correct at the interpreter floor — zero architectural divergences under exact-match lockstep, same as the C backend gets.

Each commit stands alone and can be taken separately.

Before / after

Same race savestate, uncapped, both runs back to back on the same machine. Framerate is in the window title. Left is chunk-64 without a profile; right is the same module with PGO.

unprofiled — 36.8 FPS	PGO'd — 62.3 FPS

The numbers

change	result
O(1) dispatcher lookup	+59.9% where the chunk chain is long (irregular plans, 12/12 pairs, disjoint ranges). 0.9995 — a null — on a uniform fixed-128 plan, where the chain was already 2 tests, so it costs the shipping config nothing.
chunk-64 instead of 128	+7.6%, 12/12, disjoint. 64 is a knee, not a slope: 32 loses 28.4%. 160 is fine at -2.9%, so the "power of two" idea doesn't hold — 96 is a lone unexplained 35% loss.
IR-level instrumentation PGO	+73.4% (12/12, +71.8% .. +77.1%). The C half of the same module only accounts for 1.0638 of that, so the LLVM chunks' own share is 1.6301. This is what gets the backend to ~0.957 of PGO'd C.
ps1 correctness fix	`ps1` appearances in divergence reports 2,180 → 383, and the `ps1`-only class goes to zero. Under the corrected lockstep oracle the module reports 0 architectural divergences, same as PGO'd C, while the pre-fix module still reports 34 — which is what makes the zero mean something.

How these were measured: interleaved A/B pairs, six forward then six with the order reversed, against a same-session control. This host's baseline drifts ~0.9 → ~1.0 across a session as it cools, which is bigger than most of the effects, so single runs are useless here. Anything inside ±1.4% pair-to-pair I treat as a null.

Does this hold up on other games?

That's the part that matters for upstreaming, so I ran the same recompiler changes against two more titles — Luigi's Mansion (GameCube) and Skyward Sword (Wii). Same approach each time: interleaved pairs, uncapped, per title.

	MKDD (GC)	Luigi's Mansion (GC)	Skyward Sword (Wii)
chunk knee	64	64	64
chunk-64 vs 128	+7.6% (disjoint)	+8.2% (disjoint)	+7.6% (disjoint)
IR PGO	+73.4%	+31.1%	+49.0%
full-stack FPS	—	20.6 → 43.3 (2.15x)	19.1 → 38.2 (~2x)

Chunk sizing generalises, and all three landed on 64. I'd still sweep per title rather than assume it — the point of the knob is that the optimum is a property of the game's code, not a constant — but three for three at 64 is a reasonable default to start from.

Worth knowing if you sweep: MKDD's first chunk-64 sweep came out at only +1.4% with overlapping ranges, and I nearly wrote it off. That run was under the linear dispatcher; once the lookup was indexed the same comparison resolved to +7.6% with disjoint ranges. A slow dispatcher masks a chunk-size win, so fix the lookup before you trust a sweep.

PGO generalises to all three, across both GameCube and Wii. Always large, never the same number: +31% to +73%. That spread is the honest shape of it — it depends on the title, so I wouldn't quote any single figure as "the" PGO number.

The indexed dispatcher stays a null on uniform plans. On Luigi's Mansion it measured −1.5%, which is the same story as MKDD's uniform fixed-128 null. It's a big win where the chunk chain is long and nothing where it's short. Not trying to sell it as more than that.

Correctness held on both new titles — zero interpreter fallbacks across every run, and PGO cycle counts reproducing to about five significant figures. Caveat worth being clear about: the lockstep oracle only exists for MKDD, so for Luigi's Mansion and Skyward Sword this is a strong no-divergence signal, not bit-exact proof.

Skyward Sword is the first Wii title this recompiler has produced an LLVM build for. 19,833 objects, zero unknown instructions, runs clean. The interesting bit: unprofiled, the LLVM backend is slower than the C backend on that title — PGO is what flips it into a win. If you only ever measure unprofiled LLVM on Wii you'd conclude the backend isn't worth it.

Gaps I'd rather state than have someone find: Skyward Sword's chunk curve is anchored at 256 rather than the 1024 default (I skipped the 1024 arm for time), the SS numbers are the main executable only — I haven't touched the heavy combined-module workloads — and every PGO figure here is same-scene trained. Same-scene is a fair comparison but it isn't arbitrary gameplay.

Two things that bit me

The indexed dispatcher needs a companion gen_module_tables change. It emits page-indexed run tables, so a table generator that scrapes guard conditions comes up empty and fails with "no coverage ranges found". Worth knowing before you turn the flag on.

A PGO collection run that isn't headless can silently produce a zero-count profile. The file looks valid, pgo=use accepts it, and it tells codegen every block is cold — so you get a "profiled" build that's worse than unprofiled and nothing warns you. I hit this on Wii. llvm-profdata show with a count > 0 check should be a hard gate before any pgo=use build.

The four changes

1. Dispatcher lookup is O(1). src/backend/dispatch.c walked the chunk table linearly to resolve a guest address — a scan per indirect branch, on the hottest path in the runtime. Behind DOLRECOMP_DISPATCH_LOOKUP this builds a page index. Both arms measured on a byte-identical object set, so the only difference is the lookup. Extends the existing tests/test_dispatch.c (+54 lines) to check the indexed path against the linear one for equivalence.

One thing I can't fully reconcile: at uniform-64 the index is still worth ~3.7% (12/12), which doesn't match the fixed-128 null. The value scales with how much scanning the plan actually causes. Reporting it rather than explaining it away.

2. Chunk sizing. The LLVM chunk size inherited the C backend's 1024-instruction default, which is the wrong constant for a register allocator that keeps the whole guest register file live across a chunk. 128 was already +57.9% throughput and -66% .text against 1024; 64 is another +7.6%. Below 64 the cross-chunk edge cost (materialize + call + returned-pc compare + reload) outgrows what the wider allocator scope buys.

Also in here: selectable target CPU/features (DOLRECOMP_LLVM_CPU, DOLRECOMP_LLVM_FEATURES), defaulting to the portable x86-64 baseline, and a note that dropping the vectorizers costs -27% throughput and makes the module bigger — SSE2 is in the baseline and Gekko paired-singles are 2-wide f32 pairs, so SLP has real work to do.

3. IR-level PGO. The C backend gets PGO free through CFLAGS. Nothing reaches these objects that way, because the backend emits IR and codegens in-process — clang never sees a translation unit. Sample/AutoFDO genuinely can't work (the emitter attaches no DILocation, so there's nothing to bind counts to), but IR-level instrumentation PGO needs no debug info at all: PGOInstrumentationGen/Use at the front of the pipeline and InstrProfilingLoweringPass at the end. No CMake change, no new dependency.

Both passes sit on the raw emitter output so Gen and Use see identical IR — otherwise the CFG hashes disagree and functions silently go unprofiled. Lowering runs last for the same reason clang does it last: once lowered, the counters are a load/add/store and GVN or DSE will fold loop iterations into one increment.

There's a staleness gate in a separate commit. A profile that no longer matches the emitted CFG doesn't fail, it degrades function by function while the build still looks trained. The gate checks entry-count metadata after PGOInstrumentationUse and defaults to error.

4. ps1 fix. Lockstep against the interpreter showed the backend diverging on ps1, the high slot of a paired-single register, with values carrying 0x4330000000000000 — the int-to-double bias. Two DolIR builder sites splatted a scalar result into both slots where Gekko writes ps0 only: lfd (lfs fills both, lfd doesn't), and fmr/fneg/fabs/fnabs/fsel, all SetPS0-only in Interpreter_FloatMisc. The C emitter always had both guards, which is why C never showed this. DolIR only feeds the LLVM pipeline, so no C codegen changes. Bumps DOLLLVM_CACHE_VERSION to v7.

Caveats

Split-screen still sits ~6% short of parity (0.9378) against 0.9573 single-player. More profile training doesn't close it. Known, not fixed here.
One title, one host. The mechanisms should generalise; I wouldn't assume the magnitudes do.
The 0.957 figure passes on the median, not on every pair.
CI is green on all six checks. One fix was needed to get there and is included: pipeline.c called time() without including <time.h>, which clang-with-MSVC-headers and glibc both resolve transitively but MinGW/GCC doesn't. Locally the suite passes 16/16 (C backend) and 19/19 with -DDOLRECOMP_ENABLE_LLVM=ON.
The performance numbers come from the campaign builds these commits were extracted from, not re-measured against this exact rebased series.

Not included

Experimental arms from the same work that are off by default, measured neutral, or rejected: the chunk-boundary planner, per-path chunk-edge state narrowing, inlined paired-single load/store, runtime-boundary reload narrowing, the machine outliner and hot/cold-split flags, static pricing tools, and a whole-function recompilation prototype that I measured and rejected. Happy to send any of those separately if they're wanted.

Three gaps that make the LLVM backend hard to work on. The object cache key covered the instruction words, chunk ranges and target triple, but not the pass pipeline, optimisation level, LLVM version or target CPU/feature string. Editing the pipeline therefore reused objects built with the old one and reported them as the new result. Those inputs are now named constants that feed both codegen and dolllvm_codegen_fingerprint(), which the key hashes -- so there is one place to change them and no way to change them without invalidating the cache. On Windows every chunk's progress line was printed before any work began, so the log looked complete while the build sat still and named nothing. Lines now come from inside the job, and completion carries elapsed seconds: the stuck chunk is the one with no Finished line. DOLRECOMP_LLVM_TRACE_PASSES registers a pass-instrumentation callback that names each pass and IR unit as it starts, flushed. instcombine failing to converge on a recompiled Gekko function previously spun for 49 minutes at 1.00 core and consumed 18.9 CPU-hours without ever revealing which function it was on.

Measured: dropping loop-vectorize, slp-vectorizer and vector-combine costs 27% throughput on Mario Kart and makes the module 4.6% *larger*. They are worth 1.37x. The intuition that they cannot pay off against CPU "generic" with an empty feature string is simply wrong. SSE2 is part of the x86-64 baseline ABI, so 128-bit vectors are always available, and Gekko paired-singles are inherently 2-wide f32 pairs -- ps_add, ps_mul, psq_l/psq_st map almost directly onto SSE2 pairs. SLP has real work here, not speculative widening. The comment now records the measurement so this is not retried. Also stop hiding cache reuse. On Windows the reuse path returned before the progress print, so a resumed build logged nothing for the objects it reused and was indistinguishable from a full regeneration -- which is exactly the thing that must not be assumed when checking whether a codegen change was tested.

A chunk becomes exactly one LLVM function -- every generated object carries precisely two symbols, func_X and func_X_budget, whatever the guest function boundaries are. So DOLRECOMP_LLVM_CHUNK_INSTRUCTIONS is really "how many basic blocks must the register allocator keep the entire promoted guest register file live across", and at the inherited 1024 that was far past what 16 x86-64 registers hold: 43% of every generated function was stack spill traffic. Measured on Mario Kart, one variable, separate output root per arm: chunk objects .text vs C speed vs 1024 1024 726 1,012,522,870 16.2x 0.3288 256 2,902 450,227,766 7.2x 0.4404 +33.9% 128 5,803 345,215,974 5.5x 0.5192 +57.9% 64 11,604 274,... 0.5267 (+1.4% over 128, ranges overlap) Monotonic to 128 with disjoint ranges at every step, and generation cost fell 18x along the way (24.9 -> 1.4 CPU-hours), which confirms per-function compile cost is superlinear in block count. 64 is not a proven gain: +1.4% with overlapping ranges, so 128 is the setting. 128 was previously unreachable -- it was the validator's floor, and the curve had not turned there. The LLVM floor is now 32 so the optimum can actually be found per title; the C path keeps its own 128 floor and its separately measured 1024 optimum, which is a different effect (instruction cache, not register pressure). There is a real minimum somewhere below: a call leaving the chunk returns through the dispatcher instead of branching. This is a curve to sweep, not a constant to copy.

DOLRECOMP_LLVM_CPU and DOLRECOMP_LLVM_FEATURES override the target machine; "native" resolves through sys::getHostCPUName(), which createTargetMachine does not do itself. Both feed the codegen fingerprint, so switching target cannot reuse objects built for another one -- verified: the znver5 run rebuilt 5803/5803 objects with zero reuse. Measured on this 9950X3D, chunk 128, against generic: generic speed 0.5192 (0.4852-0.5437) module 349,250,048 znver5 speed 0.5160 (0.4909-0.5359) module 331,272,192 -0.6% with overlapping ranges, so no proven gain; the module is 5.1% smaller, which is real. Default stays generic, which is also the portable choice, so nothing is traded. The mechanism is clean and worth recording: Gekko paired-singles are exactly 2 x f32 = 64 bits, and SSE2's 128-bit registers already hold them with room to spare. E001 showed removing vectorization costs 27%; E005 shows widening beyond SSE2 buys nothing. Vectorization at the baseline is essential, above it is not.

The lockstep verifier (C003) found the LLVM backend diverging from the interpreter on ps1 -- the high slot of a paired-single register -- 2,180 times against the C backend's 70, with native values carrying 0x4330000000000000, the int-to-double conversion bias. Two sites in the DolIR builder splat a scalar result into both slots of the pair where Gekko writes ps0 only: - lower_float_memory wrote ps1 for every float load. Gekko splits them: lfs fills both slots (Interpreter::lfs -> Fill), lfd writes ps0 and preserves ps1 (Interpreter::lfd -> SetPS0). The guest's int-to-double idiom is stw/stw/lfd, which is exactly where the bias came from. - fmr, fneg, fabs, fnabs and fsel wrote ps1. All five are SetPS0-only in Interpreter_FloatMisc. The C emitter has always had both guards (emit_fload/emit_floadx take the ps1 write under `if (single)`, and the FMR/FNEG/FABS/FNABS/FSEL cases write ctx->fpr alone), which is why C does not show this class. DolIR is consumed by the LLVM pipeline only, so nothing here changes C codegen. Bumps DOLLLVM_CACHE_VERSION to v7: default LLVM codegen changes, so every cached v6 object is stale.

…OOKUP dolrecomp_find_original emits a chain of range tests, collapsing only consecutive equal-stride chunks into an indexed jump table. That is 2 tests on a fixed-128 plan and 2,089 (E008b) or 5,224 (E008a) on a control-flow-aligned one, taken on every dispatch into the module -- and the two E008 arms rank by chain length exactly as they rank by measured slowness. No throughput number from an irregular-boundary plan is interpretable until that is fixed. DOLRECOMP_DISPATCH_LOOKUP=indexed emits a 4 KiB page index over the covered guest range instead. The page selects a small window of runs and the window is walked forward, so the walk is bounded by the runs that intersect one page rather than by the chunk count: 2 for fixed-128, 7 for E008b, 20 for E008a. The default stays linear, and the linear path is unchanged rather than merely intended to be: re-emitting the recorded plans of E:\mpo (the 726-chunk C build behind the pinned 760c208d module), E:\ml16, E:\ml10 and E:\ml9 through the refactored emitter reproduces each archived lookup byte for byte. Correctness of the new form is checked against the old one over the real plans by scripts/check-dispatch-lookup.py, which parses both emitted lookups back into resolvers and sweeps every aligned address in the covered range plus a 64 KiB margin, and a sample of unaligned ones: 916,518 addresses agree on all four plans, 742,616 of them resolving to a chunk.

The C backend gets PGO for free: its chunks are C source, so clang's own -fprofile-generate / -fprofile-use reach them through CFLAGS. Nothing reaches these objects the same way -- this backend emits IR and codegens it in-process, so clang never sees a translation unit. Sample/AutoFDO is genuinely unavailable here: the emitter attaches no DILocation, so a sample profile has nothing to bind to. IR-level instrumentation PGO needs no debug info, though -- it is two LLVM passes over the module this backend already builds. DOLRECOMP_LLVM_PGO=gen instrument DOLRECOMP_LLVM_PGO=use apply DOLRECOMP_LLVM_PROFILE DOLRECOMP_LLVM_PROFILE=<file> merged .profdata (use mode only) Both passes go at the very front of the pipeline, on the raw emitter output, so the CFG hashes PGOInstrumentationUse matches against are computed on exactly the IR PGOInstrumentationGen saw. Running both there makes that identity structural rather than a property of the passes in between, which stay free to change without invalidating a profile. The counter intrinsics must survive the optimizer as intrinsics: once lowered they are a load, an add and a store on a global, and GVN or DSE will fold two loop iterations into one increment. InstrProfilingLoweringPass therefore runs last, where clang runs it and for the same reason. The codegen fingerprint gains a PGO suffix keyed to the mode, and in use mode to the profile's *content* rather than its path -- a profile regenerated in place makes the path a non-identity. The suffix is absent entirely when PGO is off, so default builds stay byte-identical and the existing object cache keeps its meaning. Off by default.

A profile that no longer describes the emitted CFG does not fail. It degrades: PGOInstrumentationUse rejects the mismatched records one function at a time and leaves those functions unprofiled, so the build succeeds, the module looks trained, and the result is quietly somewhere between a profiled and an unprofiled build. Clang's warning for a C half (-Wprofile-instr-out-of-date) is easy to suppress, and the LLVM half never had one at all. The gate is a positive check rather than a warning scrape. After PGOInstrumentationUse has run -- observed through a pass-instrumentation callback, so nothing about the pipeline moves -- every defined function that matched its profile record carries entry-count metadata and every function that did not carries none. On a profile collected from this same module the second set is empty, because IR instrumentation records EVERY function at gen time whether or not it ever executed. So a non-zero count means the profile and the binary have diverged, not that the training workload was narrow. DOLRECOMP_LLVM_PGO_STALE=error fail the emit (the default) DOLRECOMP_LLVM_PGO_STALE=warn report and keep building DOLRECOMP_LLVM_PGO_STALE=off no check Default is `error`: a stale profile is a wrong measurement, not a slow one. Tallies are atomic because the job runner is threads in one process, and the summary prints once from an atexit hook so a `warn` build ends with a total rather than thousands of individually ignorable lines.

Evidence for the dispatcher, chunk-sizing, PGO and ps1 commits in this branch, so the numbers travel with the code rather than living in a reviewer's trust. Includes the protocol the numbers were produced under -- interleaved A/B pairs with a reversed-order block, same-session controls, a +/-1.4% pair-to-pair noise floor, ranges and pair counts rather than bare medians, and exact-match lockstep against the interpreter as the correctness oracle -- because on this host the baseline drifts further across a session than most of the effects being measured, and a single-run number would not mean anything. Keeps the caveats in: split-screen is ~6% short of the gate and more profile training does not fix it, everything is one title on one host, the performance gate passes on the median rather than on every pair, uniform-96's 35% loss is unexplained, and this branch is syntax-checked but its dispatcher test has not been executed here.

emit_llvm_chunk_job calls time(NULL) but pipeline.c never included <time.h>. Clang with the MSVC headers and glibc both pull it in transitively, so this built fine on the toolchains it was developed against; MinGW/GCC does not, and fails with an implicit-declaration error under -std=gnu11. Introduced alongside the timing output added with the widened cache key.

The branch builds clean with the LLVM backend enabled against LLVM 20.1.8 and passes 18/19 of the repository suite, including the dispatch test and the three LLVM tests. The one failure, codegen_compile, reproduces identically on unmodified main in the same environment, so it is environmental rather than a regression. Also corrects the dispatcher section: tests/test_dispatch.c already existed upstream; this branch extends it rather than adding it.

The earlier 18/19 note was an environment artifact, not a repository defect. codegen_compile configures a nested CMake project to compile the generated C; that configure needs a resource compiler on Windows, and the shell running the suite had no Windows SDK on PATH. With the SDK present it passes, and it passes in CI on every platform. No repository change was needed for it.

The standalone results write-up was more than this needs; the numbers live in the PR description instead. Keeps two screenshots of the same race savestate, uncapped, back to back on one machine: the unprofiled chunk-64 module against the PGO'd one, framerate in the window title.

dougchansan added 12 commits August 9, 2026 11:23

dougchansan force-pushed the llvm-dispatcher-pgo-upstream branch from fdff6e7 to 4990274 Compare August 9, 2026 23:25

dougchansan and others added 3 commits August 9, 2026 14:20

Fix LLVM 19 build

caea3f7

Fix LLVM 20 build

fa78f90

siahisaforker merged commit fa0cf61 into ExpansionPak:main Aug 10, 2026
6 checks passed

Navigation Menu

Uh oh!

LLVM backend: O(1) dispatcher lookup, register-allocator-sized chunks, IR-level PGO, and a ps1 correctness fix - #14

LLVM backend: O(1) dispatcher lookup, register-allocator-sized chunks, IR-level PGO, and a ps1 correctness fix#14
siahisaforker merged 15 commits into
ExpansionPak:mainfrom
dougchansan:llvm-dispatcher-pgo-upstream

dougchansan commented Aug 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dougchansan commented Aug 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before / after

The numbers

Does this hold up on other games?

Two things that bit me

The four changes

Caveats

Not included

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dougchansan commented Aug 9, 2026 •

edited

Loading