relocation fails on RISC-V AOT with opt level > 0

[psilva261]

The compilation for RISC-V with wamrc

wamrc --target=riscv64 \
--target-abi=lp64 \
--cpu=generic-rv64 \
--cpu-features='+i,+m,+a' \
--opt-level=1 \
--size-level=1 \
--bounds-checks=1 \
-o program.riscv64.wamr program.wasm

runs through. But during runtime I get:

[00:00:00:000 - 0]: Warning: loader mmap memory address is not in the first 2 Gigabytes of the process address space.
[00:00:00:000 - 0]: Warning: loader mmap memory address is not in the first 2 Gigabytes of the process address space.
[00:00:00:000 - 0]: Warning: loader mmap memory address is not in the first 2 Gigabytes of the process address space.
runtime load module failed: AOT module load failed: relocation truncated to fit R_RISCV_PCREL_LO12_I failed.

--opt-level=0 works though

[TianlongLiang]

It could be the optimizer, it may move relocation entries in a way that breaks the WAMR's implementation assumption:

            /* Already handled in R_RISCV_PCREL_HI20, it should be skipped for
             * most cases. But it is still needed for some special cases, e.g.
             * ```
             * label:
             *    auipc t0, %pcrel_hi(symbol)   # R_RISCV_PCREL_HI20 (symbol)
             *    lui t1, 1
             *    lw t2, t0, %pcrel_lo(label)   # R_RISCV_PCREL_LO12_I (label)
             *    add t2, t2, t1
             *    sw t2, t0, %pcrel_lo(label)   # R_RISCV_PCREL_LO12_S (label)
             * ```
             * In this case, the R_RISCV_PCREL_LO12_I/S relocation should be
             * handled after R_RISCV_PCREL_HI20 relocation.
             *
             * So, if the R_RISCV_PCREL_LO12_I/S relocation is not followed by
             * R_RISCV_PCREL_HI20 relocation, it should be handled here but
             * not implemented yet.
             */

"So, if the R_RISCV_PCREL_LO12_I/S relocation is not followed by the R_RISCV_PCREL_HI20 relocation" @no1wudi Would you mind sharing some insight on how it can be implemented?

[no1wudi]

@psilva261 Could you try different --size-level=1? what's your expected cmodel here? low or medany for riscv?

[psilva261]

Thanks for the help already. Yes, I've included LLVM 20 to try other size levels. For instance with --size-level=3 and medlow I get

runtime load module failed: AOT module load failed: relocation truncated to fit R_RISCV_HI20 failed.

And with --size-level=0 and large

runtime load module failed: AOT module load failed: resolve symbol .data.rel.ro failed

[no1wudi]

I guess the relocation logic for riscv64 is not complete enough, for this issue, I will fix it.

Could you provide a minimal demo code to reproduce this issue?

[psilva261]

Thanks a lot, I created a minimal example here with Docker:

https://github.com/psilva261/wamr-demo-pcrel-lo-reloc

It's using the Linux platform with user mode qemu and upstream WAMR

The problem was noticed originally on a custom platform https://github.com/psilva261/wasm-micro-runtime-zkvm (RISC-V 64 baremetal) but is essentially produced the same way. I can also produce an example with the custom platform/a different LLVM version if it's more convenient, but that's probably not as minimal (For some setups I needed to register extra symbols like mentioned in the Dockerfile)

[no1wudi]

@psilva261 could you try this patch #4777?

[psilva261]

Yes definitely. I tried it and the new relocation mechanism is being called. So in theory the relocation errors disappear although I'm running into an out of bounds memory access

exception: Exception: out of bounds memory access

Actually the minimal example is now extended a bit such that gcc for riscv64-ima is being compiled. This way the soft float issues shouldn't interfere with the reproduction anymore

[no1wudi]

Have you tested the program with interpreter mode? Is this Exception: out of bounds memory access related to relocation? I'm not entirely sure; the new relocation logic requires more testing.

[psilva261]

I just tested with interpreter mode, there it works on the patched branch. https://github.com/psilva261/wamr-demo-pcrel-lo-reloc/tree/interpreter-mode

Hard to say if it's strictly related to relocation. When running it with gdb I get:

Program received signal SIGSEGV, Segmentation fault.
0x000000000008eb74 in memset ()
(gdb) bt
#0  0x000000000008eb74 in memset ()
#1  0x0000000006f16d78 in ?? ()

[psilva261]

Thanks again for the patch. I did some new tests with fresh eyes. What I noticed that e.g. with code that is produced by TinyGo it actually runs through and works correctly. (With --opt-level=3) In fact even a simple Go example gets far enough to exit normally but it won't print anything. However I had to increase PCREL_CACHE_SIZE to 16 for TinyGo and to 1024 for Go.

It's also worth mentioning that wasm code generated by the Go compiler code isn't optimized yet and makes a lot of use of deeply nested block instructions with labels to allow Go routine handling.

I'm not sure to debug this though since the interesting parts seem to happen with the AOT executable. Would source level debugging be useful here?