[PW_SID:958320] bpf, riscv64: Support load-acquire and store-release instructions#354
[PW_SID:958320] bpf, riscv64: Support load-acquire and store-release instructions#354linux-riscv-bot wants to merge 9 commits into
Conversation
In preparation for supporting BPF load-acquire and store-release instructions for architectures where bpf_jit_needs_zext() returns true (e.g. riscv64), make insn_def_regno() handle load-acquires properly. Signed-off-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
We're planning to add support for the load-acquire and store-release BPF instructions. Define emit_load_<size>() and emit_store_<size>() to enable/facilitate the (re)use of their code. Tested-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Andrea Parri <parri.andrea@gmail.com> [yepeilin@google.com: cosmetic change to commit title] Signed-off-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Support BPF load-acquire (BPF_LOAD_ACQ) and store-release
(BPF_STORE_REL) instructions in the riscv64 JIT compiler. For example,
consider the following 64-bit load-acquire (assuming little-endian):
db 10 00 00 00 01 00 00 r1 = load_acquire((u64 *)(r1 + 0x0))
95 00 00 00 00 00 00 00 exit
opcode (0xdb): BPF_ATOMIC | BPF_DW | BPF_STX
imm (0x00000100): BPF_LOAD_ACQ
The JIT compiler will emit an LD instruction followed by a FENCE R,RW
instruction for the above, e.g.:
ld x7,0(x6)
fence r,rw
Similarly, consider the following 16-bit store-release:
cb 21 00 00 10 01 00 00 store_release((u16 *)(r1 + 0x0), w2)
95 00 00 00 00 00 00 00 exit
opcode (0xcb): BPF_ATOMIC | BPF_H | BPF_STX
imm (0x00000110): BPF_STORE_REL
A FENCE RW,W instruction followed by an SH instruction will be emitted,
e.g.:
fence rw,w
sh x2,0(x4)
8-bit and 16-bit load-acquires are zero-extending (cf., LBU, LHU). The
verifier always rejects misaligned load-acquires/store-releases (even if
BPF_F_ANY_ALIGNMENT is set), so the emitted load and store instructions
are guaranteed to be single-copy atomic.
Introduce primitives to emit the relevant (and the most common/used in
the kernel) fences, i.e. fences with R -> RW, RW -> W and RW -> RW.
Rename emit_atomic() to emit_atomic_rmw() to make it clear that it only
handles RMW atomics, and replace its is64 parameter to allow to perform
the required checks on the opsize (BPF_SIZE(code)).
Tested-by: Peilin Ye <yepeilin@google.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
[yepeilin@google.com: whitespace changes; cosmetic changes to commit
title and message]
Signed-off-by: Peilin Ye <yepeilin@google.com>
Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Currently, the verifier inserts a zext instruction right after every 8-, 16- or 32-bit load-acquire, which is already zero-extending. Skip such redundant zext instructions. While we are here, update that already-obsolete comment about "skip the next instruction" in build_body(). Also change emit_atomic_rmw()'s parameters to keep it consistent with emit_atomic_ld_st(). Note that checking 'insn[1]' relies on 'insn' not being the last instruction, which should have been guaranteed by the verifier; we already use 'insn[1]' elsewhere in the file for similar purposes. Additionally, we don't check if 'insn[1]' is actually a zext for our load-acquire's dst_reg, or some other registers - in other words, here we are relying on the verifier to always insert a redundant zext right after a 8/16/32-bit load-acquire, for its dst_reg. Signed-off-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Instead of open-coding the conditions, use '#ifdef CAN_USE_LOAD_ACQ_STORE_REL' to guard the following tests: verifier_precision/bpf_load_acquire verifier_precision/bpf_store_release verifier_store_release/* Note that, for the first two tests in verifier_precision.c, switching to '#ifdef CAN_USE_LOAD_ACQ_STORE_REL' means also checking if '__clang_major__ >= 18', which has already been guaranteed by the outer '#if' check. Signed-off-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Currently, we pass 0x1234567890abcdef to __retval() for the following two tests: verifier_load_acquire/load_acquire_64 verifier_store_release/store_release_64 However, the upper 32 bits of that value are being ignored, since __retval() expects an int. Actually, the tests would still pass even if I change '__retval(0x1234567890abcdef)' to e.g. '__retval(0x90abcdef)'. Restructure the tests a bit to test the entire 64-bit values properly. Do the same to their 8-, 16- and 32-bit variants as well to keep the style consistent. Fixes: ff3afe5 ("selftests/bpf: Add selftests for load-acquire and store-release instructions") Signed-off-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Verify that 8-, 16- and 32-bit load-acquires are zero-extending by using immediate values with their highest bit set. Do the same for the 64-bit variant to keep the style consistent. Signed-off-by: Peilin Ye <yepeilin@google.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
…for riscv64
For riscv64, enable all BPF_{LOAD_ACQ,STORE_REL} selftests except the
arena_atomics/* ones (not guarded behind CAN_USE_LOAD_ACQ_STORE_REL),
since arena access is not yet supported.
Signed-off-by: Peilin Ye <yepeilin@google.com>
Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 1: "[bpf-next,1/8] bpf/verifier: Handle BPF_LOAD_ACQ instructions in insn_def_regno()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 2: "[bpf-next,2/8] bpf, riscv64: Introduce emit_load_() and emit_store_()" |
|
Patch 6: "[bpf-next,6/8] selftests/bpf: Avoid passing out-of-range values to __retval()" |
|
Patch 6: "[bpf-next,6/8] selftests/bpf: Avoid passing out-of-range values to __retval()" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 7: "[bpf-next,7/8] selftests/bpf: Verify zero-extension behavior in load-acquire tests" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
|
Patch 8: "[bpf-next,8/8] selftests/bpf: Enable non-arena load-acquire/store-release selftests for riscv64" |
linux-riscv-bot
force-pushed
the
workflow__riscv__fixes
branch
2 times, most recently
from
4d9ad71 to
625be03
Compare
3 participants
PR for series 958320 applied to workflow__riscv__fixes
Name: bpf, riscv64: Support load-acquire and store-release instructions
URL: https://patchwork.kernel.org/project/linux-riscv/list/?series=958320
Version: 1