[PW_SID:1071424] [01/26] xor: assert that xor_blocks is not from preemptible user context#1663
[PW_SID:1071424] [01/26] xor: assert that xor_blocks is not from preemptible user context#1663linux-riscv-bot wants to merge 26 commits into
Conversation
Most of the optimized xor_blocks versions require FPU/vector registers, which generally are not supported in interrupt context. Both callers already are in user context, so enforce this at the highest level. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
xor_blocks can't be called from interrupt context, so remove the handling for that. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Since commit c055e3e ("crypto: xor - use ktime for template benchmarking") the benchmarking works just fine even for TT_MODE_INFCPU, so drop the workarounds. Note that for CPUs supporting AVX2, which includes almost everything built in the last 10 years, the AVX2 implementation is forced anyway. CONFIG_X86_32 is always correctly set for UM in arch/x86/um/Kconfig, so don't override it either. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Richard Weinberger <richard@nod.at> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the RAID XOR code to lib/raid/ as it has nothing to do with the crypto API. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Update the to of file comment to be correct and non-redundant, and drop the unused BH_TRACE define. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Originally, the XOR code benchmarked all algorithms at load time, but it has since then been hacked multiple times to allow forcing an algorithm, and then commit 524ccdb ("crypto: xor - defer load time benchmark to a later time") changed the logic to a two-step process or registration and benchmarking, but only when built-in. Rework this, so that the XOR_TRY_TEMPLATES macro magic now always just deals with adding the templates to the list, and benchmarking is always done in a second pass; for modular builds from module_init, and for the built-in case using a separate init call level. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Keep xor.h for the public API, and split the struct xor_block_template definition that is only needed by the xor.ko core and architecture-specific optimizations into a separate xor_impl.h header. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Drop the pretty confusing historic XOR_TRY_TEMPLATES and XOR_SELECT_TEMPLATE, and instead let the architectures provide a arch_xor_init that calls either xor_register to register candidates or xor_force to force a specific implementation. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the generic implementations from asm-generic/xor.h to per-implementaion .c files in lib/raid. This will build them unconditionally even when an architecture forces a specific implementation, but as we'll need at least one generic version for the static_call optimization later on we'll pay that price. Note that this would cause the second xor_block_8regs instance created by arch/arm/lib/xor-neon.c to be generated instead of discarded as dead code, so add a NO_TEMPLATE symbol to disable it for this case. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR code out of line into lib/raid. Note that the giant inline assembly block might be better off as a separate assembly source file now, but I'll leave that to the alpha maintainers. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Magnus Lindholm <linmag7@gmail.com> Tested-by: Magnus Lindholm <linmag7@gmail.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in the main xor.ko instead of building a separate module for it. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in the main xor.ko instead of building a separate module for it. Note that this drops the CONFIG_KERNEL_MODE_NEON dependency, as that is always set for arm64. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in xor.ko instead of always building it into the main kernel image. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in xor.ko instead of always building it into the main kernel image. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in xor.ko instead of always building it into the main kernel image. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in xor.ko instead of always building it into the main kernel image. The code should probably be split into separate files for the two implementations, but for now this just does the trivial move. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR into lib/raid and include it it in xor.ko instead of unconditionally building it into the main kernel image. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the optimized XOR code out of line into lib/raid. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Remove the inner xor_block_templates, and instead have two separate actual template that call into the neon-enabled compilation unit. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Move the asm/xor.h headers to lib/raid/xor/$(SRCARCH)/xor_arch.h and include/linux/raid/xor_impl.h to lib/raid/xor/xor_impl.h so that the xor.ko module implementation is self-contained in lib/raid/. As this remove the asm-generic mechanism a new kconfig symbol is added to indicate that a architecture-specific implementations exists, and xor_arch.h should be included. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
xor_blocks is very annoying to use, because it is limited to 4 + 1 sources / destinations, has an odd argument order and is completely undocumented. Lift the code that loops around it from btrfs and async_tx/async_xor into common code under the name xor_gen and properly document it. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Replace use of the loop around xor_blocks with the easier to use xor_gen API. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Use the new xor_gen helper instead of open coding the loop around xor_blocks. This helper is very similar to the existing run_xor helper in btrfs, except that the destination buffer is passed explicitly. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: David Sterba <dsterba@suse.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Currently the high-level xor code chunks up all operations into small units for only up to 1 + 4 vectors, and passes it to four different methods. This means the FPU/vector context is entered and left a lot for wide stripes, and a lot of indirect expensive indirect calls are performed. Switch to passing the entire gen_xor request to the low-level ops, and provide a macro to dispatch it to the existing helper. This reduce the number of indirect calls and FPU/vector context switches by a factor approaching nr_stripes / 4, and also reduces source and binary code size. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Avoid the indirect call for xor_generation by using a static_call. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Add a test case for the XOR routines loosely based on the CRC kunit test. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
|
Patch 1: "[01/26] xor: assert that xor_blocks is not from preemptible user context" |
|
Patch 1: "[01/26] xor: assert that xor_blocks is not from preemptible user context" |
|
Patch 1: "[01/26] xor: assert that xor_blocks is not from preemptible user context" |
|
Patch 1: "[01/26] xor: assert that xor_blocks is not from preemptible user context" |
|
Patch 24: "[24/26] xor: pass the entire operation to the low-level ops" |
|
Patch 24: "[24/26] xor: pass the entire operation to the low-level ops" |
|
Patch 24: "[24/26] xor: pass the entire operation to the low-level ops" |
|
Patch 24: "[24/26] xor: pass the entire operation to the low-level ops" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 25: "[25/26] xor: use static_call for xor_gen" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
|
Patch 26: "[26/26] xor: add a kunit test case" |
PR for series 1071424 applied to workflow__riscv__fixes
Name: [01/26] xor: assert that xor_blocks is not from preemptible user context
URL: https://patchwork.kernel.org/project/linux-riscv/list/?series=1071424
Version: 1