linux-stable.git/lib/raid/xor/arm, branch master

xor/arm: Replace vectorized implementation with arm64's intrinsics

2026-05-28T20:14:19+00:00

Drop the XOR implementation generated by the vectorizer: this has always
been a bit of a hack, and now that arm64 has an intrinsics version that
works on ARM too, let's use that instead.

So copy the part of the arm64 code that can be shared (so not the EOR3
version). The arm64 code will be updated in a subsequent patch to share
this implementation.

Performance (QEMU mach-virt VM running on Synquacer [Cortex-A53 @ 1 GHz]

Before:

[    3.519687] xor: measuring software checksum speed
[    3.521725]    neon            :  1660 MB/sec
[    3.524733]    32regs          :  1105 MB/sec
[    3.527751]    8regs           :  1098 MB/sec
[    3.529911]    arm4regs        :  1540 MB/sec

After:

[    3.517654] xor: measuring software checksum speed
[    3.519454]    neon            :  1896 MB/sec
[    3.522499]    32regs          :  1090 MB/sec
[    3.525560]    8regs           :  1083 MB/sec
[    3.527700]    arm4regs        :  1556 MB/sec

Signed-off-by: Ard Biesheuvel 
Acked-by: Christoph Hellwig 
Link: https://patch.msgid.link/20260422171655.3437334-12-ardb+git@google.com
Signed-off-by: Eric Biggers

xor: pass the entire operation to the low-level ops

2026-04-03T06:36:21+00:00

Currently the high-level xor code chunks up all operations into small
units for only up to 1 + 4 vectors, and passes it to four different
methods.  This means the FPU/vector context is entered and left a lot for
wide stripes, and a lot of indirect expensive indirect calls are
performed.  Switch to passing the entire gen_xor request to the low-level
ops, and provide a macro to dispatch it to the existing helper.

This reduce the number of indirect calls and FPU/vector context switches
by a factor approaching nr_stripes / 4, and also reduces source and binary
code size.

Link: https://lkml.kernel.org/r/20260327061704.3707577-27-hch@lst.de
Signed-off-by: Christoph Hellwig 
Reviewed-by: Eric Biggers 
Tested-by: Eric Biggers 
Cc: Albert Ou 
Cc: Alexander Gordeev 
Cc: Alexandre Ghiti 
Cc: Andreas Larsson 
Cc: Anton Ivanov 
Cc: Ard Biesheuvel 
Cc: Arnd Bergmann 
Cc: "Borislav Petkov (AMD)" 
Cc: Catalin Marinas 
Cc: Chris Mason 
Cc: Christian Borntraeger 
Cc: Dan Williams 
Cc: David S. Miller 
Cc: David Sterba 
Cc: Heiko Carstens 
Cc: Herbert Xu 
Cc: "H. Peter Anvin" 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jason A. Donenfeld 
Cc: Johannes Berg 
Cc: Li Nan 
Cc: Madhavan Srinivasan 
Cc: Magnus Lindholm 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Palmer Dabbelt 
Cc: Richard Henderson 
Cc: Richard Weinberger 
Cc: Russell King 
Cc: Song Liu 
Cc: Sven Schnelle 
Cc: Ted Ts'o 
Cc: Vasily Gorbik 
Cc: WANG Xuerui 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

xor: make xor.ko self-contained in lib/raid/

2026-04-03T06:36:20+00:00

Move the asm/xor.h headers to lib/raid/xor/$(SRCARCH)/xor_arch.h and
include/linux/raid/xor_impl.h to lib/raid/xor/xor_impl.h so that the
xor.ko module implementation is self-contained in lib/raid/.

As this remove the asm-generic mechanism a new kconfig symbol is added to
indicate that a architecture-specific implementations exists, and
xor_arch.h should be included.

Link: https://lkml.kernel.org/r/20260327061704.3707577-22-hch@lst.de
Signed-off-by: Christoph Hellwig 
Reviewed-by: Eric Biggers 
Tested-by: Eric Biggers 
Cc: Albert Ou 
Cc: Alexander Gordeev 
Cc: Alexandre Ghiti 
Cc: Andreas Larsson 
Cc: Anton Ivanov 
Cc: Ard Biesheuvel 
Cc: Arnd Bergmann 
Cc: "Borislav Petkov (AMD)" 
Cc: Catalin Marinas 
Cc: Chris Mason 
Cc: Christian Borntraeger 
Cc: Dan Williams 
Cc: David S. Miller 
Cc: David Sterba 
Cc: Heiko Carstens 
Cc: Herbert Xu 
Cc: "H. Peter Anvin" 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jason A. Donenfeld 
Cc: Johannes Berg 
Cc: Li Nan 
Cc: Madhavan Srinivasan 
Cc: Magnus Lindholm 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Palmer Dabbelt 
Cc: Richard Henderson 
Cc: Richard Weinberger 
Cc: Russell King 
Cc: Song Liu 
Cc: Sven Schnelle 
Cc: Ted Ts'o 
Cc: Vasily Gorbik 
Cc: WANG Xuerui 
Cc: Will Deacon 
Signed-off-by: Andrew Morton

arm: move the XOR code to lib/raid/

2026-04-03T06:36:18+00:00

Move the optimized XOR into lib/raid and include it it in the main xor.ko
instead of building a separate module for it.

Link: https://lkml.kernel.org/r/20260327061704.3707577-13-hch@lst.de
Signed-off-by: Christoph Hellwig 
Reviewed-by: Eric Biggers 
Tested-by: Eric Biggers 
Cc: Albert Ou 
Cc: Alexander Gordeev 
Cc: Alexandre Ghiti 
Cc: Andreas Larsson 
Cc: Anton Ivanov 
Cc: Ard Biesheuvel 
Cc: Arnd Bergmann 
Cc: "Borislav Petkov (AMD)" 
Cc: Catalin Marinas 
Cc: Chris Mason 
Cc: Christian Borntraeger 
Cc: Dan Williams 
Cc: David S. Miller 
Cc: David Sterba 
Cc: Heiko Carstens 
Cc: Herbert Xu 
Cc: "H. Peter Anvin" 
Cc: Huacai Chen 
Cc: Ingo Molnar 
Cc: Jason A. Donenfeld 
Cc: Johannes Berg 
Cc: Li Nan 
Cc: Madhavan Srinivasan 
Cc: Magnus Lindholm 
Cc: Matt Turner 
Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Palmer Dabbelt 
Cc: Richard Henderson 
Cc: Richard Weinberger 
Cc: Russell King 
Cc: Song Liu 
Cc: Sven Schnelle 
Cc: Ted Ts'o 
Cc: Vasily Gorbik 
Cc: WANG Xuerui 
Cc: Will Deacon 
Signed-off-by: Andrew Morton