On Wed, Oct 16, 2024 at 09:26:43PM +0200, Ard Biesheuvel wrote: > From: Ard Biesheuvel <ardb@xxxxxxxxxx> > > Now that kernel mode NEON no longer disables preemption, using FP/SIMD > in library code which is not obviously part of the crypto subsystem is > no longer problematic, as it will no longer incur unexpected latencies. > > So accelerate the CRC-32 library code on arm64 to use a 4-way > interleave, using PMULL instructions to implement the folding. > > On Apple M2, this results in a speedup of 2 - 2.8x when using input > sizes of 1k - 8k. For smaller sizes, the overhead of preserving and > restoring the FP/SIMD register file may not be worth it, so 1k is used > as a threshold for choosing this code path. > > The coefficient tables were generated using code provided by Eric. [0] > > [0] https://github.com/ebiggers/libdeflate/blob/master/scripts/gen_crc32_multipliers.c > > Cc: Eric Biggers <ebiggers@xxxxxxxxxx> > Signed-off-by: Ard Biesheuvel <ardb@xxxxxxxxxx> > --- > arch/arm64/lib/Makefile | 2 +- > arch/arm64/lib/crc32-4way.S | 242 ++++++++++++++++++++ > arch/arm64/lib/crc32-glue.c | 48 ++++ > 3 files changed, 291 insertions(+), 1 deletion(-) Reviewed-by: Eric Biggers <ebiggers@xxxxxxxxxx> > + /* Process up to 64 blocks of 64 bytes at a time */ > +.La\@: mov x3, #64 > + cmp len, #64 > + csel x3, x3, len, hi // x3 := max(len, 64) The comment should say min(len, 64), not max(len, 64). - Eric
