This series fixes the RCU stalls triggered by the x86 crypto modules discussed in https://lore.kernel.org/all/MW5PR84MB18426EBBA3303770A8BC0BDFAB759@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/ Two root causes were: - too much data processed between kernel_fpu_begin and kernel_fpu_end calls (which are heavily used by the x86 optimized drivers) - tcrypt not calling cond_resched during speed test loops These problems have always been lurking, but improving the loading of the x86/sha512 module led to it happening a lot during boot when using SHA-512 for module signature checking. Fixing these problems makes it safer to improve loading the rest of the x86 modules like the sha512 module. This series only handles the x86 modules. Testing ======= The most effective testing was by enabling CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y which creates random test vectors and compares the results of the CPU-optimized function to the generic function, and running two threads of repeated modprobe commands to exercise those tests: watch -n 0 modprobe tcrypt mode=200 watch -n 0 ./tcrypt_sweep where tcrypt_sweep walks through all the test modes: #!/usr/bin/perl use strict; my @modes; open SOURCE, "<", "/home/me/linux/crypto/tcrypt.c" or die $!; while (<SOURCE>) { if (/^\s+case ([0-9]+):$/) { push @modes, $1; } } close SOURCE; foreach (@modes) { print "$_ "; # don't run mode 300, which runs 301-399 # don't run mode 400, which runs 401-499 if (($_ eq "0") || ($_ eq "300") || ($_ eq "400")) { system "echo \"===== Skipping special modprobe tcrypt mode=$_\" > /dev/kmsg"; } else { system "echo \"Running modprobe tcrypt mode=$_\" > /dev/kmsg"; system "modprobe tcrypt mode=$_"; } } Robert Elliott (19): crypto: tcrypt - test crc32 crypto: tcrypt - test nhpoly1305 crypto: tcrypt - reschedule during cycles speed tests crypto: x86/sha - limit FPU preemption crypto: x86/crc - limit FPU preemption crypto: x86/sm3 - limit FPU preemption crypto: x86/ghash - restructure FPU context saving crypto: x86/ghash - limit FPU preemption crypto: x86 - use common macro for FPU limit crypto: x86/sha1, sha256 - load based on CPU features crypto: x86/crc - load based on CPU features crypto: x86/sm3 - load based on CPU features crypto: x86/ghash - load based on CPU features crypto: x86 - load based on CPU features crypto: x86 - add pr_fmt to all modules crypto: x86 - print CPU optimized loaded messages crypto: x86 - standardize suboptimal prints crypto: x86 - standardize not loaded prints crypto: x86/sha - register only the best function arch/x86/crypto/aegis128-aesni-glue.c | 21 ++- arch/x86/crypto/aesni-intel_glue.c | 31 ++-- arch/x86/crypto/aria_aesni_avx_glue.c | 19 +- arch/x86/crypto/blake2s-glue.c | 34 +++- arch/x86/crypto/blowfish_glue.c | 19 +- arch/x86/crypto/camellia_aesni_avx2_glue.c | 25 ++- arch/x86/crypto/camellia_aesni_avx_glue.c | 24 ++- arch/x86/crypto/camellia_glue.c | 20 ++- arch/x86/crypto/cast5_avx_glue.c | 21 ++- arch/x86/crypto/cast6_avx_glue.c | 21 ++- arch/x86/crypto/chacha_glue.c | 35 +++- arch/x86/crypto/crc32-pclmul_asm.S | 6 +- arch/x86/crypto/crc32-pclmul_glue.c | 37 ++-- arch/x86/crypto/crc32c-intel_glue.c | 51 ++++-- arch/x86/crypto/crct10dif-pclmul_glue.c | 54 ++++-- arch/x86/crypto/curve25519-x86_64.c | 27 ++- arch/x86/crypto/des3_ede_glue.c | 16 +- arch/x86/crypto/ghash-clmulni-intel_glue.c | 40 +++-- arch/x86/crypto/nhpoly1305-avx2-glue.c | 27 ++- arch/x86/crypto/nhpoly1305-sse2-glue.c | 23 ++- arch/x86/crypto/poly1305_glue.c | 64 +++++-- arch/x86/crypto/polyval-clmulni_glue.c | 14 +- arch/x86/crypto/serpent_avx2_glue.c | 25 ++- arch/x86/crypto/serpent_avx_glue.c | 21 ++- arch/x86/crypto/serpent_sse2_glue.c | 19 +- arch/x86/crypto/sha1_ssse3_glue.c | 188 +++++++++++-------- arch/x86/crypto/sha256_ssse3_glue.c | 198 ++++++++++++--------- arch/x86/crypto/sha512_ssse3_glue.c | 154 +++++++++------- arch/x86/crypto/sm3_avx_glue.c | 52 +++++- arch/x86/crypto/sm4_aesni_avx2_glue.c | 25 ++- arch/x86/crypto/sm4_aesni_avx_glue.c | 23 ++- arch/x86/crypto/twofish_avx_glue.c | 25 ++- arch/x86/crypto/twofish_glue.c | 19 +- arch/x86/crypto/twofish_glue_3way.c | 26 ++- crypto/tcrypt.c | 56 +++--- 35 files changed, 1060 insertions(+), 400 deletions(-) -- 2.37.3
