On Wed, Oct 12, 2022 at 04:59:16PM -0500, Robert Elliott wrote:
> As done by the ECB and CBC helpers in arch/x86/crypt/ecb_cbc_helpers.h,
> limit the number of bytes processed between kernel_fpu_begin() and
> kernel_fpu_end() calls.
>
> Those functions call preempt_disable() and preempt_enable(), so
> the CPU core is unavailable for scheduling while running.
>
> This leads to "rcu_preempt detected expedited stalls" with stack dumps
> pointing to the optimized hash function if the module is loaded and
> used a lot:
> rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: ...
>
> For example, that can occur during boot with the stack track pointing
> to the sha512-x86 function if the system set to use SHA-512 for
> module signing. The call trace includes:
> module_sig_check
> mod_verify_sig
> pkcs7_verify
> pkcs7_digest
> sha512_finup
> sha512_base_do_update
>
> Fixes: 66be89515888 ("crypto: sha1 - SSSE3 based SHA1 implementation for x86-64")
> Fixes: 8275d1aa6422 ("crypto: sha256 - Create module providing optimized SHA256 routines using SSSE3, AVX or AVX2 instructions.")
> Fixes: 87de4579f92d ("crypto: sha512 - Create module providing optimized SHA512 routines using SSSE3, AVX or AVX2 instructions.")
> Fixes: aa031b8f702e ("crypto: x86/sha512 - load based on CPU features")
> Suggested-by: Herbert Xu <herbert@xxxxxxxxxxxxxxxxxxx>
> Reviewed-by: Tim Chen <tim.c.chen@xxxxxxxxxxxxxxx>
> Signed-off-by: Robert Elliott <elliott@xxxxxxx>
> ---
> arch/x86/crypto/sha1_ssse3_glue.c | 32 ++++++++++++++++++++++++-----
> arch/x86/crypto/sha256_ssse3_glue.c | 32 ++++++++++++++++++++++++-----
> arch/x86/crypto/sha512_ssse3_glue.c | 32 ++++++++++++++++++++++++-----
> 3 files changed, 81 insertions(+), 15 deletions(-)
>
> diff --git a/arch/x86/crypto/sha1_ssse3_glue.c b/arch/x86/crypto/sha1_ssse3_glue.c
> index 44340a1139e0..a9f5779b41ca 100644
> --- a/arch/x86/crypto/sha1_ssse3_glue.c
> +++ b/arch/x86/crypto/sha1_ssse3_glue.c
> @@ -26,6 +26,8 @@
> #include <crypto/sha1_base.h>
> #include <asm/simd.h>
>
> +#define FPU_BYTES 4096U /* avoid kernel_fpu_begin/end scheduler/rcu stalls */
Declare this inside the function it's used as an untyped enum, and give
it a better name, like BYTES_PER_FPU.
