On Thu, Feb 20, 2025 at 1:16 PM Eric Biggers <ebiggers@xxxxxxxxxx> wrote: > > From: Eric Biggers <ebiggers@xxxxxxxxxx> > > Currently kernel-mode FPU is not always usable in softirq context on > x86, since softirqs can nest inside a kernel-mode FPU section in task > context, and nested use of kernel-mode FPU is not supported. > > Therefore, x86 SIMD-optimized code that can be called in softirq context > has to sometimes fall back to non-SIMD code. There are two options for > the fallback, both of which are pretty terrible: > > (a) Use a scalar fallback. This can be 10-100x slower than vectorized > code because it cannot use specialized instructions like AES, SHA, > or carryless multiplication. > > (b) Execute the request asynchronously using a kworker. In other > words, use the "crypto SIMD helper" in crypto/simd.c. > > Currently most of the x86 en/decryption code (skcipher and aead > algorithms) uses option (b), since this avoids the slow scalar fallback > and it is easier to wire up. But option (b) is still really bad for its > own reasons: > > - Punting the request to a kworker is bad for performance too. > > - It forces the algorithm to be marked as asynchronous > (CRYPTO_ALG_ASYNC), preventing it from being used by crypto API > users who request a synchronous algorithm. That's another huge > performance problem, which is especially unfortunate for users who > don't even do en/decryption in softirq context. > > - It makes all en/decryption operations take a detour through > crypto/simd.c. That involves additional checks and an additional > indirect call, which slow down en/decryption for *everyone*. Thank you for the detailed information. > Fortunately, the skcipher and aead APIs are only usable in task and > softirq context in the first place, nor is it supported to call them > with hardirqs disabled. Thus, if kernel-mode FPU were to be reliably > usable in softirq context, no fallback would be needed. Indeed, other > architectures such as arm, arm64, and riscv have already done this. > > Therefore, this patch updates x86 accordingly to reliably support > kernel-mode FPU in softirqs (except when hardirqs are disabled). > > This is done by just disabling softirq processing in kernel-mode FPU > sections, as that prevents the nesting that was problematic. > > This will delay some softirqs slightly, but only ones that would have > otherwise been nested inside a task context kernel-mode FPU section. > Any such softirqs would have taken the slow fallback path before if they > tried to do any en/decryption. Now these softirqs will just run at the > end of the task context kernel-mode FPU section (since local_bh_enable() > runs pending softirqs) and will no longer take the slow fallback path. I think this will delay all softirqs, including those that don't use FPU. Will there be a performance impact? (I guess you've noticed the patch I submitted last year. And this is the main reason why it was implemented in the way you mentioned as the second alternative.)
