On 10 June 2017 at 18:22, Ard Biesheuvel <ard.biesheuvel@xxxxxxxxxx> wrote: > TL;DR: preparatory work for expected changes in arm64's handling of kernel > mode SIMD > > @Herbert: The arm64 maintainers may want to take this through the arm64 tree, > and if not, we need their acks on patch #1. Thanks. > Please disregard this for merging for now. I am looking into whether it is possible to use time invariant fallbacks instead for the AES routines (and I missed a couple of AES MAC routines in the conversion) > Currently, arm64 allows kernel mode NEON (KMN) in process, softirq or hardirq > context. In the process case, we preserve/restore the NEON context lazily, > but in the softirq/hardirq cases, we eagerly stash a slice of the NEON > register file, and immediately restore it when kernel_neon_end() is called. > > Given the above, arm64 actually does not use the generic may_use_simd() API > at all*, which was added to allow async wrappers of synchronous SIMD routines > to be implemented in a generic manner. (On x86, kernel mode SIMD may be used > in process context or while serving an interrupt taken from user space. On ARM, > SIMD may only be used in process context) > > When adding support for the SVE architecture extension, which shared part of > the NEON register file with the SIMD and crypto extensions, the eager preserve/ > restore in interrupt context is becoming a problem: it should either preserve > and restore the entire SVE state (which may be up to 8 KB in size), or it > should not be allowed to interrupt the lazy preserve, which does need to deal > with the large SVE state anyway. Otherwise, such an interruption would corrupt > the NEON state the lazy preserve sees after the interruption. > > Given how > a) KMN is never actually used in hardirq context, > b) KMN is only used in softirq context by mac80211 code running on behalf of > WiFi devices that don't perform the crypto in hardware, > b) KMN in softirq context is statistically unlikely to interrupt the kernel > while it is doing kernel mode NEON in process context, > > the unconditional eager preserve/restore typically executes when no KMN in > process context is in progress, and we can simplify things substantially by > disallowing nested KMN, i.e., disallow KMN in hardirq context, and allow KMN > in softirq only if no KMN in process context is already in progress. > > The no-nesting rule leaves only the outer SVE-aware lazy preserve/restore, > which needs to execute with bottom halves disabled, but other than that, no > intrusive changes should be needed to deal with the SVE payloads. > > Given that the no-nesting rule implies that SIMD is no longer allowed in any > context, the KMN users need to be made aware of this. This series updates the > current KMN users in the arm64 tree to take may_use_simd() into account. Since > at this time, SIMD is still allowed in any context, an implementation of > may_use_simd() is added that simply returns true (#1). It will be updated in > the future when the no-nesting modifications are made. > > * may_use_simd() is only used as a hint in the SHA256 NEON code, since on some > microarchitectures, it is only marginally faster, and the eager preserve and > restore could actually make it slower. > > Ard Biesheuvel (12): > arm64: neon: replace generic definition of may_use_simd() > crypto: arm64/ghash-ce - add non-SIMD scalar fallback > crypto: arm64/crct10dif - add non-SIMD generic fallback > crypto: arm64/crc32 - add non-SIMD scalar fallback > crypto: arm64/sha1-ce - add non-SIMD generic fallback > crypto: arm64/sha2-ce - add non-SIMD scalar fallback > crypto: arm64/aes-ce-cipher - match round key endianness with generic > code > crypto: arm64/aes-ce-cipher: add non-SIMD generic fallback > crypto: arm64/aes-ce-ccm: add non-SIMD generic fallback > crypto: arm64/aes-blk - add a non-SIMD fallback for synchronous CTR > crypto: arm64/chacha20 - take may_use_simd() into account > crypto: arm64/aes-bs - implement non-SIMD fallback for AES-CTR > > arch/arm64/crypto/Kconfig | 22 ++- > arch/arm64/crypto/aes-ce-ccm-core.S | 30 ++-- > arch/arm64/crypto/aes-ce-ccm-glue.c | 152 +++++++++++++++----- > arch/arm64/crypto/aes-ce-cipher.c | 55 ++++--- > arch/arm64/crypto/aes-ce.S | 12 +- > arch/arm64/crypto/aes-ctr-fallback.h | 55 +++++++ > arch/arm64/crypto/aes-glue.c | 17 ++- > arch/arm64/crypto/aes-neonbs-glue.c | 48 ++++++- > arch/arm64/crypto/chacha20-neon-glue.c | 5 +- > arch/arm64/crypto/crc32-ce-glue.c | 11 +- > arch/arm64/crypto/crct10dif-ce-glue.c | 13 +- > arch/arm64/crypto/ghash-ce-glue.c | 49 +++++-- > arch/arm64/crypto/sha1-ce-glue.c | 18 ++- > arch/arm64/crypto/sha2-ce-glue.c | 30 +++- > arch/arm64/crypto/sha256-glue.c | 1 + > arch/arm64/include/asm/Kbuild | 1 - > arch/arm64/include/asm/simd.h | 24 ++++ > 17 files changed, 420 insertions(+), 123 deletions(-) > create mode 100644 arch/arm64/crypto/aes-ctr-fallback.h > create mode 100644 arch/arm64/include/asm/simd.h > > -- > 2.7.4 >
