As discovered by Jarkko, taking mm->mmap_sem around EADD can lead to deadlock as attempting to acquire mmap_sem while holding encl->lock violates SGX's lock ordering. Resolving the issue simply by reversing the lock ordering gets ugly due to the behavior of sgx_encl_grow(), which has a path that drops encl->lock in order to do EPC page reclaim, i.e. moving mm->mmap_sem up would require it to be dropped and reacquired as well. Luckily, the entire flow can be simplified by preventing userspace from invoking concurrent ioctls on a single enclave. Requiring ioctls to be serialized means encl->lock doesn't need to be held to grow/shrink the enclave, since only ioctls can grow/shrink the enclave. This also eliminates theoretical cases that sgx_encl_grow() has to handle, e.g. the enclave being initialized while it's waiting on reclaim, since the protection provided by serializing ioctls isn't dropped to do reclaim. v2: - Make encl->flags an atomic, reuse for "in ioctl" detection. [Jarkko] - Drop smp_{r,w}mb() patch as it is superceded by atomic flags. [Jarkko] - Tack on two interdependent bug fixes found when auditing encl->flags. - Rebase to Jarkko's latest master. Sean Christopherson (5): x86/sgx: Convert encl->flags from an unsigned int to an atomic x86/sgx: Reject concurrent ioctls on single enclave x86/sgx: Take encl->lock inside of mm->mmap_sem for EADD x86/sgx: Reject all ioctls on dead enclaves x86/sgx: Destroy the enclave if EEXTEND fails arch/x86/kernel/cpu/sgx/driver.c | 3 +- arch/x86/kernel/cpu/sgx/encl.c | 18 ++-- arch/x86/kernel/cpu/sgx/encl.h | 3 +- arch/x86/kernel/cpu/sgx/ioctl.c | 163 ++++++++++++++++-------------- arch/x86/kernel/cpu/sgx/reclaim.c | 10 +- 5 files changed, 105 insertions(+), 92 deletions(-) -- 2.22.0