When per-VMA locks were introduced in [1] several types of page faults would still fall back to mmap_lock to keep the patchset simple. Among them are swap and userfault pages. The main reason for skipping those cases was the fact that mmap_lock could be dropped while handling these faults and that required additional logic to be implemented. Implement the mechanism to allow per-VMA locks to be dropped for these cases. First, change handle_mm_fault to drop per-VMA locks when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED to be consistent with the way mmap_lock is handled. Then change folio_lock_or_retry to accept vm_fault and return vm_fault_t which simplifies later patches. Finally allow swap and uffd page faults to be handled under per-VMA locks by dropping per-VMA and retrying, the same way it's done under mmap_lock. Naturally, once VMA lock is dropped that VMA should be assumed unstable and can't be used. Changes since v3 posted at [2] - Renamed folio_lock_or_retry back to folio_lock_fault, per Peter Xu - Moved per-VMA lock release to where VM_FAULT_RETRY is returned, per Peter Xu - Dropped FAULT_FLAG_LOCK_DROPPED usage, per Peter Xu - Introduced release_fault_lock() helper function, per Peter Xu - Dropped the patch releasing per-VMA lock before migration_entry_wait, per Peter Xu - Introduced assert_fault_locked() helper function, per Peter Xu - Added BUG_ON to prevent FAULT_FLAG_RETRY_NOWAIT usage with per-VMA locks Note: patch 3/8 will cause a trivial merge conflict in arch/arm64/mm/fault.c when applied over mm-unstable branch due to a patch from ARM64 tree [3] which is missing in mm-unstable. [1] https://lore.kernel.org/all/20230227173632.3292573-1-surenb@xxxxxxxxxx/ [2] https://lore.kernel.org/all/20230627042321.1763765-1-surenb@xxxxxxxxxx/ [3] https://lore.kernel.org/all/20230524131305.2808-1-jszhang@xxxxxxxxxx/ Suren Baghdasaryan (6): swap: remove remnants of polling from read_swap_cache_async mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED mm: drop per-VMA lock when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED mm: change folio_lock_or_retry to use vm_fault directly mm: handle swap page faults under per-VMA lock mm: handle userfaults under VMA lock arch/arm64/mm/fault.c | 3 ++- arch/powerpc/mm/fault.c | 3 ++- arch/s390/mm/fault.c | 3 ++- arch/x86/mm/fault.c | 3 ++- fs/userfaultfd.c | 39 ++++++++++++++++++--------------------- include/linux/mm.h | 39 +++++++++++++++++++++++++++++++++++++++ include/linux/mm_types.h | 3 ++- include/linux/pagemap.h | 9 ++++----- mm/filemap.c | 37 +++++++++++++++++++------------------ mm/madvise.c | 4 ++-- mm/memory.c | 38 ++++++++++++++++---------------------- mm/swap.h | 1 - mm/swap_state.c | 12 +++++------- 13 files changed, 113 insertions(+), 81 deletions(-) -- 2.41.0.162.gfafddb0af9-goog
