As discussed and suggested by Linus use a seqcount to close the small race between gup_fast and copy_page_range(). Ahmed confirms that raw_write_seqcount_begin() is the correct API to use in this case and it doesn't trigger any lockdeps. I was able to test it using two threads, one forking and the other using ibv_reg_mr() to trigger GUP fast. Modifying copy_page_range() to sleep made the window large enough to reliably hit to test the logic. v4: - Use read_seqcount_retry() not read_seqcount_t_retry v3: https://lore.kernel.org/r/0-v3-7358966cab09+14e9-gup_fork_jgg@xxxxxxxxxx - Revise comment for write_protect_seq - Revise comment in copy_page_range - Use raw_write_seqcount_begin() not raw_write_seqcount_t_begin() v2: https://lore.kernel.org/r/0-v2-dfe9ecdb6c74+2066-gup_fork_jgg@xxxxxxxxxx - Use start not addr in lockless_pages_from_mm - Replace unsigned long casts with using the proper variable type - Update comments - Use raw_write_seqcount_t_begin() instead of open coding - Update commit messages v1: https://lore.kernel.org/r/0-v1-281e425c752f+2df-gup_fork_jgg@xxxxxxxxxx To: linux-kernel@xxxxxxxxxxxxxxx To: Peter Xu <peterx@xxxxxxxxxx> To: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> Cc: Peter Xu <peterx@xxxxxxxxxx> Cc: John Hubbard <jhubbard@xxxxxxxxxx> Cc: Linux-MM <linux-mm@xxxxxxxxx> Cc: Linux Kernel Mailing List <linux-kernel@xxxxxxxxxxxxxxx> Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> Cc: Jan Kara <jack@xxxxxxx> Cc: Michal Hocko <mhocko@xxxxxxxx> Cc: Kirill Tkhai <ktkhai@xxxxxxxxxxxxx> Cc: Kirill Shutemov <kirill@xxxxxxxxxxxxx> Cc: Hugh Dickins <hughd@xxxxxxxxxx> Cc: Christoph Hellwig <hch@xxxxxx> Cc: Andrea Arcangeli <aarcange@xxxxxxxxxx> Cc: Oleg Nesterov <oleg@xxxxxxxxxx> Cc: Jann Horn <jannh@xxxxxxxxxx> Cc: "Ahmed S. Darwish" <a.darwish@xxxxxxxxxxxxx> Jason Gunthorpe (2): mm: reorganize internal_get_user_pages_fast() mm: prevent gup_fast from racing with COW during fork arch/x86/kernel/tboot.c | 1 + drivers/firmware/efi/efi.c | 1 + include/linux/mm_types.h | 8 +++ kernel/fork.c | 1 + mm/gup.c | 117 +++++++++++++++++++++++-------------- mm/init-mm.c | 1 + mm/memory.c | 13 ++++- 7 files changed, 96 insertions(+), 46 deletions(-) -- 2.29.2
