On Fri, Aug 02, 2019 at 06:06:14PM +0200, Vlastimil Babka wrote:
> The x86 version of get_user_pages_fast() relies on disabled interrupts to
> synchronize gup_pte_range() between gup_get_pte(ptep); and get_page() against
> a parallel munmap. The munmap side nulls the pte, then flushes TLBs, then
> releases the page. As TLB flush is done synchronously via IPI disabling
> interrupts blocks the page release, and get_page(), which assumes existing
> reference on page, is thus safe.
> However when TLB flush is done by a hypercall, e.g. in a Xen PV guest, there is
> no blocking thanks to disabled interrupts, and get_page() can succeed on a page
> that was already freed or even reused.
>
> We have recently seen this happen with our 4.4 and 4.12 based kernels, with
> userspace (java) that exits a thread, where mm_release() performs a futex_wake()
> on tsk->clear_child_tid, and another thread in parallel unmaps the page where
> tsk->clear_child_tid points to. The spurious get_page() succeeds, but futex code
> immediately releases the page again, while it's already on a freelist. Symptoms
> include a bad page state warning, general protection faults acessing a poisoned
> list prev/next pointer in the freelist, or free page pcplists of two cpus joined
> together in a single list. Oscar has also reproduced this scenario, with a
> patch inserting delays before the get_page() to make the race window larger.
>
> Fix this by removing the dependency on TLB flush interrupts the same way as the
> generic get_user_pages_fast() code by using page_cache_add_speculative() and
> revalidating the PTE contents after pinning the page. Mainline is safe since
> 4.13 where the x86 gup code was removed in favor of the common code. Accessing
> the page table itself safely also relies on disabled interrupts and TLB flush
> IPIs that don't happen with hypercalls, which was acknowledged in commit
> 9e52fc2b50de ("x86/mm: Enable RCU based page table freeing
> (CONFIG_HAVE_RCU_TABLE_FREE=y)"). That commit with follups should also be
> backported for full safety, although our reproducer didn't hit a problem
> without that backport.
>
> Reproduced-by: Oscar Salvador <osalvador@xxxxxxx>
> Signed-off-by: Vlastimil Babka <vbabka@xxxxxxx>
> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>
> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Cc: Juergen Gross <jgross@xxxxxxxx>
> Cc: Kirill A. Shutemov <kirill.shutemov@xxxxxxxxxxxxxxx>
> Cc: Vitaly Kuznetsov <vkuznets@xxxxxxxxxx>
> Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
> Cc: Borislav Petkov <bp@xxxxxxxxx>
> Cc: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
> Cc: Andy Lutomirski <luto@xxxxxxxxxx>
> ---
>
> Hi, I'm sending this stable-only patch for consideration because it's probably
> unrealistic to backport the 4.13 switch to generic GUP. I can look at 4.4 and
> 3.16 if accepted. The RCU page table freeing could be also considered.
> Note the patch also includes page refcount protection. I found out that
> 8fde12ca79af ("mm: prevent get_user_pages() from overflowing page refcount")
> backport to 4.9 missed the arch-specific gup implementations:
> https://lore.kernel.org/lkml/6650323f-dbc9-f069-000b-f6b0f941a065@xxxxxxx/
This looks sane to me, thank you for the backport. I've queued it up
now, and if anyone has any objections, please let me know.
thanks,
greg k-h
