On Tue, Aug 11, 2020 at 09:07:17PM +0200, Jann Horn wrote:
> On Tue, Aug 11, 2020 at 8:39 PM Peter Xu <peterx@xxxxxxxxxx> wrote:
> > Starting from commit 17839856fd58 ("gup: document and work around "COW can
> > break either way" issue", 2020-06-02), explicit copy-on-write behavior is
> > enforced for private gup pages even if it's a read-only. It is achieved by
> > always passing FOLL_WRITE to emulate a write.
> >
> > That should fix the COW issue that we were facing, however above commit could
> > also break userfaultfd-wp and applications like umapsort [1,2].
> >
> > One general routine of umap-like program is: userspace library will manage page
> > allocations, and it will evict the least recently used pages from memory to
> > external storages (e.g., file systems). Below are the general steps to evict
> > an in-memory page in the uffd service thread when the page pool is full:
> >
> > (1) UFFDIO_WRITEPROTECT with mode=WP on some to-be-evicted page P, so that
> > further writes to page P will block (keep page P clean)
> > (2) Copy page P to external storage (e.g. file system)
> > (3) MADV_DONTNEED to evict page P
> >
> > Here step (1) makes sure that the page to dump will always be up-to-date, so
> > that the page snapshot in the file system is consistent with the one that was
> > in the memory. However with commit 17839856fd58, step (2) can potentially hang
> > itself because e.g. if we use write() to a file system fd to dump the page
> > data, that will be a translated read gup request in the file system driver to
> > read the page content, then the read gup will be translated to a write gup due
> > to the new enforced COW behavior. This write gup will further trigger
> > handle_userfault() and hang the uffd service thread itself.
> >
> > I think the problem will go away too if we replace the write() to the file
> > system into a memory write to a mmaped region in the userspace library, because
> > normal page faults will not enforce COW, only gup is affected. However we
> > cannot forbid users to use write() or any form of kernel level read gup.
> >
> > One solution is actually already mentioned in commit 17839856fd58, which is to
> > provide an explicit BREAK_COW scemantics for enforced COW. Then we can still
> > use FAULT_FLAG_WRITE to identify whether this is a "real write request" or an
> > "enfornced COW (read) request".
> >
> > With the enforced COW, we also need to inherit UFFD_WP bit during COW because
> > now COW can happen with UFFD_WP enabled (previously, it cannot).
> >
> > Since at it, rename the variable in __handle_mm_fault() from "dirty" to "cow"
> > to better suite its functionality.
> [...]
> > diff --git a/mm/memory.c b/mm/memory.c
> [...]
> > + * copied due to enfornced COW. When it happens, we
>
> (typo here and in the huge_memory version)
Right..
>
> [...]
> > diff --git a/mm/gup.c b/mm/gup.c
> > index d8a33dd1430d..c33e84ab9c36 100644
> > --- a/mm/gup.c
> > +++ b/mm/gup.c
> > @@ -870,6 +870,8 @@ static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma,
> > return -ENOENT;
> > if (*flags & FOLL_WRITE)
> > fault_flags |= FAULT_FLAG_WRITE;
> > + if (*flags & FOLL_BREAK_COW)
> > + fault_flags |= FAULT_FLAG_BREAK_COW;
> > if (*flags & FOLL_REMOTE)
> > fault_flags |= FAULT_FLAG_REMOTE;
> > if (locked)
> > @@ -1076,7 +1078,7 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
> > }
> > if (is_vm_hugetlb_page(vma)) {
> > if (should_force_cow_break(vma, foll_flags))
> > - foll_flags |= FOLL_WRITE;
> > + foll_flags |= FOLL_BREAK_COW;
>
> How does this interact with the FOLL_WRITE check in follow_page_pte()?
> If we want the COW to be broken, follow_page_pte() would have to treat
> FOLL_BREAK_COW similarly to FOLL_WRITE, right?
Good point... I did checked follow_page_mask() that FOLL_COW will still be set
correctly after applying the patch, though I forgot the FOLL_WRITE part.
Does below look good to you?
diff --git a/mm/gup.c b/mm/gup.c
index 9d1f44b01165..f4f2a69c6fe7 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -439,7 +439,8 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
}
if ((flags & FOLL_NUMA) && pte_protnone(pte))
goto no_page;
- if ((flags & FOLL_WRITE) && !can_follow_write_pte(pte, flags)) {
+ if ((flags & (FOLL_WRITE | FOLL_BREAK_COW)) &&
+ !can_follow_write_pte(pte, flags)) {
pte_unmap_unlock(ptep, ptl);
return NULL;
}
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 4f192efef37c..edbd42c9d576 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1340,7 +1340,8 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma,
assert_spin_locked(pmd_lockptr(mm, pmd));
- if (flags & FOLL_WRITE && !can_follow_write_pmd(*pmd, flags))
+ if (flags & (FOLL_WRITE | FOLL_BREAK_COW) &&
+ !can_follow_write_pmd(*pmd, flags))
goto out;
/* Avoid dumping huge zero page */
Thanks,
--
Peter Xu
