On Wed, Jul 20, 2022 at 11:39:23AM +0200, David Hildenbrand wrote:
> On 19.07.22 22:47, Peter Xu wrote:
> > On Mon, Jul 18, 2022 at 05:01:59AM -0700, Nadav Amit wrote:
> >> From: Nadav Amit <namit@xxxxxxxxxx>
> >>
> >> When userfaultfd makes a PTE writable, it can now change the PTE
> >> directly, in some cases, without going triggering a page-fault first.
> >> Yet, doing so might leave the PTE that was write-unprotected as old and
> >> clean. At least on x86, this would cause a >500 cycles overhead when the
> >> PTE is first accessed.
> >>
> >> Use MM_CP_WILL_NEED to set the PTE as young and dirty when userfaultfd
> >> gets a hint that the page is likely to be used. Avoid changing the PTE
> >> to young and dirty in other cases to avoid excessive writeback and
> >> messing with the page reclamation logic.
> >>
> >> Cc: Andrea Arcangeli <aarcange@xxxxxxxxxx>
> >> Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> >> Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> >> Cc: Andy Lutomirski <luto@xxxxxxxxxx>
> >> Cc: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
> >> Cc: David Hildenbrand <david@xxxxxxxxxx>
> >> Cc: Peter Xu <peterx@xxxxxxxxxx>
> >> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> >> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> >> Cc: Will Deacon <will@xxxxxxxxxx>
> >> Cc: Yu Zhao <yuzhao@xxxxxxxxxx>
> >> Cc: Nick Piggin <npiggin@xxxxxxxxx>
> >> ---
> >> include/linux/mm.h | 2 ++
> >> mm/mprotect.c | 9 ++++++++-
> >> mm/userfaultfd.c | 8 ++++++--
> >> 3 files changed, 16 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/include/linux/mm.h b/include/linux/mm.h
> >> index 9cc02a7e503b..4afd75ce5875 100644
> >> --- a/include/linux/mm.h
> >> +++ b/include/linux/mm.h
> >> @@ -1988,6 +1988,8 @@ extern unsigned long move_page_tables(struct vm_area_struct *vma,
> >> /* Whether this change is for write protecting */
> >> #define MM_CP_UFFD_WP (1UL << 2) /* do wp */
> >> #define MM_CP_UFFD_WP_RESOLVE (1UL << 3) /* Resolve wp */
> >> +/* Whether to try to mark entries as dirty as they are to be written */
> >> +#define MM_CP_WILL_NEED (1UL << 4)
> >> #define MM_CP_UFFD_WP_ALL (MM_CP_UFFD_WP | \
> >> MM_CP_UFFD_WP_RESOLVE)
> >>
> >> diff --git a/mm/mprotect.c b/mm/mprotect.c
> >> index 996a97e213ad..34c2dfb68c42 100644
> >> --- a/mm/mprotect.c
> >> +++ b/mm/mprotect.c
> >> @@ -82,6 +82,7 @@ static unsigned long change_pte_range(struct mmu_gather *tlb,
> >> bool prot_numa = cp_flags & MM_CP_PROT_NUMA;
> >> bool uffd_wp = cp_flags & MM_CP_UFFD_WP;
> >> bool uffd_wp_resolve = cp_flags & MM_CP_UFFD_WP_RESOLVE;
> >> + bool will_need = cp_flags & MM_CP_WILL_NEED;
> >>
> >> tlb_change_page_size(tlb, PAGE_SIZE);
> >>
> >> @@ -172,6 +173,9 @@ static unsigned long change_pte_range(struct mmu_gather *tlb,
> >> ptent = pte_clear_uffd_wp(ptent);
> >> }
> >>
> >> + if (will_need)
> >> + ptent = pte_mkyoung(ptent);
> >
> > For uffd path, UFFD_FLAGS_ACCESS_LIKELY|UFFD_FLAGS_WRITE_LIKELY are new
> > internal flags used with or without the new feature bit set. It means even
> > with !ACCESS_HINT we'll start to set young bit while we used not to? Is
> > that some kind of a light abi change?
> >
> > I'd suggest we only set will_need if ACCESS_HINT is set.
> >
> >> +
> >> /*
> >> * In some writable, shared mappings, we might want
> >> * to catch actual write access -- see
> >> @@ -187,8 +191,11 @@ static unsigned long change_pte_range(struct mmu_gather *tlb,
> >> */
> >> if ((cp_flags & MM_CP_TRY_CHANGE_WRITABLE) &&
> >> !pte_write(ptent) &&
> >> - can_change_pte_writable(vma, addr, ptent))
> >> + can_change_pte_writable(vma, addr, ptent)) {
> >> ptent = pte_mkwrite(ptent);
> >> + if (will_need)
> >> + ptent = pte_mkdirty(ptent);
> >
> > Can we make this unconditional? IOW to cover both:
> >
> > (1) When will_need is not set, or
> > (2) mprotect() too
> >
> > David's patch is good in that we merged the unprotect and CoW. However
> > that's not complete because the dirty bit ops are missing.
> >
> > Here IMHO we should have a standalone patch to just add the dirty bit into
> > this logic when we'll grant write bit. IMHO it'll make the write+dirty
> > bits coherent again in all paths.
>
> I'm not sure I follow.
>
> We *surely* don't want to dirty random pages (especially once in the
> pagecache/swapcache) simply because we change protection.
>
> Just like we don't set all pages write+dirty in a writable VMA on a read
> fault.
IMO unmprotect (in generic mprotect form or uffd form) has a stronger sign
of page being written, unlike read faults, as many of them happen because
page being written and message generated.
But yeah you have a point too, maybe we shouldn't assume such a condition.
Especially as long as we won't set write=1 without soft-dirty tracking
enabled, I think it should be safe.
--
Peter Xu
