On Mon, Feb 27, 2023 at 06:00:44PM -0500, Peter Xu wrote: > This is a new feature that controls how uffd-wp handles none ptes. When > it's set, the kernel will handle anonymous memory the same way as file > memory, by allowing the user to wr-protect unpopulated ptes. > > File memories handles none ptes consistently by allowing wr-protecting of > none ptes because of the unawareness of page cache being exist or not. For > anonymous it was not as persistent because we used to assume that we don't > need protections on none ptes or known zero pages. > > One use case of such a feature bit was VM live snapshot, where if without > wr-protecting empty ptes the snapshot can contain random rubbish in the > holes of the anonymous memory, which can cause misbehave of the guest when > the guest OS assumes the pages should be all zeros. > > QEMU worked it around by pre-populate the section with reads to fill in > zero page entries before starting the whole snapshot process [1]. > > Recently there's another need raised on using userfaultfd wr-protect for > detecting dirty pages (to replace soft-dirty in some cases) [2]. In that > case if without being able to wr-protect none ptes by default, the dirty > info can get lost, since we cannot treat every none pte to be dirty (the > current design is identify a page dirty based on uffd-wp bit being cleared). > > In general, we want to be able to wr-protect empty ptes too even for > anonymous. > > This patch implements UFFD_FEATURE_WP_UNPOPULATED so that it'll make > uffd-wp handling on none ptes being consistent no matter what the memory > type is underneath. It doesn't have any impact on file memories so far > because we already have pte markers taking care of that. So it only > affects anonymous. > > The feature bit is by default off, so the old behavior will be maintained. > Sometimes it may be wanted because the wr-protect of none ptes will contain > overheads not only during UFFDIO_WRITEPROTECT (by applying pte markers to > anonymous), but also on creating the pgtables to store the pte markers. So > there's potentially less chance of using thp on the first fault for a none > pmd or larger than a pmd. > > The major implementation part is teaching the whole kernel to understand > pte markers even for anonymously mapped ranges, meanwhile allowing the > UFFDIO_WRITEPROTECT ioctl to apply pte markers for anonymous too when the > new feature bit is set. > > Note that even if the patch subject starts with mm/uffd, there're a few > small refactors to major mm path of handling anonymous page faults. But > they should be straightforward. > > So far, add a very light smoke test within the userfaultfd kselftest > pagemap unit test to make sure anon pte markers work. > > [1] https://lore.kernel.org/all/20210401092226.102804-4-andrey.gruzdev@xxxxxxxxxxxxx/ > [1] https://lore.kernel.org/all/Y+v2HJ8+3i%2FKzDBu@x1n/ > > Signed-off-by: Peter Xu <peterx@xxxxxxxxxx> > --- > v1->v2: > - Use pte markers rather than populate zero pages when protect [David] > - Rename WP_ZEROPAGE to WP_UNPOPULATED [David] Some very initial performance numbers (I only ran in a VM but it should be similar, unit is "us") below as requested. The measurement is about time spent when wr-protecting 10G range of empty but mapped memory. It's done in a VM, assuming we'll get similar results on bare metal. Four test cases: - default UFFDIO_WP - pre-read the memory, then UFFDIO_WP (what QEMU does right now) - pre-fault using MADV_POPULATE_READ, then default UFFDIO_WP - UFFDIO_WP with WP_UNPOPULATED Results: Test DEFAULT: 2 Test PRE-READ: 3277099 (pre-fault 3253826) Test MADVISE: 2250361 (pre-fault 2226310) Test WP-UNPOPULATE: 20850 I'll add these information into the commit message when there's a new version. [1] https://github.com/xzpeter/clibs/blob/master/uffd-test/uffd-wp-perf.c -- Peter Xu
