On 17.02.21 16:48, David Hildenbrand wrote:
When we manage sparse memory mappings dynamically in user space - also
sometimes involving MADV_NORESERVE - we want to dynamically populate/
discard memory inside such a sparse memory region. Example users are
hypervisors (especially implementing memory ballooning or similar
technologies like virtio-mem) and memory allocators. In addition, we want
to fail in a nice way if populating does not succeed because we are out of
backend memory (which can happen easily with file-based mappings,
especially tmpfs and hugetlbfs).
While MADV_DONTNEED and FALLOC_FL_PUNCH_HOLE provide us ways to reliably
discard memory, there is no generic approach to populate ("preallocate")
memory.
Although mmap() supports MAP_POPULATE, it is not applicable to the concept
of sparse memory mappings, where we want to do populate/discard
dynamically and avoid expensive/problematic remappings. In addition,
we never actually report error during the final populate phase - it is
best-effort only.
fallocate() can be used to preallocate file-based memory and fail in a safe
way. However, it is less useful for private mappings on anonymous files
due to COW semantics. For example, using fallocate() to preallocate memory
on an anonymous memfd files that are mapped MAP_PRIVATE results in a double
memory consumption when actually writing via the mapping. In addition,
fallocate() does not actually populate page tables, so we still always
have to resolve minor faults on first access.
Because we don't have a proper interface, what applications
(like QEMU and databases) end up doing is touching (i.e., writing) all
individual pages. However, it requires expensive signal handling (SIGBUS);
for example, this is problematic in hypervisors like QEMU where SIGBUS
handlers might already be used by other subsystems concurrently to e.g,
handle hardware errors. "Simply" doing preallocation from another thread
is not that easy.
Let's introduce MADV_POPULATE with the following semantics
1. MADV_POPULATED does not work on PROT_NONE and special VMAs. It works
on everything else.
2. Errors during MADV_POPULATED (especially OOM) are reported. If we hit
hardware errors on pages, ignore them - nothing we really can or
should do.
3. On errors during MADV_POPULATED, some memory might have been
populated. Callers have to clean up if they care.
4. Concurrent changes to the virtual memory layour are tolerated - we
process each and every PFN only once, though.
5. If MADV_POPULATE succeeds, all memory in the range can be accessed
without SIGBUS. (of course, not if user space changed mappings in the
meantime or KSM kicked in on anonymous memory).
Although sparse memory mappings are the primary use case, this will
also be useful for ordinary preallocations where MAP_POPULATE is not
desired (e.g., in QEMU, where users can trigger preallocation of
guest RAM after the mapping was created).
Looking at the history, MADV_POPULATE was already proposed in 2013 [1],
however, the main motivation back than was performance improvements
(which should also still be the case, but it's a seconary concern).
Basic functionality was tested with:
- anonymous memory
- MAP_PRIVATE on anonymous file via memfd
- MAP_SHARED on anonymous file via memf
- MAP_PRIVATE on anonymous hugetlbfs file via memfd
- MAP_SHARED on anonymous hugetlbfs file via memfd
- MAP_PRIVATE on tmpfs/shmem file (we end up with double memory consumption
though, as the actual file gets populated with zeroes)
- MAP_SHARED on tmpfs/shmem file
Note: For populating/preallocating zeroed-out memory while userfaultfd is
active, it's even faster to use first fallocate() or placing zeroed pages
via userfaultfd APIs. Otherwise, we'll have to route every fault while
populating via the userfaultfd handler.
[1] https://lkml.org/lkml/2013/6/27/698
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Arnd Bergmann <arnd@xxxxxxxx>
Cc: Michal Hocko <mhocko@xxxxxxxx>
Cc: Oscar Salvador <osalvador@xxxxxxx>
Cc: Matthew Wilcox (Oracle) <willy@xxxxxxxxxxxxx>
Cc: Andrea Arcangeli <aarcange@xxxxxxxxxx>
Cc: Minchan Kim <minchan@xxxxxxxxxx>
Cc: Jann Horn <jannh@xxxxxxxxxx>
Cc: Jason Gunthorpe <jgg@xxxxxxxx>
Cc: Dave Hansen <dave.hansen@xxxxxxxxx>
Cc: Hugh Dickins <hughd@xxxxxxxxxx>
Cc: Rik van Riel <riel@xxxxxxxxxxx>
Cc: Michael S. Tsirkin <mst@xxxxxxxxxx>
Cc: Kirill A. Shutemov <kirill.shutemov@xxxxxxxxxxxxxxx>
Cc: Vlastimil Babka <vbabka@xxxxxxx>
Cc: Richard Henderson <rth@xxxxxxxxxxx>
Cc: Ivan Kokshaysky <ink@xxxxxxxxxxxxxxxxxxxx>
Cc: Matt Turner <mattst88@xxxxxxxxx>
Cc: Thomas Bogendoerfer <tsbogend@xxxxxxxxxxxxxxxx>
Cc: "James E.J. Bottomley" <James.Bottomley@xxxxxxxxxxxxxxxxxxxxx>
Cc: Helge Deller <deller@xxxxxx>
Cc: Chris Zankel <chris@xxxxxxxxxx>
Cc: Max Filippov <jcmvbkbc@xxxxxxxxx>
Cc: linux-alpha@xxxxxxxxxxxxxxx
Cc: linux-mips@xxxxxxxxxxxxxxx
Cc: linux-parisc@xxxxxxxxxxxxxxx
Cc: linux-xtensa@xxxxxxxxxxxxxxxx
Cc: linux-arch@xxxxxxxxxxxxxxx
Signed-off-by: David Hildenbrand <david@xxxxxxxxxx>
---
If we agree that this makes sense I'll do more testing to see if we
are missing any return value handling and prepare a man page update to
document the semantics.
Thoughts?
Thinking about MADV_POPULATE vs. MADV_POPULATE_WRITE I wonder if it
would be more versatile to break with existing MAP_POPULATE semantics
and directly go with
MADV_POPULATE_READ: simulate user space read access without actually
reading. Trigger a read fault if required.
MADV_POPULATE_WRITE: simulate user space write access without actually
writing. Trigger a write fault if required.
For my use case, I could use MADV_POPULATE_WRITE on anonymous memory and
RAM-backed files (shmem/hugetlb) - I would not have a minor fault when
the guest inside the VM first initializes memory. This mimics how QEMU
currently preallocates memory.
However, I would use MADV_POPULATE_READ on any !RAM-backed files where
we actually have to write-back to a (slow?) device. Dirtying everything
although the guest might not actually consume it in the near future
might be undesired.
MADV_POPULATE_READ could also come in handy in combination with
userfaulfd-wp() [1], when handling unpopulated memory via ordinary
userfaultfd MISSING events in undesired. I could imagine it can speed up
live migration of VMs in general, where we might end up reading a lot of
unpopulated memory to figure out it's all zeroes after faulting in the
shared zeropage. Especially relevant with a shared zeropage.
Thoughts?
[1] https://lkml.kernel.org/r/20210219211054.GL6669@xz-x1
--
Thanks,
David / dhildenb
