On Mon, 2024-11-18 at 20:42 +0100, David Hildenbrand wrote:
> On 18.11.24 15:13, Avi Kivity wrote:
> > #define _GNU_SOURCE
> > #include <sys/mman.h>
> > #include <linux/memfd.h>
> > #include <unistd.h>
> > #include <stddef.h>
> >
> > int main(int ac, char** av) {
> > size_t memsz = (size_t)2048 << 20;
> > int fd = memfd_create("memory", MFD_CLOEXEC);
> > ftruncate(fd, memsz);
> > void* p = mmap((void*)0x40000000, memsz, PROT_READ|PROT_WRITE,
> > MAP_PRIVATE|MAP_FILE, fd, 0);
> > madvise(p, memsz, MADV_HUGEPAGE);
> > madvise(p, memsz, MADV_POPULATE_WRITE);
> > madvise(p, memsz, MADV_COLLAPSE);
> > pause();
> > return 0;
> > }
>
> Hi,
>
> >
> > While memfd is documented as using anonymous pages, AnonHugePages
> > shows
> > as zero.
>
> it's an anonymous file, not anonymous memory. memfd is shmem in
> diguise.
>
> >
> > If memfd incompatible with transparent hugepages?
>
> Shmem supports THP.
>
> But you have one flaws in your program: you have to mmap it
> MAP_SHARED.
> It's shmem.
>
> Otherwise during MADV_POPULATE_WRITE you're simply allocating a shmem
> page (pageache), and the COW it to map an anonymous page in your page
> tables, resulting in a double memory consumption (pagecache+anonymous
> memory). khugepaged does currently not operate on such VMAs.
>
> If you use MAP_SHARED in your example above:
>
Thanks a lot - this works.
In case anyone's interested, I'm using this to provide the equivalent
of vmalloc() in my application. It usually uses a large THP arena
(consuming all of the machine's memory) to allocate from, but sometimes
I need large contiguous memory areas for third-party libraries. I can't
easily allocate large contiguous areas, but 128kB allocations are fine.
So I allocate a few of those, and use mmap(MAP_FIXED) to stitch
together several 128kB allocations into a single large contiguous map.
