On 14.10.20 20:31, Mike Kravetz wrote: > On 10/14/20 11:18 AM, David Hildenbrand wrote: >> On 14.10.20 19:56, Mina Almasry wrote: >>> On Wed, Oct 14, 2020 at 9:15 AM David Hildenbrand <david@xxxxxxxxxx> wrote: >>>> >>>> On 14.10.20 17:22, David Hildenbrand wrote: >>>>> Hi everybody, >>>>> >>>>> Michal Privoznik played with "free page reporting" in QEMU/virtio-balloon >>>>> with hugetlbfs and reported that this results in [1] >>>>> >>>>> 1. WARNING: CPU: 13 PID: 2438 at mm/page_counter.c:57 page_counter_uncharge+0x4b/0x5 >>>>> >>>>> 2. Any hugetlbfs allocations failing. (I assume because some accounting is wrong) >>>>> >>>>> >>>>> QEMU with free page hinting uses fallocate(FALLOC_FL_PUNCH_HOLE) >>>>> to discard pages that are reported as free by a VM. The reporting >>>>> granularity is in pageblock granularity. So when the guest reports >>>>> 2M chunks, we fallocate(FALLOC_FL_PUNCH_HOLE) one huge page in QEMU. >>>>> >>>>> I was also able to reproduce (also with virtio-mem, which similarly >>>>> uses fallocate(FALLOC_FL_PUNCH_HOLE)) on latest v5.9 >>>>> (and on v5.7.X from F32). >>>>> >>>>> Looks like something with fallocate(FALLOC_FL_PUNCH_HOLE) accounting >>>>> is broken with cgroups. I did *not* try without cgroups yet. >>>>> >>>>> Any ideas? >>> >>> Hi David, >>> >>> I may be able to dig in and take a look. How do I reproduce this >>> though? I just fallocate(FALLOC_FL_PUNCH_HOLE) one 2MB page in a >>> hugetlb region? >>> >> >> Hi Mina, >> >> thanks for having a look. I started poking around myself but, >> being new to cgroup code, I even failed to understand why that code gets >> triggered though the hugetlb controller isn't even enabled. >> >> I assume you at least have to make sure that there is >> a page populated (MMAP_POPULATE, or read/write it). But I am not >> sure yet if a single fallocate(FALLOC_FL_PUNCH_HOLE) is >> sufficient, or if it will require a sequence of >> populate+discard(punch) (or multi-threading). > > FWIW - I ran libhugetlbfs tests which do a bunch of hole punching > with (and without) hugetlb controller enabled and did not see this issue. > > May need to reproduce via QEMU as below. Not sure if relevant, but QEMU should be using memfd_create(MFD_HUGETLB|MFD_HUGE_2MB) to obtain a hugetlbfs file. Also, QEMU fallocate(FALLOC_FL_PUNCH_HOLE)'s a significant of memory of the md (e.g., > 90%). -- Thanks, David / dhildenb
