On 15.10.20 10:57, David Hildenbrand wrote: > On 15.10.20 09:56, David Hildenbrand wrote: >> On 14.10.20 20:31, Mike Kravetz wrote: >>> On 10/14/20 11:18 AM, David Hildenbrand wrote: >>>> On 14.10.20 19:56, Mina Almasry wrote: >>>>> On Wed, Oct 14, 2020 at 9:15 AM David Hildenbrand <david@xxxxxxxxxx> wrote: >>>>>> >>>>>> On 14.10.20 17:22, David Hildenbrand wrote: >>>>>>> Hi everybody, >>>>>>> >>>>>>> Michal Privoznik played with "free page reporting" in QEMU/virtio-balloon >>>>>>> with hugetlbfs and reported that this results in [1] >>>>>>> >>>>>>> 1. WARNING: CPU: 13 PID: 2438 at mm/page_counter.c:57 page_counter_uncharge+0x4b/0x5 >>>>>>> >>>>>>> 2. Any hugetlbfs allocations failing. (I assume because some accounting is wrong) >>>>>>> >>>>>>> >>>>>>> QEMU with free page hinting uses fallocate(FALLOC_FL_PUNCH_HOLE) >>>>>>> to discard pages that are reported as free by a VM. The reporting >>>>>>> granularity is in pageblock granularity. So when the guest reports >>>>>>> 2M chunks, we fallocate(FALLOC_FL_PUNCH_HOLE) one huge page in QEMU. >>>>>>> >>>>>>> I was also able to reproduce (also with virtio-mem, which similarly >>>>>>> uses fallocate(FALLOC_FL_PUNCH_HOLE)) on latest v5.9 >>>>>>> (and on v5.7.X from F32). >>>>>>> >>>>>>> Looks like something with fallocate(FALLOC_FL_PUNCH_HOLE) accounting >>>>>>> is broken with cgroups. I did *not* try without cgroups yet. >>>>>>> >>>>>>> Any ideas? >>>>> >>>>> Hi David, >>>>> >>>>> I may be able to dig in and take a look. How do I reproduce this >>>>> though? I just fallocate(FALLOC_FL_PUNCH_HOLE) one 2MB page in a >>>>> hugetlb region? >>>>> >>>> >>>> Hi Mina, >>>> >>>> thanks for having a look. I started poking around myself but, >>>> being new to cgroup code, I even failed to understand why that code gets >>>> triggered though the hugetlb controller isn't even enabled. >>>> >>>> I assume you at least have to make sure that there is >>>> a page populated (MMAP_POPULATE, or read/write it). But I am not >>>> sure yet if a single fallocate(FALLOC_FL_PUNCH_HOLE) is >>>> sufficient, or if it will require a sequence of >>>> populate+discard(punch) (or multi-threading). >>> >>> FWIW - I ran libhugetlbfs tests which do a bunch of hole punching >>> with (and without) hugetlb controller enabled and did not see this issue. >>> >>> May need to reproduce via QEMU as below. >> >> Not sure if relevant, but QEMU should be using >> memfd_create(MFD_HUGETLB|MFD_HUGE_2MB) to obtain a hugetlbfs file. >> >> Also, QEMU fallocate(FALLOC_FL_PUNCH_HOLE)'s a significant of memory of >> the md (e.g., > 90%). >> > > I just tried to reproduce by doing random accesses + random fallocate(FALLOC_FL_PUNCH_HOLE) within a file - without success. > > So could be > 1. KVM is involved messing this up > 2. Multi-threading is involved > Able to reproduce with TCG under QEMU, so not a KVM issue. -- Thanks, David / dhildenb
