On 10/14/20 11:31 AM, Mike Kravetz wrote: > On 10/14/20 11:18 AM, David Hildenbrand wrote: > > FWIW - I ran libhugetlbfs tests which do a bunch of hole punching > with (and without) hugetlb controller enabled and did not see this issue. > I took a closer look after running just the fallocate_stress test in libhugetlbfs. Here are the cgroup counter values: hugetlb.2MB.failcnt 0 hugetlb.2MB.limit_in_bytes 9223372036854771712 hugetlb.2MB.max_usage_in_bytes 209715200 hugetlb.2MB.rsvd.failcnt 0 hugetlb.2MB.rsvd.limit_in_bytes 9223372036854771712 hugetlb.2MB.rsvd.max_usage_in_bytes 601882624 hugetlb.2MB.rsvd.usage_in_bytes 392167424 hugetlb.2MB.usage_in_bytes 0 We did not hit the WARN_ON_ONCE(), but the 'rsvd.usage_in_bytes' value is not correct in that it should be zero. No huge page reservations remain after the test. HugePages_Total: 1024 HugePages_Free: 1024 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 2097152 kB To try and better understand the reservation cgroup controller, I addded a few printks to the code. While running fallocate_stress with the printks, I can consistently hit the WARN_ON_ONCE() due to the counter going negative. Here are the cgroup counter values after such a run: hugetlb.2MB.failcnt 0 hugetlb.2MB.limit_in_bytes 9223372036854771712 hugetlb.2MB.max_usage_in_bytes 209715200 hugetlb.2MB.rsvd.failcnt 3 hugetlb.2MB.rsvd.limit_in_bytes 9223372036854771712 hugetlb.2MB.rsvd.max_usage_in_bytes 251658240 hugetlb.2MB.rsvd.usage_in_bytes 18446744073487253504 hugetlb.2MB.usage_in_bytes 0 Again, no reserved pages after the test. HugePages_Total: 1024 HugePages_Free: 1024 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 2097152 kB I have some basic hugetlb hole punch functionality tests. Running these on the kernel with added printk's does not cause any issues. In order to reproduce, I need to run fallocate_stress test which will cause hole punch to race with page fault. Best guess at this time is that some of the error/race detection reservation back out code is not properly dealing with cgroup accounting. I'll take a look at this as well. -- Mike Kravetz
