Mike Kravetz <mike.kravetz@xxxxxxxxxx> writes: > On 10/14/2016 01:48 AM, Jan Stancek wrote: >> On 10/14/2016 01:26 AM, Mike Kravetz wrote: >>> >>> Hi Jan, >>> >>> Any chance you can get the contents of /sys/kernel/mm/hugepages >>> before and after the first run of libhugetlbfs testsuite on Power? >>> Perhaps a script like: >>> >>> cd /sys/kernel/mm/hugepages >>> for f in hugepages-*/*; do >>> n=`cat $f`; >>> echo -e "$n\t$f"; >>> done >>> >>> Just want to make sure the numbers look as they should. >>> >> >> Hi Mike, >> >> Numbers are below. I have also isolated a single testcase from "func" >> group of tests: corrupt-by-cow-opt [1]. This test stops working if I >> run it 19 times (with 20 hugepages). And if I disable this test, >> "func" group tests can all pass repeatedly. > > Thanks Jan, > > I appreciate your efforts. > >> >> [1] https://github.com/libhugetlbfs/libhugetlbfs/blob/master/tests/corrupt-by-cow-opt.c >> >> Regards, >> Jan >> >> Kernel is v4.8-14230-gb67be92, with reboot between each run. >> 1) Only func tests >> System boot >> After setup: >> 20 hugepages-16384kB/free_hugepages >> 20 hugepages-16384kB/nr_hugepages >> 20 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 0 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> >> After func tests: >> ********** TEST SUMMARY >> * 16M >> * 32-bit 64-bit >> * Total testcases: 0 85 >> * Skipped: 0 0 >> * PASS: 0 81 >> * FAIL: 0 4 >> * Killed by signal: 0 0 >> * Bad configuration: 0 0 >> * Expected FAIL: 0 0 >> * Unexpected PASS: 0 0 >> * Strange test result: 0 0 >> >> 26 hugepages-16384kB/free_hugepages >> 26 hugepages-16384kB/nr_hugepages >> 26 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 1 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> >> After test cleanup: >> umount -a -t hugetlbfs >> hugeadm --pool-pages-max ${HPSIZE}:0 >> >> 1 hugepages-16384kB/free_hugepages >> 1 hugepages-16384kB/nr_hugepages >> 1 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 1 hugepages-16384kB/resv_hugepages >> 1 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> > > I am guessing the leaked reserve page is which is triggered by > running the test you isolated corrupt-by-cow-opt. > > >> --- >> >> 2) Only stress tests >> System boot >> After setup: >> 20 hugepages-16384kB/free_hugepages >> 20 hugepages-16384kB/nr_hugepages >> 20 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 0 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> >> After stress tests: >> 20 hugepages-16384kB/free_hugepages >> 20 hugepages-16384kB/nr_hugepages >> 20 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 17 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> >> After cleanup: >> 17 hugepages-16384kB/free_hugepages >> 17 hugepages-16384kB/nr_hugepages >> 17 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 17 hugepages-16384kB/resv_hugepages >> 17 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> > > This looks worse than the summary after running the functional tests. > >> --- >> >> 3) only corrupt-by-cow-opt >> >> System boot >> After setup: >> 20 hugepages-16384kB/free_hugepages >> 20 hugepages-16384kB/nr_hugepages >> 20 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 0 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages >> >> libhugetlbfs-2.18# env LD_LIBRARY_PATH=./obj64 ./tests/obj64/corrupt-by-cow-opt; /root/grab.sh >> Starting testcase "./tests/obj64/corrupt-by-cow-opt", pid 3298 >> Write s to 0x3effff000000 via shared mapping >> Write p to 0x3effff000000 via private mapping >> Read s from 0x3effff000000 via shared mapping >> PASS >> 20 hugepages-16384kB/free_hugepages >> 20 hugepages-16384kB/nr_hugepages >> 20 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 1 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages > > Leaked one reserve page > >> >> # env LD_LIBRARY_PATH=./obj64 ./tests/obj64/corrupt-by-cow-opt; /root/grab.sh >> Starting testcase "./tests/obj64/corrupt-by-cow-opt", pid 3312 >> Write s to 0x3effff000000 via shared mapping >> Write p to 0x3effff000000 via private mapping >> Read s from 0x3effff000000 via shared mapping >> PASS >> 20 hugepages-16384kB/free_hugepages >> 20 hugepages-16384kB/nr_hugepages >> 20 hugepages-16384kB/nr_hugepages_mempolicy >> 0 hugepages-16384kB/nr_overcommit_hugepages >> 2 hugepages-16384kB/resv_hugepages >> 0 hugepages-16384kB/surplus_hugepages >> 0 hugepages-16777216kB/free_hugepages >> 0 hugepages-16777216kB/nr_hugepages >> 0 hugepages-16777216kB/nr_hugepages_mempolicy >> 0 hugepages-16777216kB/nr_overcommit_hugepages >> 0 hugepages-16777216kB/resv_hugepages >> 0 hugepages-16777216kB/surplus_hugepages > > It is pretty consistent that we leak a reserve page every time this > test is run. > > The interesting thing is that corrupt-by-cow-opt is a very simple > test case. commit 67961f9db8c4 potentially changes the return value > of the functions vma_has_reserves() and vma_needs/commit_reservation() > for the owner (HPAGE_RESV_OWNER) of private mappings. running the > test with and without the commit results in the same return values for > these routines on x86. And, no leaked reserve pages. looking at that commit, I am not sure region_chg output indicate a hole punched. ie, w.r.t private mapping when we mmap, we don't do a region_chg (hugetlb_reserve_page()). So with a fault later when we call vma_needs_reservation, we will find region_chg returning >= 0 right ? > > Is it possible to revert this commit and run the libhugetlbs tests > (func and stress) again while monitoring the counts in /sys? The > counts should go to zero after cleanup as you describe above. I just > want to make sure that this commit is causing all the problems you > are seeing. If it is, then we can consider reverting and I can try > to think of another way to address the original issue. > > Thanks for your efforts on this. I can not reproduce on x86 or sparc > and do not see any similar symptoms on these architectures. > Not sure how any of this is arch specific. So on both x86 and sparc we don't find the count going wrong as above ? -aneesh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>