On Mon, Jul 17, 2023 at 07:27:13PM +0200, David Hildenbrand wrote: > On 17.07.23 12:31, Ryan Roberts wrote: > > It is very unclear to me how one is supposed to run all the mm selftests > > consistently and get clear results. > > > > Most of the test programs are launched by both run_vmtests.sh and > > run_kselftest.sh: > > > > hugepage-mmap > > hugepage-shm > > map_hugetlb > > hugepage-mremap > > hugepage-vmemmap > > hugetlb-madvise > > map_fixed_noreplace > > gup_test > > gup_longterm > > uffd-unit-tests > > uffd-stress > > compaction_test > > on-fault-limit > > map_populate > > mlock-random-test > > mlock2-tests > > mrelease_test > > mremap_test > > thuge-gen > > virtual_address_range > > va_high_addr_switch > > mremap_dontunmap > > hmm-tests > > madv_populate > > memfd_secret > > ksm_tests > > ksm_functional_tests > > soft-dirty > > cow > > > > However, of this set, when launched by run_vmtests.sh, some of the > > programs are invoked multiple times with different arguments. When > > invoked by run_kselftest.sh, they are invoked without arguments (and as > > a consequence, some fail immediately). > > > > Some test programs are only launched by run_vmtests.sh: > > > > test_vmalloc.sh > > > > And some test programs and only launched by run_kselftest.sh: > > > > khugepaged > > migration > > mkdirty > > transhuge-stress > > split_huge_page_test > > mdwe_test > > write_to_hugetlbfs > > > > Furthermore, run_vmtests.sh is invoked by run_kselftest.sh, so in this > > case all the test programs invoked by both scripts are run twice! > > > > Needless to say, this is a bit of a mess. In the absence of fully > > understanding the history here, it looks to me like the best solution is > > to launch ALL test programs from run_vmtests.sh, and ONLY invoke > > run_vmtests.sh from run_kselftest.sh. This way, we get full control over > > the parameters, each program is only invoked the intended number of > > times, and regardless of which script is used, the same tests get run in > > the same way. > > > > The only drawback is that if using run_kselftest.sh, it's top-level tap > > result reporting reports only a single test and it fails if any of the > > contained tests fail. I don't see this as a big deal though since we > > still see all the nested reporting from multiple layers. The other issue > > with this is that all of run_vmtests.sh must execute within a single > > kselftest timeout period, so let's increase that to something more > > suitable. > > > > In the Makefile, TEST_GEN_PROGS will compile and install the tests and > > will add them to the list of tests that run_kselftest.sh will run. > > TEST_GEN_FILES will compile and install the tests but will not add them > > to the test list. So let's move all the programs from TEST_GEN_PROGS to > > TEST_GEN_FILES so that they are built but not executed by > > run_kselftest.sh. Note that run_vmtests.sh is added to TEST_PROGS, which > > means it ends up in the test list. (the lack of "_GEN" means it won't be > > compiled, but simply copied). > > > > Signed-off-by: Ryan Roberts <ryan.roberts@xxxxxxx> > > --- > > Acked-by: David Hildenbrand <david@xxxxxxxxxx> Thanks for letting me know, David. Sorry for the late response, still catching up things. I used to justify that from mm/ itself that everything should be PROG, but I see that from higher level where TEST_GEN_FILE|PROG is really used this makes sense. As long as vm_utils.o will be properly linked I'll be happy enough.. Acked-by: Peter Xu <peterx@xxxxxxxxxx> Thanks, -- Peter Xu
