On 19/07/2023 21:45, Peter Xu wrote: > On Mon, Jul 17, 2023 at 07:27:13PM +0200, David Hildenbrand wrote: >> On 17.07.23 12:31, Ryan Roberts wrote: >>> It is very unclear to me how one is supposed to run all the mm selftests >>> consistently and get clear results. >>> >>> Most of the test programs are launched by both run_vmtests.sh and >>> run_kselftest.sh: >>> >>> hugepage-mmap >>> hugepage-shm >>> map_hugetlb >>> hugepage-mremap >>> hugepage-vmemmap >>> hugetlb-madvise >>> map_fixed_noreplace >>> gup_test >>> gup_longterm >>> uffd-unit-tests >>> uffd-stress >>> compaction_test >>> on-fault-limit >>> map_populate >>> mlock-random-test >>> mlock2-tests >>> mrelease_test >>> mremap_test >>> thuge-gen >>> virtual_address_range >>> va_high_addr_switch >>> mremap_dontunmap >>> hmm-tests >>> madv_populate >>> memfd_secret >>> ksm_tests >>> ksm_functional_tests >>> soft-dirty >>> cow >>> >>> However, of this set, when launched by run_vmtests.sh, some of the >>> programs are invoked multiple times with different arguments. When >>> invoked by run_kselftest.sh, they are invoked without arguments (and as >>> a consequence, some fail immediately). >>> >>> Some test programs are only launched by run_vmtests.sh: >>> >>> test_vmalloc.sh >>> >>> And some test programs and only launched by run_kselftest.sh: >>> >>> khugepaged >>> migration >>> mkdirty >>> transhuge-stress >>> split_huge_page_test >>> mdwe_test >>> write_to_hugetlbfs >>> >>> Furthermore, run_vmtests.sh is invoked by run_kselftest.sh, so in this >>> case all the test programs invoked by both scripts are run twice! >>> >>> Needless to say, this is a bit of a mess. In the absence of fully >>> understanding the history here, it looks to me like the best solution is >>> to launch ALL test programs from run_vmtests.sh, and ONLY invoke >>> run_vmtests.sh from run_kselftest.sh. This way, we get full control over >>> the parameters, each program is only invoked the intended number of >>> times, and regardless of which script is used, the same tests get run in >>> the same way. >>> >>> The only drawback is that if using run_kselftest.sh, it's top-level tap >>> result reporting reports only a single test and it fails if any of the >>> contained tests fail. I don't see this as a big deal though since we >>> still see all the nested reporting from multiple layers. The other issue >>> with this is that all of run_vmtests.sh must execute within a single >>> kselftest timeout period, so let's increase that to something more >>> suitable. >>> >>> In the Makefile, TEST_GEN_PROGS will compile and install the tests and >>> will add them to the list of tests that run_kselftest.sh will run. >>> TEST_GEN_FILES will compile and install the tests but will not add them >>> to the test list. So let's move all the programs from TEST_GEN_PROGS to >>> TEST_GEN_FILES so that they are built but not executed by >>> run_kselftest.sh. Note that run_vmtests.sh is added to TEST_PROGS, which >>> means it ends up in the test list. (the lack of "_GEN" means it won't be >>> compiled, but simply copied). >>> >>> Signed-off-by: Ryan Roberts <ryan.roberts@xxxxxxx> >>> --- >> >> Acked-by: David Hildenbrand <david@xxxxxxxxxx> > > Thanks for letting me know, David. Sorry for the late response, still > catching up things. > > I used to justify that from mm/ itself that everything should be PROG, but > I see that from higher level where TEST_GEN_FILE|PROG is really used this > makes sense. As long as vm_utils.o will be properly linked I'll be happy > enough.. Yep that's the case; I've set it up so that vm_utils.o is linked in for both TEST_GEN_FILE and TEST_GEN_PROG binaries. > > Acked-by: Peter Xu <peterx@xxxxxxxxxx> Thanks! > > Thanks, >
