Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh

[David Hildenbrand <david@xxxxxxxxxx>]

On 13.07.23 15:54, Ryan Roberts wrote:
> It is very unclear to me how one is supposed to run all the mm selftests
> consistently and get clear results.
>
> Most of the test programs are launched by both run_vmtests.sh and
> run_kselftest.sh:
>
>    hugepage-mmap
>    hugepage-shm
>    map_hugetlb
>    hugepage-mremap
>    hugepage-vmemmap
>    hugetlb-madvise
>    map_fixed_noreplace
>    gup_test
>    gup_longterm
>    uffd-unit-tests
>    uffd-stress
>    compaction_test
>    on-fault-limit
>    map_populate
>    mlock-random-test
>    mlock2-tests
>    mrelease_test
>    mremap_test
>    thuge-gen
>    virtual_address_range
>    va_high_addr_switch
>    mremap_dontunmap
>    hmm-tests
>    madv_populate
>    memfd_secret
>    ksm_tests
>    ksm_functional_tests
>    soft-dirty
>    cow

Which run_kselftest.sh are you referring to, the one in the parent directory?

How to invoke it to run these mm tests?

(I never dared invoking something different than
run_vmtests.sh ;) )

[...]

> Signed-off-by: Ryan Roberts <ryan.roberts@xxxxxxx>
> ---
>   tools/testing/selftests/mm/Makefile       | 79 ++++++++++++-----------
>   tools/testing/selftests/mm/run_vmtests.sh | 23 +++++++
>   tools/testing/selftests/mm/settings       |  2 +-
>   3 files changed, 64 insertions(+), 40 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 66d7c07dc177..881ed96d96fd 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -35,39 +35,39 @@ MAKEFLAGS += --no-builtin-rules
>   CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
>   LDLIBS = -lrt -lpthread
>   
> -TEST_GEN_PROGS = cow
> -TEST_GEN_PROGS += compaction_test
> -TEST_GEN_PROGS += gup_longterm
> -TEST_GEN_PROGS += gup_test
> -TEST_GEN_PROGS += hmm-tests
> -TEST_GEN_PROGS += hugetlb-madvise
> -TEST_GEN_PROGS += hugepage-mmap
> -TEST_GEN_PROGS += hugepage-mremap
> -TEST_GEN_PROGS += hugepage-shm
> -TEST_GEN_PROGS += hugepage-vmemmap
> -TEST_GEN_PROGS += khugepaged
> -TEST_GEN_PROGS += madv_populate
> -TEST_GEN_PROGS += map_fixed_noreplace
> -TEST_GEN_PROGS += map_hugetlb
> -TEST_GEN_PROGS += map_populate
> -TEST_GEN_PROGS += memfd_secret
> -TEST_GEN_PROGS += migration
> -TEST_GEN_PROGS += mkdirty
> -TEST_GEN_PROGS += mlock-random-test
> -TEST_GEN_PROGS += mlock2-tests
> -TEST_GEN_PROGS += mrelease_test
> -TEST_GEN_PROGS += mremap_dontunmap
> -TEST_GEN_PROGS += mremap_test
> -TEST_GEN_PROGS += on-fault-limit
> -TEST_GEN_PROGS += thuge-gen
> -TEST_GEN_PROGS += transhuge-stress
> -TEST_GEN_PROGS += uffd-stress
> -TEST_GEN_PROGS += uffd-unit-tests
> -TEST_GEN_PROGS += soft-dirty
> -TEST_GEN_PROGS += split_huge_page_test
> -TEST_GEN_PROGS += ksm_tests
> -TEST_GEN_PROGS += ksm_functional_tests
> -TEST_GEN_PROGS += mdwe_test
> +TEST_GEN_FILES = cow
> +TEST_GEN_FILES += compaction_test
> +TEST_GEN_FILES += gup_longterm
> +TEST_GEN_FILES += gup_test
> +TEST_GEN_FILES += hmm-tests
> +TEST_GEN_FILES += hugetlb-madvise
> +TEST_GEN_FILES += hugepage-mmap
> +TEST_GEN_FILES += hugepage-mremap
> +TEST_GEN_FILES += hugepage-shm
> +TEST_GEN_FILES += hugepage-vmemmap
> +TEST_GEN_FILES += khugepaged
> +TEST_GEN_FILES += madv_populate
> +TEST_GEN_FILES += map_fixed_noreplace
> +TEST_GEN_FILES += map_hugetlb
> +TEST_GEN_FILES += map_populate
> +TEST_GEN_FILES += memfd_secret
> +TEST_GEN_FILES += migration
> +TEST_GEN_FILES += mkdirty
> +TEST_GEN_FILES += mlock-random-test
> +TEST_GEN_FILES += mlock2-tests
> +TEST_GEN_FILES += mrelease_test
> +TEST_GEN_FILES += mremap_dontunmap
> +TEST_GEN_FILES += mremap_test
> +TEST_GEN_FILES += on-fault-limit
> +TEST_GEN_FILES += thuge-gen
> +TEST_GEN_FILES += transhuge-stress
> +TEST_GEN_FILES += uffd-stress
> +TEST_GEN_FILES += uffd-unit-tests
> +TEST_GEN_FILES += soft-dirty
> +TEST_GEN_FILES += split_huge_page_test
> +TEST_GEN_FILES += ksm_tests
> +TEST_GEN_FILES += ksm_functional_tests
> +TEST_GEN_FILES += mdwe_test

IIRC, we recently converted all to TEST_GEN_PROGS. See

commit aef6fde75d8c6c1cad4a0e017a8d4cbee2143723
Author: Peter Xu <peterx@xxxxxxxxxx>
Date:   Wed Apr 12 12:42:18 2023 -0400

    selftests/mm: use TEST_GEN_PROGS where proper
    
    TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the mm/Makefile to
    specify programs that need to build.  Logically all these binaries should
    all fall into TEST_GEN_PROGS.
    
    Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can reference
    all the tests easily later.


Why is that change required, and how does it interact with
run_kselftest.sh? (Not clear from you patch description.)

--
Cheers,

David / dhildenb