On Wed, Jun 12, 2019 at 11:19 AM Dmitry Vyukov <dvyukov@xxxxxxxxxx> wrote: > > > > On Tue, Jun 11, 2019 at 9:20 PM shuah <shuah@xxxxxxxxxx> wrote: > > > > > > > > > > On 6/11/19 10:03 AM, Dmitry Vyukov wrote: > > > > > > On Tue, Jun 11, 2019 at 5:16 PM shuah <shuah@xxxxxxxxxx> wrote: > > > > > >> > > > > > >> Hi Dmitry, > > > > > >> > > > > > >> On 6/11/19 4:30 AM, Dmitry Vyukov wrote: > > > > > >>> Hi, > > > > > >>> > > > > > >>> I've tried to build kselftests for several years now, but I always > > > > > >>> find the build broken. Which makes me wonder if the instructions are > > > > > >>> broken or something. I follow the instructions in > > > > > >>> Documentation/dev-tools/kselftest.rst and start with "make -C > > > > > >>> tools/testing/selftests". Here is the errors I get on the upstream > > > > > >>> commit 16d72dd4891fecc1e1bf7ca193bb7d5b9804c038: > > > > > >>>> error: unable to create target: 'No available targets are compatible > > > > > >>> with triple "bpf"' > > > > > >>> 1 error generated. > > > > > >>> Makefile:259: recipe for target 'elfdep' failed > > > > > >>> Makefile:156: recipe for target 'all' failed > > > > > >>> Makefile:106: recipe for target > > > > > >>> '/linux/tools/testing/selftests/bpf/libbpf.a' failed > > > > > >>> test_execve.c:4:10: fatal error: cap-ng.h: No such file or directory > > > > > >> > > > > > >> These errors are due to missing dependencies. You will need > > > > > >> > > > > > >> libmount-dev > > > > > >> libcap-ng-dev > > > > > >> libelf-dev > > > > > >> > > > > > >> for bpf to build and also clang > > > > > >> > > > > > >>> ../lib.mk:138: recipe for target > > > > > >>> '/linux/tools/testing/selftests/capabilities/test_execve' failed > > > > > >>> gpio-mockup-chardev.c:20:10: fatal error: libmount.h: No such file or directory > <builtin>: recipe for target 'gpio-mockup-chardev' failed > > > > > >>> fuse_mnt.c:17:10: fatal error: fuse.h: No such file or directory > > > > > >> > > > > > >> libfuse-dev is missing. > > > > > >> > > > > > >>> ../lib.mk:138: recipe for target > > > > > >>> '/linux/tools/testing/selftests/memfd/fuse_mnt' failed > > > > > >>> collect2: error: ld returned 1 exit status > > > > > >>> ../lib.mk:138: recipe for target > > > > > >>> '/linux/tools/testing/selftests/mqueue/mq_open_tests' failed > > > > > >> > > > > > >> Needs libpopt-dev > > > > > >> > > > > > >>> reuseport_bpf_numa.c:24:10: fatal error: numa.h: No such file or directory > > > > > >> > > > > > >> Needs libnuma-dev > > > > > >> > > > > > >>> ../lib.mk:138: recipe for target > > > > > >>> '/linux/tools/testing/selftests/net/reuseport_bpf_numa' failed > > > > > >>> mlock-random-test.c:8:10: fatal error: sys/capability.h: No such file > > > > > >>> or directory > ../lib.mk:138: recipe for target > > > > > >>> '/linux/tools/testing/selftests/vm/mlock-random-test' failed > > > > > >>> > > > > > >>> Here is full log: > > > > > >>> > > > > > >>> https://gist.githubusercontent.com/dvyukov/47430636e160f297b657df5ba2efa82b/raw/7babc4db228b88f341a376c15e8bc9c4c3b02160/gistfile1.txt > > > > > >>> > > > > > >>> I have libelf-dev installed. Do I need to install something else? Or > > > > > >>> run some other command? > > > > > >> > > > > > >> ii libelf-dev:amd 0.170-0.4ubu amd64 libelf1 development > > > > > >> libraries and > > > > > >> ii libelf1:amd64 0.170-0.4ubu amd64 library to read and write > > > > > >> ELF fil > > > > > >> > > > > > >> > > > > > >> All of the above built for me on Linux 5.2-rc4. Try installing all of > > > > > >> these and let me know if you still see problems. > > > > > > > > > > > > > > > > > > Hi Shuah, > > > > > > > > > > > > Thanks for quick reply! > > > > > > > > > > > > I've installed these: libmount-dev libcap-ng-dev libfuse-dev > > > > > > libpopt-dev libnuma-dev. > > > > > > libelf-dev I already had. And for clang I switched to distro-provided one. > > > > > > > > > > > > This reduced number of errors, but I still see some: > > > > > > > > > > > > clang: error: unable to execute command: Broken pipe > > > > > > clang: error: clang frontend command failed due to signal (use -v to > > > > > > see invocation) > > > > > > Makefile:259: recipe for target 'elfdep' failed > > > > > > Makefile:156: recipe for target 'all' failed > > > > > > Makefile:106: recipe for target > > > > > > '/linux/tools/testing/selftests/bpf/libbpf.a' failed > > > > > > > > > > Getting bpf compile to work take a few steps. If I remember correctly, > > > > > You will need llvm as well. Here is what I have on my system: > > > > > > > > > > ii libllvm6.0:amd 1:6.0-1ubunt amd64 Modular compiler and > > > > > toolchain te > > > > > ii llvm 1:6.0-41~exp amd64 Low-Level Virtual Machine > > > > > (LLVM) > > > > > ii llvm-6.0 1:6.0-1ubunt amd64 Modular compiler and > > > > > toolchain te > > > > > ii llvm-6.0-dev 1:6.0-1ubunt amd64 Modular compiler and > > > > > toolchain te > > > > > un llvm-6.0-doc <none> <none> (no description available) > > > > > ii llvm-6.0-runti 1:6.0-1ubunt amd64 Modular compiler and > > > > > toolchain te > > > > > ii llvm-runtime 1:6.0-41~exp amd64 Low-Level Virtual Machine > > > > > (LLVM), > > > > > > > > > > > timestamping.c:249:19: error: ‘SIOCGSTAMP’ undeclared (first use in > > > > > > this function); did you mean ‘SIOCGSTAMPNS’? > > > > > > ../../lib.mk:138: recipe for target > > > > > > '/linux/tools/testing/selftests/networking/timestamping/timestamping' > > > > > > failed > > > > > > mlock-random-test.c:8:10: fatal error: sys/capability.h: No such file > > > > > > or directory > > > > > > > > > > Do you have libcap-dev installed? > > > > > > > > > > ii libcap-dev:amd 1:2.25-1.2 amd64 POSIX 1003.1e capabilities > > > > > (devel > > > > > ii libcap-ng-dev 0.7.7-3.1 amd64 Development and header > > > > > files for > > > > > ii libcap-ng0:amd 0.7.7-3.1 amd64 An alternate POSIX > > > > > capabilities l > > > > > ii libcap2:amd64 1:2.25-1.2 amd64 POSIX 1003.1e capabilities > > > > > (libra > > > > > ii libcap2-bin 1:2.25-1.2 amd64 POSIX 1003.1e capabilities > > > > > (utili > > > > > un libcap2-dev <none> <none> (no description available) > > > > > > > > I've installed libcap-dev and resolved the missing header. > > > > > > > > I've also installed llvm llvm-6.0 llvm-6.0-dev llvm-6.0-doc > > > > libllvm6.0 llvm-6.0-runtime llvm-runtime and it fixed crashing > > > > compiler. > > > > But bpf tests build was still failing due to missing libelf. But I had > > > > the library, so I went and removed some random files: > > > > tools/testing/selftests/bpf/{feature,FEATURE-DUMP.libbpf}. Don't ask > > > > me why these. > > > > > > > > I am now down to just 1 build error: > > > > > > > > CC /usr/local/google/home/dvyukov/src/linux/tools/testing/selftests/bpf/str_error.o > > > > timestamping.c:249:19: error: ‘SIOCGSTAMP’ undeclared (first use in > > > > this function); did you mean ‘SIOCGSTAMPNS’? > > > > > > > > > Is this a non-fatal error? Usually when make produces errors, one > > > expects that nothing is done and it was aborted mid-way. But make > > > seems to produce some test binaries by now. > > > > > > > > > Reading the doc further, these command seem to implicitly assume that > > > the tests will run right on my host machine: > > > > > > $ make -C tools/testing/selftests run_tests > > > $ make kselftest > > > > > > Is it right? At least I don't see how it's configured to run them > > > somewhere else? Or it uses something like qemu by default to run the > > > kernel under test? > > > If it runs the tests on the host, it can't work for me. I don't have > > > the test kernel installed and there is no way I can do this. Policy > > > rules aside, this is yet untested kernel, so by installing it I am > > > risking losing my whole machine and all data... > > > > > > What am I missing? > > > > Reading further. "Install selftests" and "Running installed selftests" > > sections. Is it something I can use to copy the pre-built tests to the > > test machine? The sections don't spell it, so I am just trying to > > second guess. Or what's the purpose of installing? > > > > The "Running installed selftests" section says: > > "Kselftest install as well as the Kselftest tarball provide a script > > named "run_kselftest.sh" to run the tests". > > > > What is the "Kselftest tarball"? Where does one get one? I don't see > > any mentions of "tarball" anywhere else in the doc. > > > > Running ./kselftest_install.sh I am getting: > > /bin/sh: llvm-readelf: command not found > > I can't find any package that would provide this. I happened to have a > custom llvm build that has that binary, but I am interested how I was > supposed to get this for the purposes of documentation and reuse of > instructions by others. > > After adding my llvm-readelf to PATH, I am then getting: > > make[1]: *** No rule to make target 'emit_tests'. Stop. > > Looks like an error, or is it? > > It produced something in the output dir, so copied that to the test > machine and tried to run run_kselftest.sh there, but it failed too: > > ~/kselftest# ./run_kselftest.sh > ./run_kselftest.sh: 2: ./run_kselftest.sh: realpath: not found > ./run_kselftest.sh: 4: .: Can't open ./kselftest/runner.sh > > Is there some kind of prerequisites that I am supposed to install there? > Since the target may have non-x86 arch and a custom distro, any > additional dependency there may be very painful to get... Hi Shuah, I am asking lots of questions, but I did not provide my motivation and end goal. I am trying to understand overall state of the kernel testing better and understand (1) if there are working instructions to run kernel testing that I can give to a new team member or a new external kernel developer, (2) if/how I can ask a kernel developer fixing a bug to add a regression test and ensure that it works. Note in these cases a user may not have lots of specific expertise (e.g. any unsaid/implicit thing may be a showstopper) and/or don't have infinite motivation/time (may give up given a single excuse to do so) and/or don't have specific interest/expertise in the tested subsystem (e.g. a drive-by fix). So now I am trying to follow this route myself, documenting steps here. Back to: ./run_kselftest.sh: 2: ./run_kselftest.sh: realpath: not found dpkg on my host says that this binary comes from coreutils: $ dpkg -S realpath coreutils: /usr/bin/realpath but installing coreutils does not help. Seems that I need to install realpath itself. But this package happens to be broken on my test distro: # apt-get install realpath Reading package lists... Done Building dependency tree... Done The following NEW packages will be installed: realpath 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 16.7 kB of archives. After this operation, 115 kB of additional disk space will be used. Err http://deb.debian.org/debian/ wheezy/main realpath amd64 1.18 404 Not Found [IP: 151.101.120.204 80] Failed to fetch http://deb.debian.org/debian/pool/main/r/realpath/realpath_1.18_amd64.deb 404 Not Found [IP: 151.101.120.204 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? I've switched to another distro (fortunately I had another one pre-built), and run_kselftest.sh started running tests. But now I have even more questions :) 1. Meta-question: "Running a subset of selftests" section talks about "subsystems". How can I map a source file I changed in a drive-by fix to a subsystem? Say, I changed net/ipv6/netfilter/nft_redir_ipv6.c or drivers/usb/c67x00/c67x00-drv.c, what subsystems do I need to run? 2. All C tests seem to fail with: # ./test_maps: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.26' not found (required by ./test_maps) Which means that my image is still unsuitable. How can I get an image that is suitable to run the tests? 3. Lots of tests that do run (probably shell tests), fail/skipped with some cryptic for me errors like: # Cannot find device "ip6gre11" # selftests: [SKIP] Could not run test without the ip xdpgeneric support # modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/5.1.0+/modules.dep.bin' # selftests: bpf: test_tc_edt.sh # nc is not available not ok 40 selftests: bpf: test_tc_edt.sh Say, I either want to run tests for a specific subsystem because I am doing a drive-by fix (a typical newcomer/good Samaritan scenario), or I want to run as many tests as possible (a typical CI scenario). Is there a way to bulk satisfy all these prerequisite (configs, binaries and whatever they are asking for)? 4. There is a test that consistently reboots my machine: # selftests: breakpoints: step_after_suspend_test [ 514.024889] PM: suspend entry (deep) [ 514.025959] PM: Syncing filesystems ... done. [ 514.051573] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 514.054140] OOM killer disabled. [ 514.054764] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 514.057695] printk: Suspending console(s) (use no_console_suspend to debug) early console in extract_kernel input_data: 0x0000000007ddc2e9 input_len: 0x0000000002c26bf0 output: 0x0000000001000000 output_len: 0x0000000008492a48 kernel_total_size: 0x0000000009a26000 trampoline_32bit: 0x000000000009d000 Decompressing Linux... Parsing ELF... done. Booting the kernel. [ 0.000000] Linux version 5.0.0 (gcc version 7.3.0 (Debian 7.3.0-18)) #7 SMP PREEMPT Wed Jun 12 11:38:12 CEST 2019 Is it a bug in the test? in the kernel? Or how is this supposed to work/what am I supposed to do with this? 5. There is a test that triggers a use-after-free: [ 262.639848][ C1] BUG: KASAN: use-after-free in ip6gre_tunnel_lookup+0x1a27/0x1ae0 I went back to v5.0 (3+ months ago), and I see that it's also the case there. Do you know if anybody running these tests? With KMEMLEAK, LOCKDEP, etc? 6. Do we know what's the current code coverage achieved by these tests? What's covered? What's not? Overall percent/per-subsystem/etc? Thanks
