On Wed, Jun 12, 2019 at 1:05 PM Dmitry Vyukov <dvyukov@xxxxxxxxxx> wrote: > > On Wed, Jun 12, 2019 at 11:19 AM Dmitry Vyukov <dvyukov@xxxxxxxxxx> wrote: > > > > > On Tue, Jun 11, 2019 at 9:20 PM shuah <shuah@xxxxxxxxxx> wrote: > > > > > > > > > > > > On 6/11/19 10:03 AM, Dmitry Vyukov wrote: > > > > > > > On Tue, Jun 11, 2019 at 5:16 PM shuah <shuah@xxxxxxxxxx> wrote: > > > > > > >> > > > > > > >> Hi Dmitry, > > > > > > >> > > > > > > >> On 6/11/19 4:30 AM, Dmitry Vyukov wrote: > > > > > > >>> Hi, > > > > > > >>> > > > > > > >>> I've tried to build kselftests for several years now, but I always > > > > > > >>> find the build broken. Which makes me wonder if the instructions are > > > > > > >>> broken or something. I follow the instructions in > > > > > > >>> Documentation/dev-tools/kselftest.rst and start with "make -C > > > > > > >>> tools/testing/selftests". Here is the errors I get on the upstream > > > > > > >>> commit 16d72dd4891fecc1e1bf7ca193bb7d5b9804c038: > > > > > > >>>> error: unable to create target: 'No available targets are compatible > > > > > > >>> with triple "bpf"' > > > > > > >>> 1 error generated. > > > > > > >>> Makefile:259: recipe for target 'elfdep' failed > > > > > > >>> Makefile:156: recipe for target 'all' failed > > > > > > >>> Makefile:106: recipe for target > > > > > > >>> '/linux/tools/testing/selftests/bpf/libbpf.a' failed > > > > > > >>> test_execve.c:4:10: fatal error: cap-ng.h: No such file or directory > > > > > > >> > > > > > > >> These errors are due to missing dependencies. You will need > > > > > > >> > > > > > > >> libmount-dev > > > > > > >> libcap-ng-dev > > > > > > >> libelf-dev > > > > > > >> > > > > > > >> for bpf to build and also clang > > > > > > >> > > > > > > >>> ../lib.mk:138: recipe for target > > > > > > >>> '/linux/tools/testing/selftests/capabilities/test_execve' failed > > > > > > >>> gpio-mockup-chardev.c:20:10: fatal error: libmount.h: No such file or directory > <builtin>: recipe for target 'gpio-mockup-chardev' failed > > > > > > >>> fuse_mnt.c:17:10: fatal error: fuse.h: No such file or directory > > > > > > >> > > > > > > >> libfuse-dev is missing. > > > > > > >> > > > > > > >>> ../lib.mk:138: recipe for target > > > > > > >>> '/linux/tools/testing/selftests/memfd/fuse_mnt' failed > > > > > > >>> collect2: error: ld returned 1 exit status > > > > > > >>> ../lib.mk:138: recipe for target > > > > > > >>> '/linux/tools/testing/selftests/mqueue/mq_open_tests' failed > > > > > > >> > > > > > > >> Needs libpopt-dev > > > > > > >> > > > > > > >>> reuseport_bpf_numa.c:24:10: fatal error: numa.h: No such file or directory > > > > > > >> > > > > > > >> Needs libnuma-dev > > > > > > >> > > > > > > >>> ../lib.mk:138: recipe for target > > > > > > >>> '/linux/tools/testing/selftests/net/reuseport_bpf_numa' failed > > > > > > >>> mlock-random-test.c:8:10: fatal error: sys/capability.h: No such file > > > > > > >>> or directory > ../lib.mk:138: recipe for target > > > > > > >>> '/linux/tools/testing/selftests/vm/mlock-random-test' failed > > > > > > >>> > > > > > > >>> Here is full log: > > > > > > >>> > > > > > > >>> https://gist.githubusercontent.com/dvyukov/47430636e160f297b657df5ba2efa82b/raw/7babc4db228b88f341a376c15e8bc9c4c3b02160/gistfile1.txt > > > > > > >>> > > > > > > >>> I have libelf-dev installed. Do I need to install something else? Or > > > > > > >>> run some other command? > > > > > > >> > > > > > > >> ii libelf-dev:amd 0.170-0.4ubu amd64 libelf1 development > > > > > > >> libraries and > > > > > > >> ii libelf1:amd64 0.170-0.4ubu amd64 library to read and write > > > > > > >> ELF fil > > > > > > >> > > > > > > >> > > > > > > >> All of the above built for me on Linux 5.2-rc4. Try installing all of > > > > > > >> these and let me know if you still see problems. > > > > > > > > > > > > > > > > > > > > > Hi Shuah, > > > > > > > > > > > > > > Thanks for quick reply! > > > > > > > > > > > > > > I've installed these: libmount-dev libcap-ng-dev libfuse-dev > > > > > > > libpopt-dev libnuma-dev. > > > > > > > libelf-dev I already had. And for clang I switched to distro-provided one. > > > > > > > > > > > > > > This reduced number of errors, but I still see some: > > > > > > > > > > > > > > clang: error: unable to execute command: Broken pipe > > > > > > > clang: error: clang frontend command failed due to signal (use -v to > > > > > > > see invocation) > > > > > > > Makefile:259: recipe for target 'elfdep' failed > > > > > > > Makefile:156: recipe for target 'all' failed > > > > > > > Makefile:106: recipe for target > > > > > > > '/linux/tools/testing/selftests/bpf/libbpf.a' failed > > > > > > > > > > > > Getting bpf compile to work take a few steps. If I remember correctly, > > > > > > You will need llvm as well. Here is what I have on my system: > > > > > > > > > > > > ii libllvm6.0:amd 1:6.0-1ubunt amd64 Modular compiler and > > > > > > toolchain te > > > > > > ii llvm 1:6.0-41~exp amd64 Low-Level Virtual Machine > > > > > > (LLVM) > > > > > > ii llvm-6.0 1:6.0-1ubunt amd64 Modular compiler and > > > > > > toolchain te > > > > > > ii llvm-6.0-dev 1:6.0-1ubunt amd64 Modular compiler and > > > > > > toolchain te > > > > > > un llvm-6.0-doc <none> <none> (no description available) > > > > > > ii llvm-6.0-runti 1:6.0-1ubunt amd64 Modular compiler and > > > > > > toolchain te > > > > > > ii llvm-runtime 1:6.0-41~exp amd64 Low-Level Virtual Machine > > > > > > (LLVM), > > > > > > > > > > > > > timestamping.c:249:19: error: ‘SIOCGSTAMP’ undeclared (first use in > > > > > > > this function); did you mean ‘SIOCGSTAMPNS’? > > > > > > > ../../lib.mk:138: recipe for target > > > > > > > '/linux/tools/testing/selftests/networking/timestamping/timestamping' > > > > > > > failed > > > > > > > mlock-random-test.c:8:10: fatal error: sys/capability.h: No such file > > > > > > > or directory > > > > > > > > > > > > Do you have libcap-dev installed? > > > > > > > > > > > > ii libcap-dev:amd 1:2.25-1.2 amd64 POSIX 1003.1e capabilities > > > > > > (devel > > > > > > ii libcap-ng-dev 0.7.7-3.1 amd64 Development and header > > > > > > files for > > > > > > ii libcap-ng0:amd 0.7.7-3.1 amd64 An alternate POSIX > > > > > > capabilities l > > > > > > ii libcap2:amd64 1:2.25-1.2 amd64 POSIX 1003.1e capabilities > > > > > > (libra > > > > > > ii libcap2-bin 1:2.25-1.2 amd64 POSIX 1003.1e capabilities > > > > > > (utili > > > > > > un libcap2-dev <none> <none> (no description available) > > > > > > > > > > I've installed libcap-dev and resolved the missing header. > > > > > > > > > > I've also installed llvm llvm-6.0 llvm-6.0-dev llvm-6.0-doc > > > > > libllvm6.0 llvm-6.0-runtime llvm-runtime and it fixed crashing > > > > > compiler. > > > > > But bpf tests build was still failing due to missing libelf. But I had > > > > > the library, so I went and removed some random files: > > > > > tools/testing/selftests/bpf/{feature,FEATURE-DUMP.libbpf}. Don't ask > > > > > me why these. > > > > > > > > > > I am now down to just 1 build error: > > > > > > > > > > CC /usr/local/google/home/dvyukov/src/linux/tools/testing/selftests/bpf/str_error.o > > > > > timestamping.c:249:19: error: ‘SIOCGSTAMP’ undeclared (first use in > > > > > this function); did you mean ‘SIOCGSTAMPNS’? > > > > > > > > > > > > Is this a non-fatal error? Usually when make produces errors, one > > > > expects that nothing is done and it was aborted mid-way. But make > > > > seems to produce some test binaries by now. > > > > > > > > > > > > Reading the doc further, these command seem to implicitly assume that > > > > the tests will run right on my host machine: > > > > > > > > $ make -C tools/testing/selftests run_tests > > > > $ make kselftest > > > > > > > > Is it right? At least I don't see how it's configured to run them > > > > somewhere else? Or it uses something like qemu by default to run the > > > > kernel under test? > > > > If it runs the tests on the host, it can't work for me. I don't have > > > > the test kernel installed and there is no way I can do this. Policy > > > > rules aside, this is yet untested kernel, so by installing it I am > > > > risking losing my whole machine and all data... > > > > > > > > What am I missing? > > > > > > Reading further. "Install selftests" and "Running installed selftests" > > > sections. Is it something I can use to copy the pre-built tests to the > > > test machine? The sections don't spell it, so I am just trying to > > > second guess. Or what's the purpose of installing? > > > > > > The "Running installed selftests" section says: > > > "Kselftest install as well as the Kselftest tarball provide a script > > > named "run_kselftest.sh" to run the tests". > > > > > > What is the "Kselftest tarball"? Where does one get one? I don't see > > > any mentions of "tarball" anywhere else in the doc. > > > > > > > > Running ./kselftest_install.sh I am getting: > > > > /bin/sh: llvm-readelf: command not found > > > > I can't find any package that would provide this. I happened to have a > > custom llvm build that has that binary, but I am interested how I was > > supposed to get this for the purposes of documentation and reuse of > > instructions by others. > > > > After adding my llvm-readelf to PATH, I am then getting: > > > > make[1]: *** No rule to make target 'emit_tests'. Stop. > > > > Looks like an error, or is it? > > > > It produced something in the output dir, so copied that to the test > > machine and tried to run run_kselftest.sh there, but it failed too: > > > > ~/kselftest# ./run_kselftest.sh > > ./run_kselftest.sh: 2: ./run_kselftest.sh: realpath: not found > > ./run_kselftest.sh: 4: .: Can't open ./kselftest/runner.sh > > > > Is there some kind of prerequisites that I am supposed to install there? > > Since the target may have non-x86 arch and a custom distro, any > > additional dependency there may be very painful to get... > > > Hi Shuah, > > I am asking lots of questions, but I did not provide my motivation and end goal. > I am trying to understand overall state of the kernel testing better > and understand (1) if there are working instructions to run kernel > testing that I can give to a new team member or a new external kernel > developer, (2) if/how I can ask a kernel developer fixing a bug to add > a regression test and ensure that it works. Note in these cases a user > may not have lots of specific expertise (e.g. any unsaid/implicit > thing may be a showstopper) and/or don't have infinite motivation/time > (may give up given a single excuse to do so) and/or don't have > specific interest/expertise in the tested subsystem (e.g. a drive-by > fix). > So now I am trying to follow this route myself, documenting steps here. > > > Back to: > ./run_kselftest.sh: 2: ./run_kselftest.sh: realpath: not found > > dpkg on my host says that this binary comes from coreutils: > > $ dpkg -S realpath > coreutils: /usr/bin/realpath > > but installing coreutils does not help. Seems that I need to install > realpath itself. But this package happens to be broken on my test > distro: > > # apt-get install realpath > Reading package lists... Done > Building dependency tree... Done > The following NEW packages will be installed: > realpath > 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. > Need to get 16.7 kB of archives. > After this operation, 115 kB of additional disk space will be used. > Err http://deb.debian.org/debian/ wheezy/main realpath amd64 1.18 > 404 Not Found [IP: 151.101.120.204 80] > Failed to fetch > http://deb.debian.org/debian/pool/main/r/realpath/realpath_1.18_amd64.deb > 404 Not Found [IP: 151.101.120.204 80] > E: Unable to fetch some archives, maybe run apt-get update or try with > --fix-missing? > > > I've switched to another distro (fortunately I had another one > pre-built), and run_kselftest.sh started running tests. > But now I have even more questions :) > > 1. Meta-question: "Running a subset of selftests" section talks about > "subsystems". How can I map a source file I changed in a drive-by fix > to a subsystem? Say, I changed net/ipv6/netfilter/nft_redir_ipv6.c or > drivers/usb/c67x00/c67x00-drv.c, what subsystems do I need to run? > > 2. All C tests seem to fail with: > # ./test_maps: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.26' > not found (required by ./test_maps) > Which means that my image is still unsuitable. How can I get an image > that is suitable to run the tests? > > 3. Lots of tests that do run (probably shell tests), fail/skipped with > some cryptic for me errors like: > > # Cannot find device "ip6gre11" > > # selftests: [SKIP] Could not run test without the ip xdpgeneric support > > # modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could > not open moddep file '/lib/modules/5.1.0+/modules.dep.bin' > > # selftests: bpf: test_tc_edt.sh > # nc is not available > not ok 40 selftests: bpf: test_tc_edt.sh > > Say, I either want to run tests for a specific subsystem because I am > doing a drive-by fix (a typical newcomer/good Samaritan scenario), or > I want to run as many tests as possible (a typical CI scenario). Is > there a way to bulk satisfy all these prerequisite (configs, binaries > and whatever they are asking for)? > > 4. There is a test that consistently reboots my machine: > > # selftests: breakpoints: step_after_suspend_test > [ 514.024889] PM: suspend entry (deep) > [ 514.025959] PM: Syncing filesystems ... done. > [ 514.051573] Freezing user space processes ... (elapsed 0.001 seconds) done. > [ 514.054140] OOM killer disabled. > [ 514.054764] Freezing remaining freezable tasks ... (elapsed 0.001 > seconds) done. > [ 514.057695] printk: Suspending console(s) (use no_console_suspend to debug) > early console in extract_kernel > input_data: 0x0000000007ddc2e9 > input_len: 0x0000000002c26bf0 > output: 0x0000000001000000 > output_len: 0x0000000008492a48 > kernel_total_size: 0x0000000009a26000 > trampoline_32bit: 0x000000000009d000 > Decompressing Linux... Parsing ELF... done. > Booting the kernel. > [ 0.000000] Linux version 5.0.0 (gcc version 7.3.0 (Debian > 7.3.0-18)) #7 SMP PREEMPT Wed Jun 12 11:38:12 CEST 2019 > > Is it a bug in the test? in the kernel? Or how is this supposed to > work/what am I supposed to do with this? > > > 5. There is a test that triggers a use-after-free: > > [ 262.639848][ C1] BUG: KASAN: use-after-free in > ip6gre_tunnel_lookup+0x1a27/0x1ae0 > > I went back to v5.0 (3+ months ago), and I see that it's also the case there. > Do you know if anybody running these tests? With KMEMLEAK, LOCKDEP, etc? > > 6. Do we know what's the current code coverage achieved by these > tests? What's covered? What's not? Overall percent/per-subsystem/etc? > > Thanks I've deleted the test that caused reboot for now and the run_kselftest.sh script finished running, but I can't understand the result. Is it all passed? Or something failed? What failed? The output is as follows: ... screens of output ... # make swap with zram device(s) [ 5335.215175] Adding 1020k swap on /dev/zram0. Priority:-2 extents:1 across:1020k SSFS # done with /dev/zram0 # zram making zram mkswap and swapon: OK # zram swapoff: OK # zram cleanup [ 5335.420825] zram0: detected capacity change from 1048576 to 0 # zram02 : [PASS] ok 1 selftests: zram: zram.sh ~/kselftest# docs say: "The above commands by default run the tests and print full pass/fail report. Kselftest supports "summary" option to make it easier to understand the test results.... $ make summary=1 kselftest " But this is for make. How can I understand the result with run_kselftest.sh?
