On Thu, 13 Jun 2019 at 02:43, Daniel Díaz <daniel.diaz@xxxxxxxxxx> wrote: > > Hello! > > On Wed, 12 Jun 2019 at 14:32, shuah <shuah@xxxxxxxxxx> wrote: > > On 6/12/19 12:29 PM, Dmitry Vyukov wrote: > [...] > > > 1. You suggested to install a bunch of packages. That helped to some > > > degree. Is there a way to figure out what packages one needs to > > > install to build the tests other than asking you? > > > > I have to go through discovery at times when new tests get added. I > > consider this a part of being a open source developer figuring out > > dependencies for compiling and running. I don't have a magic answer > > for you and there is no way to make sure all dependencies will be > > documented. > > This is something we, as users of Kselftests, would very much like to > see improved. We also go by trial-and-error finding out what is > missing, but keeping up with the new tests or subsystems is often > difficult and tend to remain broken (in usage) for some time, until we > have the resources to look into that and fix it. The config fragments > is an excellent example of how the test developers and the framework > complement each other to make things work. Even documenting > dependencies would go a long way, as a starting point, but I do > believe that the test writers should do that and not the users go > figure out what all is needed to run their tests. > > Maybe a precheck() on the tests in order to ensure that the needed > binaries are around? > > For what it's worth, this is the list of run-time dependencies package > for OpenEmbedded: bash bc ethtool fuse-utils iproute2 iproute2-tc > iputils-ping iputils-ping6 ncurses perl sudo python3-argparse > python3-datetime python3-json python3-pprint python3-subprocess > util-linux-uuidgen cpupower glibc-utils. We are probably missing a > few. > > [...] > > > 10. Do you know if anybody is running kselftests? Running as in > > > running continuously, noticing new failures, reporting these failures, > > > keeping them green, etc. > > > I am asking because one of the tests triggers a use-after-free and I > > > checked it was the same 3+ months ago. And I have some vague memories > > > of trying to run kselftests 3 or so years ago, and there was a bunch > > > of use-after-free's as well. > > > > Yes Linaro test rings run them and kernel developers do. I am cc'ing > > Naresh and Anders to help with tips on how they run tests in their > > environment. They have several test systems that they install tests > > and run tests routine on all stable releases. > > > > Naresh and Anders! Can you share your process for running kselftest > > in Linaro test farm. Thanks in advance. > > They're both in time zones where it's better to be sleeping at the > moment, so I'll let them chime in with more info tomorrow (their > time). I can share that we, as part of LKFT [1], run Kselftests with > Linux 4.4, 4.9, 4.14, 4.19, 5.1, Linus' mainline, and linux-next, on > arm, aarch64, x86, and x86-64, *very* often: Our test counter recently > exceeded 5 million! You can see today's mainline results of Kselftests > [2] and all tests therein. Thanks Daniel. In the recent past we have found kernel oops, bugs and warnings while running kselftest suite on our environment. It is worth running them in CI. Linaro 's test farm have been reporting these issues to kernel sub-system maintainers and test authors and they have investigated and fixed. The test cases which are known to fail due to missing dependency it could be Kconfig or userland packages. There is one more case we see failures when running latest test cases on older kernel branches. We have marked them as known failures XFAIL [3]. qa reports will parse actual results and applies xfails as blue in color. Best regards Naresh Kamboju > > Daniel Díaz > daniel.diaz@xxxxxxxxxx > > [1] https://lkft.linaro.org/ > [2] https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v5.2-rc4-20-gaa7235483a83/#!?details=175 [3] https://github.com/Linaro/qa-reports-known-issues/blob/master/kselftests-production.yaml
