Hi Tom, On 06/08/2017 09:37 AM, Tom Gall wrote: > We've been running kselftests for ARM and x86 hardware in an effort to > detect regressions in various kernels including LTS and candidate > patches. I am glad to hear that kselftests are being run on LTS and candidate patches. > > One general question we've had is what's the right thing to do when it > comes to running kselftest on older LTS kernels. Let's pick on 4.4 for > the purposes of this discussion tho obviously the example applies to > other versions. > > 1) Run current top of tree, kselftest on a 4.4 LTS? Yes. Absolutely. Some tests that verify a a new feature won't run and tests should fail gracefully without impacting the test run. If there are any problems with any tests, please report them to me. Tests detect feature and config dependencies before running. > 2) Run kselftest from 4.4 on 4.4 LTS? Yes. If you prefer to run the ksefltest from the 4.4 on 4.4, you are welcome to do so. Any fixes to existing tests might be included in stable release if they are marked for inclusion. This is no different from the stable release process. Any fixes to existing tests might be included in the stable as per the process. > > #1 gets latest greatest set of tests but obviously there can be > breakage because of how the kernel evolves over time. I don't expect this and if it happens, it is something that needs fixing. New tests and enhancements to existing tests to cover a new feature should not fail on older kernels. Tests detect feature based on module presence and/or configuration status before the test is run. There could be some bugs and tests might misbehave, however it isn't by design. Please reprot any such instances. > #2 misses tests that are later developed but otherwise fine > > Regardless of the right answer, some obvious questions but I'll ask anyway: > -) Shouldn't new additions to kselftest that work on older kernels be > backported and submitted on the stable list? Not all new tests are back-ported. There are two categories of changes that happen in kselftest sub-system 1. New tests for new features - These will not be back-ported and however these new tests should detect feature before they run. They shouldn't compromise test run. 2. Existing test changes to cover enhancements/changes to kernel features - These will not be back-ported and however these new tests should detect feature before they run. They shouldn't compromise test run. 3. Fixes to existing tests. - Some of these will go into stables kernels following the stable release process. 4. Kselftest infrastructure changes - These will fall in the bucket of new features and will not be back-ported. > -) In the case where some test is kernel version specific for whatever > reason, I presume building in version detection into kselftest makes > sense? Feature detection yes, but not release detection. Feature detection via a presence of a module or config enablement is the sustainable way for maintainability as well as usefulness. So I would say release detection doesn't make sense. Hope this helps. thanks, -- Shuah