On 2/5/20 3:09 PM, Russell Currey wrote:
On Thu, Feb 6, 2020, at 7:00 AM, Brendan Higgins wrote:
On Wed, Feb 5, 2020 at 9:58 AM David Gow <davidgow@xxxxxxxxxx> wrote:
One thing we'd like to do with kunit_tool is to make its functionality
a bit more independent: in particular, allowing the configuration,
running the kernel, and parsing the results to be done independently.
If that's the case, it may make sense for "kunit.py run" or similar to
not do anything with the .config, and to relegate that to a separate
"configuration" step, which would allow someone to modify the
configuration themselves (e.g., using make menuconfig) and re-run the
tests, but also allow the config to be explicitly regenerated when
helpful.
Exactly what that'd end up looking like (and to what extent we'd still
want to support a single command that'd do both) are still up in the
air: but I think a general "separation of concerns" like this is
probably the right path forward for kunit_tool.
You and I have talked about splitting up kunit_tool's functionality
before. I agree with the idea.
I imagine it that we would have
- configuration
- running tests
- dmesg/TAP parsing
as separate runnable scripts. I think that would make it a lot easier
for people with various test bed setups to reuse our code in their
test harness.
Nevertheless, I think it would also be nice to have, as Ted has
previously suggested, a short easy to remember one line command that
just works; it is easily said, and much harder to do, but I think it
is at odds with the separation of functionality. I guess one solution
might just be to have these three separate tools, and then the classic
kunit.py script that combines the functionalities in a single step, or
as Ted suggested we could have some sort of default "make kunit"
command or something like that. I am not really sure what is best
here.
It doesn't address the problem of separation of functionality in
anyway, but one way we could achieve the idea of having a command that
just works, is by putting a line in MAINTAINERS file entries that have
a command that a maintainer expects a submitter to run before sending
a patch to LKML. That might at least make it possible to hack together
a single line KUnit command for every relevant MAINTAINERS entry.
(Obviously there is no reason we have to do this particular idea just
for KUnit. We could do this for other tests as well.) Russel, I think
this was your idea at LCA?
Hi Brendan, it wasn't me, it was someone in the audience during questions in my
testing talk. I don't recall who.
They were suggesting a script like get_maintainers - i.e. get_tests - that for a
given file/patch/commit it gives you a suggested set of tests, whether that's
KUnit you can run there and then, or selftests you can run once it's booted,
or maybe external test suites that are relevant.
I like this idea of get_tests type script that could be run separately
as well as part of check_patch or get_maintainers will serve as a
reminder or hint to patch submitter.
We have some pieces in the MAINTAINERS file now. Selftest files are
usually listed under subsystem entries. get_tests could leverage
that and we will definitely more information to for a complete set
of tests for a subsystem.
thanks,
-- Shuah
