On Sat, 2 May 2020 at 04:11, David Gow <davidgow@xxxxxxxxxx> wrote: > > On Sat, May 2, 2020 at 4:31 AM Brendan Higgins > <brendanhiggins@xxxxxxxxxx> wrote: > > > > On Fri, May 1, 2020 at 1:35 AM Anders Roxell <anders.roxell@xxxxxxxxxx> wrote: > > > > > > Make it easier to enable all KUnit fragments. This is needed for kernel > > > test-systems, so its easy to get all KUnit tests enabled and if new gets > > > added they will be enabled as well. Fragments that has to be builtin > > > will be missed if CONFIG_KUNIT_RUN_ALL is set as a module. > > > > > > Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if > > > someone wants that even though KUNIT_RUN_ALL is enabled. > > > > I would LOVE IT, if you could make this work! I have been trying to > > figure out the best way to run all KUnit tests for a long time now. > > > > That being said, I am a bit skeptical that this approach will be much > > more successful than just using allyesconfig. Either way, there are > > tests coming down the pipeline that are incompatible with each other > > (the KASAN test and the KCSAN test will be incompatible). Even so, > > tests like the apparmor test require a lot of non-default > > configuration to compile. In the end, I am not sure how many tests we > > will really be able to turn on this way. > > > > Thoughts? > > I think there's still some value in this which the allyesconfig option > doesn't provide. As you point out, it's not possible to have a generic > "run all tests" option due to potential conflicting dependencies, but > this does provide a way to run all tests for things enabled in the > current config. This could be really useful for downstream developers > who want a way of running all tests relevant to their config without > the overhead of running irrelevant tests (e.g., for drivers they don't > build). It will solve that as well as for a tester doesn't have to go through all KUnit tests fragments to turn them on. > Using allyesconfig doesn't make that distinction. We could also create a config fragment file in kernel/configs/kunit.config where we set ------start CONFIG_KUNIT=y CONFIG_KUNIT_RUN_ALL=y CONFIG_SECURITY_APPARMOR=y ------end So, these two can only be enabled if KUNIT=y CONFIG_KUNIT_DRIVER_PE_TEST=y CONFIG_PM_QOS_KUNIT_TEST=y and for this one we have a pre-request of SECURITY_APPARMOR=y CONFIG_SECURITY_APPARMOR_KUNIT_TEST=y Other tests solves the dependencies with 'select' like CONFIG_EXT4_KUNIT_TESTS, that adds this row in fs/ext4/Kconfig, 'select EXT4_FS' But I think we should try to minimize the number of 'select' statements, in order to avoid circular dependencies and unexpected behaviours. Maybe we should add the CONFIG_EXT4_FS=y into the kunit.config file instead ? > > Ultimately, we'll probably still want something which enables a > broader set of tests for upstream development: whether that's based on > this, allyesconfig, or something else entirely remains to be seen, I > think. I suspect we're going to end up with something > subsystem-specific (having a kunitconfig per subsystem, or a testing > line in MAINTAINERS or similar are ideas which have been brought up in > the past). > > This is a great looking tool to have in the toolbox, though. I agree! I'll prepare a patchset with individual patches as was suggested by Marco shortly. Cheers, Anders
