On Sat, May 26, 2018 at 08:35:40PM +0300, Dima Stepanov wrote: > On Sat, May 26, 2018 at 04:09:29PM +0200, Radim Krčmář wrote: > > 2018-05-26 16:37+0300, Dima Stepanov: > > > During custom linux kernel development it was found out that sometimes it > > > is really useful to exclude the tests from automatic CI step. It is not > > > always easy to modify the unittest.cfg file so the exclude options were > > > added to the test framework: > > > -x: Exclude tests from running > > > -e: Exclude test groups from running > > > As a result the command like: > > > ./run_tests.sh -e "vmexit vmx" -x "apic vmware_backdoors" > > > Will exclude apic and vmware_backdoors tests from running. And will > > > exclude vmexit and vmx groups from running. If -a option is set, then > > > both options will be ignored. > > > The changes below are to share the fix for the kvm unit test framework. > > > > > > Add two custom options to exclude tests and groups from running. It can be > > > useful for running the tests automatically in the custom environment > > > without modifying the unittest.cfg file. > > > > > > Signed-off-by: Dima Stepanov <dstepanov.src@xxxxxxxxx> > > > --- > > > > Good idea! I'm not applying it right now only because I wanted to > > gather more opinions about the "-a" option. > Agree, lets gather more opinions. > > > > > > diff --git a/scripts/runtime.bash b/scripts/runtime.bash > > > @@ -76,6 +76,18 @@ function run() > > > return; > > > fi > > > > > > + if [ -n "$exclude_groups" ] && grep -qw "$groups" <<<$exclude_groups && > > > + skip_nodefault; then > > > > I'd drop the skip_nodefault. > > > > Even though the "-a" option says "run all test", I think that when the > > user says "-a -e undesired", the expectation would be to run tests with > > group nodefault and still exclude undesired groups, just like we do with > > "-o". (Documentation of "-a" would be need to be changed a little.) > I'm okay with such behaviour. Anyway i think it will be pretty unusual > case. > > > > > > + echo -e "`SKIP` $testname (test group is marked as exclude -e option)" > > > > I'd prefer a concise message, maybe "excluded by -e"? > > > Yes, it sounds better, will update it after other review comments. > > Thanks, Dima. > > > Thanks. I like the idea and Radim's suggestions. Thanks, drew
