When using kvm-unit-tests inside automated testing frameworks, variable test naming becomes a problem. Some frameworks recognise tests by their test output line and group the outputs from various runs for statistical and reporting purposes. Having variable output like timer values in there spoils this approach. Also the test name should be somewhat self-explanatory, which is not true for every test. Some examples highlighting the problem (TAP output from run-tests.sh -t): ok 1 - selftest: setup: smp: nr_cpus = 2 ok 2 - selftest: setup: mem: size = 256 MB ok 8 - selftest: smp: CPU( 1) mpidr=0080000001 ok 9 - selftest: smp: CPU( 2) mpidr=0080000002 ok 54 - gicv2: mmio: ITARGETSR: byte writes successful (0x1f => 0x01010001) ok 55 - gicv2: mmio: all 3 CPUs have interrupts ok 73 - invalid-function ok 76 - cpu-on ok 90 - ptimer-busy-loop: timer has expired (-8445) This series aims to fix most of the problems, by making the actual test report output line stable. I think this is best practises in the testing world, at least when using TAP. We still retain the full information, by moving every variable output into INFO: lines (which are still logged, but typically filtered for automated processing). The above lines now look like this: ok 1 - selftest: setup: smp: number of CPUs matches expectation ok 2 - selftest: setup: mem: memory size matches expectation ok 8 - selftest: smp: MPIDR test on all CPUs ok 49 - gicv2: mmio: ITARGETSR: byte writes successful ok 50 - gicv2: mmio: all CPUs have interrupts ok 68 - psci: invalid-function ok 71 - psci: cpu-on ok 85 - ptimer-busy-loop: timer has expired Looks a bit more boring, but it's nicer for automated processing and logging. I am open for a discussion about the general approach, thus this is dealing with ARM tests for now only. Looking forward to any feedback! Cheers, Andre Andre Przywara (6): arm: gic: check_acked: add test description arm: gic: Split variable output data from test name arm: timer: Split variable output data from test name arm: selftest: Split variable output data from test name arm: selftest: Make MPIDR output stable arm: Add missing test name prefix calls arm/gic.c | 64 ++++++++++++++++++++++++++++++-------------------- arm/pci-test.c | 2 ++ arm/psci.c | 2 ++ arm/selftest.c | 23 ++++++++++++++---- arm/timer.c | 3 ++- 5 files changed, 62 insertions(+), 32 deletions(-) -- 2.17.1
