On Sun 2020-10-25 22:48:41, Rasmus Villemoes wrote:
> Some test suites make use of random numbers to increase the test
> coverage when the test suite gets run on different machines and
> increase the chance of some corner case bug being discovered - and I'm
> planning on extending some existing ones in that direction as
> well. However, should a bug be found this way, it's important that the
> exact same series of tests can be repeated to verify the bug is
> fixed. That means the random numbers must be obtained
> deterministically from a generator private to the test module.
>
> To avoid adding boilerplate to various test modules, put some logic
> into kselftest_module.h: If the module declares that it will use
> random numbers, add a "seed" module parameter. If not explicitly given
> when the module is loaded (or via kernel command line), obtain a
> random one. In either case, print the seed used, and repeat that
> information if there was at least one test failing.
>
> Signed-off-by: Rasmus Villemoes <linux@xxxxxxxxxxxxxxxxxx>
> ---
> tools/testing/selftests/kselftest_module.h | 35 ++++++++++++++++++++--
> 1 file changed, 32 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/kselftest_module.h b/tools/testing/selftests/kselftest_module.h
> index c81c0b0c054befaf665b..43f3ca58fcd550b8ac83 100644
> --- a/tools/testing/selftests/kselftest_module.h
> +++ b/tools/testing/selftests/kselftest_module.h
> @@ -3,14 +3,31 @@
> #define __KSELFTEST_MODULE_H
>
> #include <linux/module.h>
> +#include <linux/prandom.h>
> +#include <linux/random.h>
>
> /*
> * Test framework for writing test modules to be loaded by kselftest.
> * See Documentation/dev-tools/kselftest.rst for an example test module.
> */
>
> +/*
> + * If the test module makes use of random numbers, define KSTM_RANDOM
> + * to 1 before including this header. Then a module parameter "seed"
> + * will be defined. If not given, a random one will be obtained. In
> + * either case, the used seed is reported, so the exact same series of
> + * tests can be repeated by loading the module with that seed
> + * given.
> + */
> +
> +#ifndef KSTM_RANDOM
> +#define KSTM_RANDOM 0
> +#endif
> +
> static unsigned int total_tests __initdata;
> static unsigned int failed_tests __initdata;
> +static struct rnd_state rnd_state __initdata;
> +static u64 seed __initdata;
>
> #define KSTM_CHECK_ZERO(x) do { \
> total_tests++; \
> @@ -22,11 +39,13 @@ static unsigned int failed_tests __initdata;
>
> static inline int kstm_report(unsigned int total_tests, unsigned int failed_tests)
> {
> - if (failed_tests == 0)
> + if (failed_tests == 0) {
> pr_info("all %u tests passed\n", total_tests);
> - else
> + } else {
> pr_warn("failed %u out of %u tests\n", failed_tests, total_tests);
> -
> + if (KSTM_RANDOM)
> + pr_info("random seed used was 0x%016llx\n", seed);
I have a bit mixed feelings about this. It is genial and dirty hack at the
same time ;-) Well, it is basically the same approach as with
IS_ENABLED(CONFIG_bla_bla).
Reviewed-by: Petr Mladek <pmladek@xxxxxxxx>
Best Regards,
Petr
