On Fri, 1 Sept 2023 at 05:49, Michal Wajdeczko
<michal.wajdeczko@xxxxxxxxx> wrote:
>
> If we skip one parametrized test case then test status remains
> SKIP for all subsequent test params leading to wrong reports:
>
> $ ./tools/testing/kunit/kunit.py run \
> --kunitconfig ./lib/kunit/.kunitconfig *.example_params*
> --raw_output \
>
> [ ] Starting KUnit Kernel (1/1)...
> KTAP version 1
> 1..1
> # example: initializing suite
> KTAP version 1
> # Subtest: example
> # module: kunit_example_test
> 1..1
> KTAP version 1
> # Subtest: example_params_test
> # example_params_test: initializing
> # example_params_test: cleaning up
> ok 1 example value 3 # SKIP unsupported param value 3
> # example_params_test: initializing
> # example_params_test: cleaning up
> ok 2 example value 2 # SKIP unsupported param value 3
> # example_params_test: initializing
> # example_params_test: cleaning up
> ok 3 example value 1 # SKIP unsupported param value 3
> # example_params_test: initializing
> # example_params_test: cleaning up
> ok 4 example value 0 # SKIP unsupported param value 0
> # example_params_test: pass:0 fail:0 skip:4 total:4
> ok 1 example_params_test # SKIP unsupported param value 0
> # example: exiting suite
> ok 1 example # SKIP
>
> Reset test status and status comment after each param iteration
> to avoid using stale results.
>
> Signed-off-by: Michal Wajdeczko <michal.wajdeczko@xxxxxxxxx>
> Cc: David Gow <davidgow@xxxxxxxxxx>
> Cc: Rae Moar <rmoar@xxxxxxxxxx>
> ---
Nice catch, thanks!
This looks good to me.
Reviewed-by: David Gow <davidgow@xxxxxxxxxx>
Cheers,
-- David
> lib/kunit/kunit-example-test.c | 5 +++--
> lib/kunit/test.c | 6 ++++--
> 2 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
> index 01a769f35e1d..6bb5c2ef6696 100644
> --- a/lib/kunit/kunit-example-test.c
> +++ b/lib/kunit/kunit-example-test.c
> @@ -190,6 +190,7 @@ static void example_static_stub_test(struct kunit *test)
> static const struct example_param {
> int value;
> } example_params_array[] = {
> + { .value = 3, },
> { .value = 2, },
> { .value = 1, },
> { .value = 0, },
> @@ -213,8 +214,8 @@ static void example_params_test(struct kunit *test)
> KUNIT_ASSERT_NOT_NULL(test, param);
>
> /* Test can be skipped on unsupported param values */
> - if (!param->value)
> - kunit_skip(test, "unsupported param value");
> + if (!is_power_of_2(param->value))
> + kunit_skip(test, "unsupported param value %d", param->value);
>
> /* You can use param values for parameterized testing */
> KUNIT_EXPECT_EQ(test, param->value % param->value, 0);
I'm a little tempted to change this to something more power-of-two
specific now, like param->value & (param->value - 1). Thoughts?
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index 49698a168437..a53fd7e6d5bf 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -648,12 +648,14 @@ int kunit_run_tests(struct kunit_suite *suite)
> param_desc,
> test.status_comment);
>
> + kunit_update_stats(¶m_stats, test.status);
> +
> /* Get next param. */
> param_desc[0] = '\0';
> test.param_value = test_case->generate_params(test.param_value, param_desc);
> test.param_index++;
> -
> - kunit_update_stats(¶m_stats, test.status);
> + test.status = KUNIT_SUCCESS;
> + test.status_comment[0] = '\0';
> }
> }
>
> --
> 2.25.1
>
