On 4/15/23 12:38 AM, Luis Chamberlain wrote: > The default timeout for selftests tests is 45 seconds. Although > we already have 13 settings for tests of about 96 sefltests which > use a timeout greater than this, we want to try to avoid encouraging > more tests to forcing a higher test timeout as selftests strives to > run all tests quickly. Selftests also uses the timeout as a non-fatal > error. Only tests runners which have control over a system would know > if to treat a timeout as fatal or not. > > To help with all this: > > o Enhance documentation to avoid future increases of insane timeouts > o Add the option to allow overriding the default timeout with test > runners with a command line option > > Suggested-by: Shuah Khan <skhan@xxxxxxxxxxxxxxxxxxx> > Signed-off-by: Luis Chamberlain <mcgrof@xxxxxxxxxx> Reviewed-by: Muhammad Usama Anjum <usama.anjum@xxxxxxxxxxxxx> Tested-by:Muhammad Usama Anjum <usama.anjum@xxxxxxxxxxxxx> > --- > Documentation/dev-tools/kselftest.rst | 22 +++++++++++++++++++++ > tools/testing/selftests/kselftest/runner.sh | 11 ++++++++++- > tools/testing/selftests/run_kselftest.sh | 5 +++++ > 3 files changed, 37 insertions(+), 1 deletion(-) > > diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst > index 12b575b76b20..dd214af7b7ff 100644 > --- a/Documentation/dev-tools/kselftest.rst > +++ b/Documentation/dev-tools/kselftest.rst > @@ -168,6 +168,28 @@ the `-t` option for specific single tests. Either can be used multiple times:: > > For other features see the script usage output, seen with the `-h` option. > > +Timeout for selftests > +===================== > + > +Selftests are designed to be quick and so a default timeout is used of 45 > +seconds for each test. Tests can override the default timeout by adding > +a settings file in their directory and set a timeout variable there to the > +configured a desired upper timeout for the test. Only a few tests override > +the timeout with a value higher than 45 seconds, selftests strives to keep > +it that way. Timeouts in selftests are not considered fatal because the > +system under which a test runs may change and this can also modify the > +expected time it takes to run a test. If you have control over the systems > +which will run the tests you can configure a test runner on those systems to > +use a greater or lower timeout on the command line as with the `-o` or > +the `--override-timeout` argument. For example to use 165 seconds instead > +one would use: > + > + $ ./run_kselftest.sh --override-timeout 165 > + > +You can look at the TAP output to see if you ran into the timeout. Test > +runners which know a test must run under a specific time can then optionally > +treat these timeouts then as fatal. > + > Packaging selftests > =================== > > diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh > index 294619ade49f..1c952d1401d4 100644 > --- a/tools/testing/selftests/kselftest/runner.sh > +++ b/tools/testing/selftests/kselftest/runner.sh > @@ -8,7 +8,8 @@ export logfile=/dev/stdout > export per_test_logging= > > # Defaults for "settings" file fields: > -# "timeout" how many seconds to let each test run before failing. > +# "timeout" how many seconds to let each test run before running > +# over our soft timeout limit. > export kselftest_default_timeout=45 > > # There isn't a shell-agnostic way to find the path of a sourced file, > @@ -90,6 +91,14 @@ run_one() > done < "$settings" > fi > > + # Command line timeout overrides the settings file > + if [ -n "$kselftest_override_timeout" ]; then > + kselftest_timeout="$kselftest_override_timeout" > + echo "# overriding timeout to $kselftest_timeout" >> "$logfile" > + else > + echo "# timeout set to $kselftest_timeout" >> "$logfile" > + fi > + > TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST" > echo "# $TEST_HDR_MSG" > if [ ! -e "$TEST" ]; then > diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh > index 97165a83df63..9a981b36bd7f 100755 > --- a/tools/testing/selftests/run_kselftest.sh > +++ b/tools/testing/selftests/run_kselftest.sh > @@ -26,6 +26,7 @@ Usage: $0 [OPTIONS] > -l | --list List the available collection:test entries > -d | --dry-run Don't actually run any tests > -h | --help Show this usage info > + -o | --override-timeout Number of seconds after which we timeout > EOF > exit $1 > } > @@ -33,6 +34,7 @@ EOF > COLLECTIONS="" > TESTS="" > dryrun="" > +kselftest_override_timeout="" > while true; do > case "$1" in > -s | --summary) > @@ -51,6 +53,9 @@ while true; do > -d | --dry-run) > dryrun="echo" > shift ;; > + -o | --override-timeout) > + kselftest_override_timeout="$2" > + shift 2 ;; > -h | --help) > usage 0 ;; > "") -- BR, Muhammad Usama Anjum
