On 2024-02-15 17:08:02, Luis Chamberlain wrote:
> Often times one is running a new test baseline we want to continue to
> start testing where we left off if the last test was a crash. To do
> this the first thing that occurred to me was to use the check.time
> file as an expunge file but that doesn't work so well if you crashed
> as the file turns out empty.
>
> So instead add super simple argument --start-after which let's you
> skip all tests until the test infrastructure has "seen" the test
> you want to skip. This does obviously work best if you are not using
> a random order, but that is rather implied. If you do use a random
> order --start-after still works, the final output will however just
> be randomized of course, but it should let you skip a failed known
> crash at least. The real value to --start-after though is for when
> you use a non-randomized order.
>
> If the target test is not found in your test list we complain and
> bail. This is not as obvious when you specify groups, so likewise
> we do a special check when you use groups to ensure the test is at
> least part of one group.
>
> Demo:
>
> root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink -n -g soak --start-after generic/025
> Start after test generic/025 not found in any group specified.
> Be sure you specify a test present in one of your test run groups if using --start-after.
>
> Your set of groups have these tests:
>
> generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/650
>
> root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink -n -g soak --start-after generic/522
> SECTION -- xfs_reflink
> RECREATING -- xfs on /dev/loop16
> FSTYP -- xfs (non-debug)
> PLATFORM -- Linux/x86_64 demo-xfs-reflink 6.5.0-5-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.13-1 (2023-11-29)
> MKFS_OPTIONS -- -f -f -m reflink=1,rmapbt=1, -i sparse=1, /dev/loop5
> MOUNT_OPTIONS -- /dev/loop5 /media/scratch
>
> generic/616
> generic/617
> generic/642
> generic/650
>
> Signed-off-by: Luis Chamberlain <mcgrof@xxxxxxxxxx>
> ---
>
> Changes since v1:
>
> This all addresses Anand Jain's feedback.
>
> - Skip tests completely which are not going to be run
> - Sanity test to ensure the test is part of a group, if you listed
> groups, and if not provide a useful output giving the list of all
> tests in your group so you can know better which one is a valid test
> to skip
> - Sanity test to ensure the test you specified is valid
> - Moves the trim during file processing now using a routine
> trim_start_after()
>
> check | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 52 insertions(+)
>
> diff --git a/check b/check
> index 71b9fbd07522..1c76f33192ba 100755
> --- a/check
> +++ b/check
> @@ -18,6 +18,8 @@ showme=false
> have_test_arg=false
> randomize=false
> exact_order=false
> +start_after=false
> +start_after_test=""
> export here=`pwd`
> xfile=""
> subdir_xfile=""
> @@ -80,6 +82,7 @@ check options
> -b brief test summary
> -R fmt[,fmt] generate report in formats specified. Supported formats: xunit, xunit-quiet
> --large-fs optimise scratch device for large filesystems
> + --start-after only start testing after the test specified
> -s section run only specified section from config file
> -S section exclude the specified section from the config file
> -L <n> loop tests <n> times following a failure, measuring aggregate pass/fail metrics
> @@ -120,6 +123,8 @@ examples:
> check -x stress xfs/*
> check -X .exclude -g auto
> check -E ~/.xfstests.exclude
> + check --start-after btrfs/010
> + check -n -g soak --start-after generic/522
> '
> exit 1
> }
> @@ -204,6 +209,24 @@ trim_test_list()
> rm -f $tmp.grep
> }
>
> +# takes the list of tests to run in $tmp.list and skips all tests until
> +# the specified test is found. This will ensure the tests start after the
> +# test specified, it skips the test specified.
> +trim_start_after()
> +{
> + local skip_test="$1"
> + local starts_regexp=$(echo $skip_test | sed -e 's|\/|\\/|')
> + local grep_start_after=" | awk 'f;/.*'$starts_regexp'/{f=1}'"
Looks like grep_start_after is not used
Otherwise, LGTM:
Reviewed-by: Andrey Albershteyn <aalbersh@xxxxxxxxxx>
--
- Andrey
