Re: [PATCH v2 fstests] check: add support for --start-after

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 2024-02-15 17:08:02, Luis Chamberlain wrote:
> Often times one is running a new test baseline we want to continue to
> start testing where we left off if the last test was a crash. To do
> this the first thing that occurred to me was to use the check.time
> file as an expunge file but that doesn't work so well if you crashed
> as the file turns out empty.
> 
> So instead add super simple argument --start-after which let's you
> skip all tests until the test infrastructure has "seen" the test
> you want to skip. This does obviously work best if you are not using
> a random order, but that is rather implied. If you do use a random
> order --start-after still works, the final output will however just
> be randomized of course, but it should let you skip a failed known
> crash at least. The real value to --start-after though is for when
> you use a non-randomized order.
> 
> If the target test is not found in your test list we complain and
> bail. This is not as obvious when you specify groups, so likewise
> we do a special check when you use groups to ensure the test is at
> least part of one group.
> 
> Demo:
> 
> root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink -n -g soak --start-after generic/025
> Start after test generic/025 not found in any group specified.
> Be sure you specify a test present in one of your test run groups if using --start-after.
> 
> Your set of groups have these tests:
> 
> generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/650
> 
> root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink -n -g soak --start-after generic/522
> SECTION       -- xfs_reflink
> RECREATING    -- xfs on /dev/loop16
> FSTYP         -- xfs (non-debug)
> PLATFORM      -- Linux/x86_64 demo-xfs-reflink 6.5.0-5-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.13-1 (2023-11-29)
> MKFS_OPTIONS  -- -f -f -m reflink=1,rmapbt=1, -i sparse=1, /dev/loop5
> MOUNT_OPTIONS -- /dev/loop5 /media/scratch
> 
> generic/616
> generic/617
> generic/642
> generic/650
> 
> Signed-off-by: Luis Chamberlain <mcgrof@xxxxxxxxxx>
> ---
> 
> Changes since v1:
> 
> This all addresses Anand Jain's feedback.
> 
>  - Skip tests completely which are not going to be run
>  - Sanity test to ensure the test is part of a group, if you listed
>    groups, and if not provide a useful output giving the list of all
>    tests in your group so you can know better which one is a valid test
>    to skip
>  - Sanity test to ensure the test you specified is valid
>  - Moves the trim during file processing now using a routine
>    trim_start_after()
> 
>  check | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 52 insertions(+)
> 
> diff --git a/check b/check
> index 71b9fbd07522..1c76f33192ba 100755
> --- a/check
> +++ b/check
> @@ -18,6 +18,8 @@ showme=false
>  have_test_arg=false
>  randomize=false
>  exact_order=false
> +start_after=false
> +start_after_test=""
>  export here=`pwd`
>  xfile=""
>  subdir_xfile=""
> @@ -80,6 +82,7 @@ check options
>      -b			brief test summary
>      -R fmt[,fmt]	generate report in formats specified. Supported formats: xunit, xunit-quiet
>      --large-fs		optimise scratch device for large filesystems
> +    --start-after	only start testing after the test specified
>      -s section		run only specified section from config file
>      -S section		exclude the specified section from the config file
>      -L <n>		loop tests <n> times following a failure, measuring aggregate pass/fail metrics
> @@ -120,6 +123,8 @@ examples:
>   check -x stress xfs/*
>   check -X .exclude -g auto
>   check -E ~/.xfstests.exclude
> + check --start-after btrfs/010
> + check -n -g soak --start-after generic/522
>  '
>  	    exit 1
>  }
> @@ -204,6 +209,24 @@ trim_test_list()
>  	rm -f $tmp.grep
>  }
>  
> +# takes the list of tests to run in $tmp.list and skips all tests until
> +# the specified test is found. This will ensure the tests start after the
> +# test specified, it skips the test specified.
> +trim_start_after()
> +{
> +	local skip_test="$1"
> +	local starts_regexp=$(echo $skip_test | sed -e 's|\/|\\/|')
> +	local grep_start_after=" | awk 'f;/.*'$starts_regexp'/{f=1}'"

Looks like grep_start_after is not used
Otherwise, LGTM:
Reviewed-by: Andrey Albershteyn <aalbersh@xxxxxxxxxx>

-- 
- Andrey





[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux