On Sun, Apr 17, 2022 at 08:40:23PM +0300, Amir Goldstein wrote:
> Some tests are written before a fix is merged upstream and some tests
> are written without a known available fix at all. Such is the case with
> test overlay/061.
>
> Introduce a helper _known_issues_on_fs() which can be used to document
> this situation and print a hint on failure.
> The helper supports specifying specifc fs with the known issue for
> generic tests as well as the notation ^FS1 ^FS2 to indicate a known issue
> on all filesystems expect for FS1 FS2.
>
> Setting the variable SKIP_KNOWN_ISSUES=yes, will cause all tests
> annotated as known issues for the tested fs to be skipped.
>
> A future improvement may provide a run option to skip tests based on
> _known_issues_before_kernel when running the tests on an older kernel
> version.
>
> Signed-off-by: Amir Goldstein <amir73il@xxxxxxxxx>
> ---
> README | 2 ++
> common/rc | 16 +++++++++++++++-
> tests/overlay/061 | 2 ++
> 3 files changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/README b/README
> index 7da66cb6..8abc3840 100644
> --- a/README
> +++ b/README
> @@ -241,6 +241,8 @@ Misc:
> this option is supported for all filesystems currently only -overlay is
> expected to run without issues. For other filesystems additional patches
> and fixes to the test suite might be needed.
> + - set SKIP_KNOWN_ISSUES=yes to skip tests for bug without a known fix.
> + Those tests are annotated with _known_issue_on_fs helper.
>
> ______________________
> USING THE FSQA SUITE
> diff --git a/common/rc b/common/rc
> index 2e9dc408..3cf60a7e 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1597,8 +1597,23 @@ _supported_fs()
> _notrun "not suitable for this filesystem type: $FSTYP"
> }
>
> +_known_issue_on_fs()
> +{
> + # Only "supported" fs have the known issue
> + _check_supported_fs $* || return
> +
> + if [ "$SKIP_KNOWN_ISSUES" = "yes" ]; then
> + _notrun "known issue for this filesystem type: $FSTYP"
> + fi
> +
> + echo "HINT: You _MAY_ be hit by a known issue for filesystem type $FSTYP." >> $seqres.hints
> + echo >> $seqres.hints
> +}
> +
> _known_issue_before_kernel()
> {
> + # TODO: don't run if $SKIP_KNOWN_ISSUES and kernel version < $1
> +
As I said in last patch review, it's only for one kernel base line, it might cause
downstream kernel testing can't run a case, even if it had backported the patches.
> echo "HINT: You _MAY_ be hit by a known issue on kernel version < $1." >> $seqres.hints
> echo >> $seqres.hints
> }
> @@ -4929,7 +4944,6 @@ _require_kernel_config()
> _has_kernel_config $1 || _notrun "Installed kernel not built with $1"
> }
>
> -
> init_rc
>
> ################################################################################
> diff --git a/tests/overlay/061 b/tests/overlay/061
> index b80cf5a0..36be3391 100755
> --- a/tests/overlay/061
> +++ b/tests/overlay/061
> @@ -22,6 +22,8 @@ _begin_fstest posix copyup
>
> # real QA test starts here
> _supported_fs overlay
> +_known_issue_on_fs overlay
If it's a overlay case, it's absolutely cover an overlay known issue at first.
Even if it hit a mm or underlying fs bug, the _known_issue_on_fs is helpless
for that.
I can understand above 2 patches, but this patch looks weird to me. We can
give a hint to downstream testing (from upstream commit side). But deal with
known issues, that's another story. I'd like to let testers from different
downstream kernels to deal with their known issues checking by themselves,
don't depend on upstream fstests for all.
Thanks,
Zorro
> +
> _require_scratch
> _require_xfs_io_command "open"
>
> --
> 2.35.1
>
