In a perfect worlds, tests are expected to pass, but we do not live in a perfect world. In reality, fstests are often merged before a fix is merged to Linus' tree or before a fix to all fs is available [1][2]. We all know that day when we pull fstests upstream branch and get a bunch of new test failures. This series [3] adds several new ways to annotate tests to provide some hints for testers about which tests are expected to pass on which fs and or which kernel versions: _fixed_by_kernel_commit _known_issue_before_kernel _known_issue_on_fs And a new run option SKIP_KNOWN_ISSUES=yes. These annotation and run option are inspired by a similar feature in LTP. These annotation could be used by testers to auto-generate expunge list for testing stable kernels from the information in _fixed_by_kernel_commit and _known_issue_before_kernel. This trivial task is left is an excercise. Thanks, Amir. [1] https://lore.kernel.org/fstests/20220412172853.GG16799@magnolia/ [2] https://lore.kernel.org/fstests/20220414155007.GC17014@magnolia/ [3] https://github.com/amir73il/xfstests/commits/hints Amir Goldstein (3): common: support black listing fs in _supported_fs() common: print hints for reasons of test failures common: add run option to skip tests for known issues README | 2 ++ check | 10 ++++++++- common/preamble | 2 +- common/rc | 54 ++++++++++++++++++++++++++++++++++++++++------- tests/generic/500 | 3 +-- tests/generic/631 | 3 +-- tests/generic/679 | 3 +-- tests/overlay/009 | 3 +++ tests/overlay/010 | 3 +++ tests/overlay/014 | 2 ++ tests/overlay/016 | 4 +++- tests/overlay/017 | 4 +++- tests/overlay/018 | 4 +++- tests/overlay/020 | 3 +++ tests/overlay/022 | 2 ++ tests/overlay/029 | 2 ++ tests/overlay/038 | 2 ++ tests/overlay/041 | 2 ++ tests/overlay/042 | 3 +++ tests/overlay/043 | 4 +++- tests/overlay/044 | 4 +++- tests/overlay/054 | 3 +++ tests/overlay/055 | 3 +++ tests/overlay/061 | 2 ++ tests/overlay/063 | 3 +++ tests/overlay/065 | 4 ++++ tests/overlay/067 | 3 +++ tests/overlay/070 | 4 +++- tests/overlay/071 | 4 +++- tests/overlay/072 | 2 ++ tests/overlay/074 | 5 +++++ tests/overlay/077 | 5 +++++ tests/overlay/078 | 1 + 33 files changed, 135 insertions(+), 23 deletions(-) -- 2.35.1
