On Thu, May 19, 2022 at 07:24:50PM +0800, Zorro Lang wrote: > Yes, we talked about this, but if I don't rememeber wrong, I recommended each > downstream testers maintain their own "testing data/config", likes exclude > list, failed ratio, known failures etc. I think they're not suitable to be > fixed in the mainline fstests. This assumes a certain level of expertise, which is a barrier to entry. For someone who wants to check "Did my patch to filesystem Y that I have never touched before break anything?", having non-deterministic tests run by default is bad. As an example, run xfstests against jfs. Hundreds of failures, including some very scary-looking assertion failures from the page allocator. They're (mostly) harmless in fact, just being a memory leak, but it makes xfstests useless for this scenario. Even for well-maintained filesystems like xfs which is regularly tested, I expect generic/270 and a few others to fail. They just do, and they're not an indication that *I* broke anything. By all means, we want to keep tests around which have failures, but they need to be restricted to people who have a level of expertise and interest in fixing long-standing problems, not people who are looking for regressions.
