On Tue, Mar 19, 2024 at 09:06:18AM +1100, Dave Chinner wrote: > On Mon, Mar 18, 2024 at 02:48:51PM -0400, Gabriel Krisman Bertazi wrote: > > +1 for the idea of having this in fstests. Even if we > > lack the infrastructure to do anything useful with it in ./check, > > having them in fstests will improve collaboration throughout > > different fstests wrappers (kernelci, xfstests-bld, etc.) > > Except that this places the maintenance burden on fstests, in > an environment where we can do -nothing- to validate the correctness > of these lists, nor have any idea of when tests should or > shouldn't be placed in these lists. > > i.e. If your test runner needs to expunge tests for some reason, > either keep the expunge lists with the test runner, or add detection > to the test that automatically _notrun()s the test in enviroments > where it shouldn't be run.... > > I'd much prefer the improvement of _notrun detection over spreading > the expunge file mess further into fstests. THis helps remove the > technical debt (lack of proper checking in the test) rather than > kicking it down the road for someone else to have to deal with in > future. > > Centralisation of third party expunge file management is not the > answer. We should be trying to reduce our reliance on expunges and > the maintenance overhead they require, not driving that expunge file > maintaintenance overhead into fstests itself... kdevops has been using expunges since day 1 and shared them. We have one per filesystem test section and parallelize each test section. While useful for a baseline, over time I have to agree that a desirable goal is to not rely on them. But that just means your test runner can deal with crashes automatically. That is work we've been doing for kdevops and hope to get there. That does not preclude the value of a baseline for a kernel too though test section section. While I agree that it will depend on your version of fstests and userspace too, its worthwile asking if a generic kernel baseline is desirable. The answer to this really is about scaling and doing the work. An example of a baseline of known critical failures for v6.6: https://github.com/linux-kdevops/kdevops/blob/main/docs/xfs-bugs.md Is something like this useful? If so, should we collaborate on a central one? How? Luis
