On Wed, May 25, 2022 at 02:23:10PM -0700, Darrick J. Wong wrote: > Hi everyone, > > 3. fstesting -- new patches proposed for stable branches shouldn't > introduce new regressions, and ideally there would also be a regression > test that would now pass. As Dave and I have stated in the past, > fstests is a big umbrella of a test suite, which implies that A/B > testing is the way to go. I think at least Zorro and I would like to > improve the tagging in fstests to make it more obvious which tests > contain enough randomness that they cannot be expected to behave 100% > reliably. It would be nice to find an agreement on testing requirements. I have attached some ideas on configs/number of tests/etc as well as the status of my work on 5.15 below. > a> I've been following the recent fstests threads, and it seems to me > that there are really two classes of users -- sustaining people who want > fstests to run reliably so they can tell if their backports have broken > anything; and developers, who want the randomness to try to poke into > dusty corners of the filesystem. Can we make it easier to associate > random bits of data (reliability rates, etc.) with a given fstests > configuration? And create a test group^Wtag for the tests that rely on > RNGs to shake things up? This would be great! > > > Thoughts? Flames? > > --D This thread had good timing :) I have been working on setting up some automated testing. Currently, 5.15.y is our priority so I have started working on this branch. Patches are being selected by simply searching for the “Fixes” tag and applying if the commit-to-be-fixed is in the stable branch, but AUTOSEL would be nice, so I’ll start playing around with that. Amir, it would be nice to sync up the patch selection process. I can help share the load, especially for 5.15. Selecting just the tagged “Fixes” for 5.15.y for patches through 5.17.2, 15 patches were found and applied - if there are no complaints about the testing setup, I can go ahead and send out this batch: c30a0cbd07ec xfs: use kmem_cache_free() for kmem_cache objects 5ca5916b6bc9 xfs: punch out data fork delalloc blocks on COW writeback failure a1de97fe296c xfs: Fix the free logic of state in xfs_attr_node_hasname 1090427bf18f xfs: remove xfs_inew_wait 089558bc7ba7 xfs: remove all COW fork extents when remounting readonly 7993f1a431bc xfs: only run COW extent recovery when there are no live extents 09654ed8a18c xfs: check sb_meta_uuid for dabuf buffer recovery f8d92a66e810 xfs: prevent UAF in xfs_log_item_in_current_chkpt b97cca3ba909 xfs: only bother with sync_filesystem during readonly remount eba0549bc7d1 xfs: don't generate selinux audit messages for capability testing e014f37db1a2 xfs: use setattr_copy to set vfs inode attributes 70447e0ad978 xfs: async CIL flushes need pending pushes to be made stable c8c568259772 xfs: don't include bnobt blocks when reserving free block pool cd6f79d1fb32 xfs: run callbacks before waking waiters in xlog_state_shutdown_callbacks 919edbadebe1 xfs: drop async cache flushes from CIL commits. Tests are being run through gce-xfstests with the 5.15 kernel config from xfstests-bld (https://github.com/tytso/xfstests-bld/blob/master/kernel-configs/x86_ 64-config-5.15). The configs being tested are the following: xfs defaults quota quota 1k v4 pmem and fsdax realtime 8k directory blocks external log realtime and external log devices realtime with 28k extents, external log devices overlayfs atop xfs overlayfs atop ext4 ext4 defaults The test set will be run for each batch of backports, running each test 3 times, and if no new failures are seen compared to the same branch without the backports, the batch of patches will be deemed good. No regressions were seen for the first set of patches listed above when applied to 5.15.33. If new failures are seen during testing, a bisect can be run to find the offending commits, remove these from the batch, and confirm there are no remaining new failures. A bug report can be sent to indicate which commits would cause new test failures. The test results can be posted publicly after each run. The easiest option would be to send the test results to a mailing list, such as a google groups mailing list, similar to what syzkaller does, or directly to linux-xfs if it isn’t too spammy. - Leah
