On Fri, Mar 11, 2022 at 01:09:35PM +0100, Jan Kara wrote: > On Thu 10-03-22 14:41:30, Luis Chamberlain wrote: > > On Thu, Mar 10, 2022 at 01:51:22PM -0500, Josef Bacik wrote: > > > On Wed, Mar 09, 2022 at 05:28:28PM -0800, Luis Chamberlain wrote: > > > > On Wed, Mar 09, 2022 at 04:19:21PM -0500, Josef Bacik wrote: > > > > > On Wed, Mar 09, 2022 at 11:00:49AM -0800, Luis Chamberlain wrote: > > > > > > > > That's great! > > > > > > > > But although this runs nightly, it seems this runs fstest *once* to > > > > ensure if there are no regressions. Is that right? > > > > > > > > > > Yup once per config, so 8 full fstest runs. > > > > From my experience that is not enough to capture all failures given > > lower failure rates on tests other than 1/1, like 1/42 or > > 1/300. So minimum I'd go for 500 loops of fstests per config. > > This does mean this is not possible nightly though, yes. 5 days > > on average. And so much more work is needed to bring this down > > further. > > Well, yes, 500 loops have better chance of detecting rare bugs. But if you > did only say 100 loops, you are likely to detect the bug just 5 days later > on average. Sure that makes finding the bug somewhat harder (you generally > need to investigate larger time span to find the bug) but testing costs are > lower... It is a tradeoff. Crap sorry I had my numbers mixed, yes 100 takes about 5 days (for btrfs or xfs running all confgurations in parallel), so indeed, 100 was reasonable goal today. 500 would take almost a month and if that doesn't give you much time to fix issues either if you have a kernel release per month! Luis
