On Tue, Nov 19, 2024 at 12:45:05PM +1100, Dave Chinner wrote: > On Mon, Nov 18, 2024 at 03:03:43PM -0800, Darrick J. Wong wrote: > > From: Darrick J. Wong <djwong@xxxxxxxxxx> > > > > On my test fleet, this test can run for well in excess of 20 minutes: > > > > 613 generic/251 > > 616 generic/251 > > 624 generic/251 > > 630 generic/251 > > 634 generic/251 > > 652 generic/251 > > 675 generic/251 > > 749 generic/251 > > 777 generic/251 > > 808 generic/251 > > 832 generic/251 > > 946 generic/251 > > 1082 generic/251 > > 1221 generic/251 > > 1241 generic/251 > > 1254 generic/251 > > 1305 generic/251 > > 1366 generic/251 > > 1646 generic/251 > > 1936 generic/251 > > 1952 generic/251 > > 2358 generic/251 > > 4359 generic/251 > > 5325 generic/251 > > 34046 generic/251 > > > > because it hardcodes 20 threads and 10 copies. It's not great to have a > > test that results in a significant fraction of the total test runtime. > > Fix the looping and load on this test to use LOAD and TIME_FACTOR to > > scale up its operations, along with the usual SOAK_DURATION override. > > That brings the default runtime down to less than a minute. > > > > Signed-off-by: "Darrick J. Wong" <djwong@xxxxxxxxxx> > > Question for you: Does your $here directory contain a .git subdir? > > One of the causes of long runtime for me has been that $here might > only contain 30MB of files, but the .git subdir balloons to several > hundred MB over time, resulting is really long runtimes because it's > copying GBs of data from the .git subdir. > > I have this patch in my tree: > > --- a/tests/generic/251 > +++ b/tests/generic/251 > @@ -175,9 +175,12 @@ nproc=20 > # Copy $here to the scratch fs and make coipes of the replica. The fstests > # output (and hence $seqres.full) could be in $here, so we need to snapshot > # $here before computing file checksums. > +# > +# $here/* as the files to copy so we avoid any .git directory that might be > +# much, much larger than the rest of the fstests source tree we are copying. > content=$SCRATCH_MNT/orig > mkdir -p $content > -cp -axT $here/ $content/ > +cp -ax $here/* $content/ > > mkdir -p $tmp > > And that's made the runtime drop from (typically) 10-15 minutes > down to around 5 minutes.... > > Does this have any impact on the runtime on your test systems? Nope, I do vpath builds (sort of) so there's no .git history getting sucked up by generic/251. The fstests directory on the test VMs is ~34MB spread across ~4800 files. --D > -Dave. > > -- > Dave Chinner > david@xxxxxxxxxxxxx >
