On Wed, Sep 25, 2024 at 07:15:32PM +0800, Zorro Lang wrote:
> On Tue, Sep 24, 2024 at 10:45:48AM +0200, Christoph Hellwig wrote:
> > From: Dave Chinner <dchinner@xxxxxxxxxx>
> >
> > These tests create substantial file fragmentation as a result of
> > application actions that defeat post-EOF preallocation
> > optimisations. They are intended to replicate known vectors for
> > these problems, and provide a check that the fragmentation levels
> > have been controlled. The mitigations we make may not completely
> > remove fragmentation (e.g. they may demonstrate speculative delalloc
> > related extent size growth) so the checks don't assume we'll end up
> > with perfect layouts and hence check for an exceptable level of
> > fragmentation rather than none.
> >
> > Signed-off-by: Dave Chinner <dchinner@xxxxxxxxxx>
> > [move to different test number, update to current xfstest APIs]
> > Signed-off-by: Christoph Hellwig <hch@xxxxxx>
> > ---
>
> This patch looks good to me, just a few nit-picking below...
>
> > tests/xfs/1500 | 66 +++++++++++++++++++++++++++++++++++++++
> > tests/xfs/1500.out | 9 ++++++
> > tests/xfs/1501 | 68 ++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/1501.out | 9 ++++++
> > tests/xfs/1502 | 68 ++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/1502.out | 9 ++++++
> > tests/xfs/1503 | 77 ++++++++++++++++++++++++++++++++++++++++++++++
> > tests/xfs/1503.out | 33 ++++++++++++++++++++
> > 8 files changed, 339 insertions(+)
> > create mode 100755 tests/xfs/1500
> > create mode 100644 tests/xfs/1500.out
> > create mode 100755 tests/xfs/1501
> > create mode 100644 tests/xfs/1501.out
> > create mode 100755 tests/xfs/1502
> > create mode 100644 tests/xfs/1502.out
> > create mode 100755 tests/xfs/1503
> > create mode 100644 tests/xfs/1503.out
> >
> > diff --git a/tests/xfs/1500 b/tests/xfs/1500
> > new file mode 100755
> > index 000000000..de0e1df62
> > --- /dev/null
> > +++ b/tests/xfs/1500
> > @@ -0,0 +1,66 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test xfs/500
> > +#
> > +# Post-EOF preallocation defeat test for O_SYNC buffered I/O.
> > +#
> > +
> > +. ./common/preamble
> > +_begin_fstest auto quick prealloc rw
> > +
> > +. ./common/rc
> > +. ./common/filter
Above two lines are not necessary.
(same for other cases)
> > +
> > +_require_scratch
> > +
> > +_cleanup()
> > +{
> > + # try to kill all background processes
>
> I didn't see "kill" below, maybe "wait all background processes done"? Or you'd
> like to use "kill" but forgot? If you don't want to use "kill", please tell me,
> then I'll help to change the comment when I merge it.
>
> > + wait
> > + cd /
> > + rm -r -f $tmp.*
> > +}
> > +
> > +_scratch_mkfs > "$seqres.full" 2>&1
> > +_scratch_mount
> > +
> > +# Write multiple files in parallel using synchronous buffered writes. Aim is to
> > +# interleave allocations to fragment the files. Synchronous writes defeat the
> > +# open/write/close heuristics in xfs_file_release() that prevent EOF block
> > +# removal, so this should fragment badly. Typical problematic behaviour shows
> > +# per-file extent counts of >900 (almost worse case) whilst fixed behaviour
> > +# typically shows extent counts in the low 20s.
> > +#
> > +# Failure is determined by golden output mismatch from _within_tolerance().
> > +
> > +workfile=$SCRATCH_MNT/file
> > +nfiles=8
> > +wsize=4096
> > +wcnt=1000
> > +
> > +write_sync_file()
> > +{
> > + idx=$1
> > +
> > + for ((cnt=0; cnt<$wcnt; cnt++)); do
> > + $XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
> > + done
> > +}
> > +
> > +rm -f $workfile*
>
> Hmm, "rm -f $XXX*", but looks like the $workdfile doesn't have chance to be
> null :) Maybe rm -f $workfile.* is safer, as all test files are $workfile.$idx
> or $workfile.$n. I can do this change when I merge it.
>
> Thanks,
> Zorro
>
> > +for ((n=0; n<$nfiles; n++)); do
> > + write_sync_file $n > /dev/null 2>&1 &
> > +done
> > +wait
> > +sync
> > +
> > +for ((n=0; n<$nfiles; n++)); do
> > + count=$(_count_extents $workfile.$n)
> > + # Acceptible extent count range is 1-40
> > + _within_tolerance "file.$n extent count" $count 21 19 -v
> > +done
> > +
> > +status=0
> > +exit
> > diff --git a/tests/xfs/1500.out b/tests/xfs/1500.out
> > new file mode 100644
> > index 000000000..414df87ed
> > --- /dev/null
> > +++ b/tests/xfs/1500.out
> > @@ -0,0 +1,9 @@
> > +QA output created by 1500
> > +file.0 extent count is in range
> > +file.1 extent count is in range
> > +file.2 extent count is in range
> > +file.3 extent count is in range
> > +file.4 extent count is in range
> > +file.5 extent count is in range
> > +file.6 extent count is in range
> > +file.7 extent count is in range
> > diff --git a/tests/xfs/1501 b/tests/xfs/1501
> > new file mode 100755
> > index 000000000..cf3cbf8b5
> > --- /dev/null
> > +++ b/tests/xfs/1501
> > @@ -0,0 +1,68 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2019 Red Hat, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test xfs/501
> > +#
> > +# Post-EOF preallocation defeat test for buffered I/O with extent size hints.
> > +#
> > +
> > +. ./common/preamble
> > +_begin_fstest auto quick prealloc rw
> > +
> > +. ./common/rc
> > +. ./common/filter
> > +
> > +_require_scratch
> > +
> > +_cleanup()
> > +{
> > + # try to kill all background processes
> > + wait
> > + cd /
> > + rm -r -f $tmp.*
> > +}
> > +
> > +_scratch_mkfs > "$seqres.full" 2>&1
> > +_scratch_mount
> > +
> > +# Write multiple files in parallel using buffered writes with extent size hints.
> > +# Aim is to interleave allocations to fragment the files. Writes w/ extent size
> > +# hints set defeat the open/write/close heuristics in xfs_file_release() that
> > +# prevent EOF block removal, so this should fragment badly. Typical problematic
> > +# behaviour shows per-file extent counts of 1000 (worst case!) whilst
> > +# fixed behaviour should show very few extents (almost best case).
> > +#
> > +# Failure is determined by golden output mismatch from _within_tolerance().
> > +
> > +workfile=$SCRATCH_MNT/file
> > +nfiles=8
> > +wsize=4096
> > +wcnt=1000
> > +extent_size=16m
> > +
> > +write_extsz_file()
> > +{
> > + idx=$1
> > +
> > + $XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
_require_xfs_io_command "extsize"
> > + for ((cnt=0; cnt<$wcnt; cnt++)); do
> > + $XFS_IO_PROG -f -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
> > + done
> > +}
> > +
> > +rm -f $workfile*
> > +for ((n=0; n<$nfiles; n++)); do
> > + write_extsz_file $n > /dev/null 2>&1 &
> > +done
> > +wait
> > +sync
> > +
> > +for ((n=0; n<$nfiles; n++)); do
> > + count=$(_count_extents $workfile.$n)
_count_extents uses fiemap command, so maybe:
_require_xfs_io_command "fiemap"
> > + # Acceptible extent count range is 1-10
> > + _within_tolerance "file.$n extent count" $count 2 1 8 -v
> > +done
> > +
> > +status=0
> > +exit
[snap]
> > +read_file()
> > +{
> > + idx=$1
> > +
> > + for ((cnt=0; cnt<$wcnt; cnt++)); do
> > + $XFS_IO_PROG -f -r -c "pread 0 28" $workfile.$idx
> > + done
> > +}
> > +
> > +rm -f $workdir/file*
> > +for ((n=0; n<$((nfiles)); n++)); do
What's the $(( )) for?
Thanks,
Zorro
> > + write_file $n > /dev/null 2>&1 &
> > + read_file $n > /dev/null 2>&1 &
> > +done
> > +wait
> > +
> > +for ((n=0; n<$nfiles; n++)); do
> > + count=$(_count_extents $workfile.$n)
> > + # Acceptible extent count range is 1-40
> > + _within_tolerance "file.$n extent count" $count 6 5 10 -v
> > +done
> > +
> > +status=0
> > +exit
> > diff --git a/tests/xfs/1503.out b/tests/xfs/1503.out
> > new file mode 100644
> > index 000000000..1780b16df
> > --- /dev/null
> > +++ b/tests/xfs/1503.out
> > @@ -0,0 +1,33 @@
> > +QA output created by 1503
> > +file.0 extent count is in range
> > +file.1 extent count is in range
> > +file.2 extent count is in range
> > +file.3 extent count is in range
> > +file.4 extent count is in range
> > +file.5 extent count is in range
> > +file.6 extent count is in range
> > +file.7 extent count is in range
> > +file.8 extent count is in range
> > +file.9 extent count is in range
> > +file.10 extent count is in range
> > +file.11 extent count is in range
> > +file.12 extent count is in range
> > +file.13 extent count is in range
> > +file.14 extent count is in range
> > +file.15 extent count is in range
> > +file.16 extent count is in range
> > +file.17 extent count is in range
> > +file.18 extent count is in range
> > +file.19 extent count is in range
> > +file.20 extent count is in range
> > +file.21 extent count is in range
> > +file.22 extent count is in range
> > +file.23 extent count is in range
> > +file.24 extent count is in range
> > +file.25 extent count is in range
> > +file.26 extent count is in range
> > +file.27 extent count is in range
> > +file.28 extent count is in range
> > +file.29 extent count is in range
> > +file.30 extent count is in range
> > +file.31 extent count is in range
> > --
> > 2.45.2
> >
> >
