On Wed, Sep 18, 2019 at 02:41:41PM -0700, Darrick J. Wong wrote:
> On Thu, Sep 12, 2019 at 10:32:22AM -0400, Brian Foster wrote:
> > The bmap block allocation code issues a sequence of retries to
> > perform an optimal allocation, gradually loosening constraints as
> > allocations fail. For example, the first attempt might begin at a
> > particular bno, with maxlen == minlen and alignment incorporated. As
> > allocations fail, the parameters fall back to different modes, drop
> > alignment requirements and reduce the minlen and total block
> > requirements.
> >
> > For large extent allocations with an args.total value that exceeds
> > the allocation length (i.e., non-delalloc), the total value tends to
> > dominate despite these fallbacks. For example, an aligned extent
> > allocation request of tens to hundreds of MB that cannot be
> > satisfied from a particular AG will not succeed after dropping
> > alignment or minlen because xfs_alloc_space_available() never
> > selects an AG that can't satisfy args.total. The retry sequence
>
> "..that can satisfy args.total"?
>
Heh. "can't" was intended there, but after reading it back it is poorly
worded as a double negative. :P Basically it's just saying that
args.total is what restricts AG selection in this corner case.
> > eventually reduces total and ultimately succeeds if a minlen extent
> > is available somewhere, but the first several retries are
> > effectively pointless in this scenario.
> >
> > Beyond simply being inefficient, another side effect of this
> > behavior is that we drop alignment requirements too aggressively.
> > Consider a 1GB fallocate on a 15GB fs with 16 AGs and 128k stripe
> > unit:
> >
> > # xfs_io -c "falloc 0 1g" /mnt/file
> > # <xfstests>/src/t_stripealign /mnt/file 32
> > /mnt/file: Start block 347176 not multiple of sunit 32
> >
> > Despite the filesystem being completely empty, the fact that the
> > allocation request cannot be satisifed from a single AG means the
> > allocation doesn't succeed until xfs_bmap_btalloc() drops total from
> > the original value based on maxlen. This occurs after we've dropped
> > minlen and alignment (unnecessarily).
> >
> > As a step towards addressing this problem, insert a new retry in the
> > bmap allocation sequence to drop minlen (from maxlen) before tossing
> > alignment. This should still result in as large of an extent as
> > possible as the block allocator prioritizes extent size in all but
> > exact allocation modes. By itself, this does not change the behavior
> > of the command above because the preallocation code still specifies
> > total based on maxlen. Instead, this facilitates preservation of
> > alignment once extra reservation is separated from the extent length
> > portion of the total block requirement.
> >
> > Signed-off-by: Brian Foster <bfoster@xxxxxxxxxx>
> > ---
> > fs/xfs/libxfs/xfs_bmap.c | 12 +++++++++---
> > 1 file changed, 9 insertions(+), 3 deletions(-)
> >
> > diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c
> > index 054b4ce30033..eaa965920a03 100644
> > --- a/fs/xfs/libxfs/xfs_bmap.c
> > +++ b/fs/xfs/libxfs/xfs_bmap.c
> > @@ -3573,6 +3573,14 @@ xfs_bmap_btalloc(
> > if ((error = xfs_alloc_vextent(&args)))
> > return error;
> > }
> > + if (args.fsbno == NULLFSBLOCK && nullfb &&
> > + args.minlen > ap->minlen) {
>
> Maybe a comment here to point out that we're retrying the allocation
> with the minimum acceptable minlen? I say this mostly because (some of)
> the other clauses have a quick description of the constraints that are
> being fed to the allocation request, and it makes it easier to keep
> track of what's going on.
>
Yeah..
> > + args.minlen = ap->minlen;
> > + args.fsbno = ap->blkno;
> > + error = xfs_alloc_vextent(&args);
> > + if (error)
> > + return error;
> > + }
> > if (isaligned && args.fsbno == NULLFSBLOCK) {
> > /*
> > * allocation failed, so turn off alignment and
> > @@ -3584,9 +3592,7 @@ xfs_bmap_btalloc(
> > if ((error = xfs_alloc_vextent(&args)))
> > return error;
> > }
> > - if (args.fsbno == NULLFSBLOCK && nullfb &&
> > - args.minlen > ap->minlen) {
> > - args.minlen = ap->minlen;
> > + if (args.fsbno == NULLFSBLOCK && nullfb) {
> > args.type = XFS_ALLOCTYPE_START_BNO;
>
> Particularly when we get here and I have to look pretty closely to
> figure out what this retry is now attempting to do. This one is trying
> the allocation again, but now with no alignment and the caller's minlen,
> right?
>
Yeah, but this branch also (potentially) changes the allocation type.
IIRC, this wasn't relevant to the corner case I was trying to address
with this patch. I basically just wanted to get minlen dropped before
tossing alignment and at the same time didn't want to screw around with
the existing logic that was unrelated (hence the separation of the
minlen update into a new retry as opposed to just reordering code).
That said, the other subthread suggests this patch can be replaced with
more localized fixes to bma.minlen alignment handling code. My original
reproducer of this problem was based on some of the extent allocation
rework bits that have been deferred (rework of the size allocation
mode). I wasn't aware of the bma.minlen alignment logic at the time, so
I may have misanalyzed the problem and my current development tree
doesn't reproduce. I'll make the tweaks to this patch locally in the
event that I run into the problem again when I get back to that work and
if there still happens to be corner cases not covered by the minlen
fixup patch that Carlos has posted..
Brian
> --D
>
> > args.fsbno = ap->blkno;
> > if ((error = xfs_alloc_vextent(&args)))
> > --
> > 2.20.1
> >
