On 02/04/2024 22:26, Dave Chinner wrote:
And then this gives:
args->minlen=48 blen=64
Which is perfectly reasonable - it fits the bounds specified just
fine, and we'll get a 64 block allocation if that free space is
exactly aligned. Otherwise we'll get a 48 block allocation.
But xfs_alloc_vextent_start_ag() -> xfs_alloc_vextent_iterate_ags() does not
seem to find something suitable.
Entirely possible. The AGFL might have needed refilling, so there
really wasn't enough blocks remaining for an aligned allocation to
take place after doing that. That's a real ENOSPC condition, despite
the best length sampling indicating that the allocation could be
done.
Remember, bestlen sampling is racy - it does not hold the AGF
locked from the point of sampling to the point of allocation.
ok, I assumed that some lock was held.
Hence
when we finally go to do the allocation after setting it all up, we
might have raced with another allocation that took the free space
sampled during the bestlen pass and so then the allocation fails
despite the setup saying it should succeed.
My test is single threaded, so I did not think that would be an issue.
Fundamentally, if the filesystem's best free space length is the
same size as the requested allocation,*failure is expected* and the
fallback attempts progressively remove restrictions (such as
alignment) to allow sub-optimal extents to be allocated down to
minlen in size. Forced alignment turns off these fallbacks, so you
are going to see hard ENOSPC errors the moment the filesystem runs
out of contiguous free space extents large enough to hold aligned
allocations.
This can happen a -long- way away from a real enospc condition -
depending on alignment constraints, you can start seeing this sort
of behaviour (IIRC my math correctly) at around 70% full. The larger
the alignment and the older the filesystem, the earlier this sort of
ENOSPC event will occur.
For this scenario, statvfs returns - as a sample - f_blocks=73216,
f_bfree=19950, f_bavail=19950
So ~27% free.
Use `xfs_spaceman -c 'freesp'` to dump the free space extent size
histogram. That will quickly tell you if there is no remaining free
extents large enough for an aligned 48 block allocation to succeed.
With an alignment of 16 blocks, this requires at least a 63 block
free space extent to succeed.
# xfs_spaceman -c 'freesp' /root/mnt2/
from to extents blocks pct
4 7 4 25 0.10
16 31 90 1440 5.77
32 63 12 400 1.60
64 127 1 64 0.26
512 1023 1 640 2.56
16384 22400 1 22390 89.71
IOWs, we should expect ENOSPC to occur long before the filesystem
reports that it is out of space when we are doing forced alignment
allocation.
For my test, once ENOSPC occurs and statvfs tells us more than 10% space
free, we exit as something seems wrong. As you say, deducing an error
for this condition may not be the proper thing to do.
I do also note that if I then manually attempt to write the same data to
that same empty file after the test exits, it succeeds. So something
racy. I also notice that the FSB range we scan in
xfs_alloc_ag_vextent_size() -> xfs_alloc_compute_aligned() ->
xfs_extent_busy_trim() returns busy=1 when ENOSPC occurs - I have not
checked that further.
Thanks,
John
