Re: xfs_extent_busy_flush vs. aio

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 01/29/2018 11:56 PM, Dave Chinner wrote:
On Mon, Jan 29, 2018 at 01:44:14PM +0200, Avi Kivity wrote:

On 01/29/2018 01:35 PM, Dave Chinner wrote:
On Mon, Jan 29, 2018 at 11:40:27AM +0200, Avi Kivity wrote:
On 01/25/2018 03:08 PM, Brian Foster wrote:
On Thu, Jan 25, 2018 at 10:50:40AM +0200, Avi Kivity wrote:
On 01/23/2018 07:39 PM, Brian Foster wrote:
Yeah, could be.. perhaps the issue is that despite the large amount of
total free space, the free space is too fragmented to satisfy a
particular allocation request..?
      from      to extents  blocks    pct
         1       1    2702    2702   0.00
         2       3     690    1547   0.00
         4       7     115     568   0.00
         8      15      60     634   0.00
        16      31      63    1457   0.00
        32      63     102    4751   0.00
        64     127    7940  895365   0.19
       128     255   49680 12422100   2.67
       256     511    1025  417078   0.09
       512    1023    4170 3660771   0.79
      1024    2047    2168 3503054   0.75
      2048    4095    2567 7729442   1.66
      4096    8191    8688 59394413  12.76
      8192   16383     310 3100186   0.67
     16384   32767     112 2339935   0.50
     32768   65535      35 1381122   0.30
     65536  131071       8  651391   0.14
    131072  262143       2  344196   0.07
    524288 1048575       4 2909925   0.62
1048576 2097151       3 3550680   0.76
4194304 8388607      10 82497658  17.72
8388608 16777215      10 158022653  33.94
16777216 24567552       5 122778062  26.37
total free extents 80469
total free blocks 465609690
average free extent size 5786.2

Looks like plenty of free large extents, with most of the free space
completely, unfragmented.

Indeed..
You need to look at each AG, not the overall summary. You could have
a suboptimal AG hidden in amongst that (e.g. near ENOSPC) and it's
that one AG that is causing all your problems.

There's many reasons this can happen, but the most common is the
working files in a directory (or subset of directories in the same
AG) have a combined space usage of larger than an AG ....
That's certainly possible, even likely (one huge directory with all of the
files).

This layout is imposed on us by the compatibility gods. Is there a way to
tell XFS to change its policy of on-ag-per-directory?
mount with inode32. That rotors files around all AGs in a round
robin fashion instead of trying to keep directory locality for  a
working set. i.e. it distributes the files evenly across the
filesystem.

I remember you recommending it a couple of years ago, with the qualification to only do it if we see a problem. I think we qualify now.


However, this can have substantial impact on performance if the
workload requires locality of files for performance and you're
running on spinning rust.

This is not a problem. We do have a few users on spinning disks, but they're a small minority, and our installer can avoid the inode32 option if it sees a rotational device.

  OTOH, if locality is not needed then
distributing all files across all AGs evenly should give more even
capacity usage.

And, based on what was discussed on #xfs overnight, don't bother
with the filestreams allocator - it will not solve the "full AG"
problem, but it will introduce a whole bunch of new problems for
you.

That system has since been recycled, but we'll keep it in mind the next time
we see it. Also, we'll have to fix the one-huge-directory problem for sure,
one way or another.
If inode32 doesn't work for you, the easiest way is to use a small
directory hash - create top level directory with ~2x AG count child
directories and hash your files into those. This will distribute the
files roughly evenly across all AGs in the filesystem whilst still
maintaining locality within directories. It's kind of half way
between inode64 and inode32....


Yeah. Directory layout is sorta imposed on us, but if we have to break it, we will.
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux