Hi,
lxmhs45:~ # xfs_info /dev/sdb
meta-data=/dev/sdb isize=256 agcount=4, agsize=655360 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=2621440, imaxpct=50
^^^^^^^^^^
And there lies the reason you are getting the filesystem into this
situation - you're allowing a very large number of inodes to be created
in the filesystem.
Ah, sorry, I changed that to 50% _after_ the first fuckup due to a
suggestion on the postfix ml, it used to be the default 25% before.
I'd suggest that for your workload, you need to allow at least 10GB
of disk space per million inodes. Because of the number of small
files, XFS is going to need a much larger amount of free space
available to prevent aging related freespace fragmentation problems.
The above ratio results in a maximum space usage of about 50%, which
will avoid such issues. If you need to hold 2 million files, use a
20GB filesystem...
I don't need to hold 2 million files, 1 million might be enough, I have
to make sure I cannot run out of inodes way before I run out of free space.
Generally speaking I have the following problem:
External nodes are submitting data (mails) to this system as fast as
they can. The mails can be between 800 bytes and several megabytes.
There are 50 receiver that write those mails as single files flat in a
single directory.
There are 4 worker threads that process a _random_ file out of this
directory. To process it they need to be able to create a temporary file
on the same filesystem. Together they are slower than the 50 receivers
(they can process maybe 20% of the incoming rate), which means that this
incoming directory is going to fill. For the sake of the argument lets
assume that the amount of mails to be sent is unlimited.
The only knob the software knows to prevent this from going over is free
disk space. When free disk space is lower than 2 Gigabyte, the
acceptance of new mails is blocked gracefully until there is free space
again.
It has, however, no way to deal with ENOSPC before that. When it cannot
create new files due no free inodes (ext4 with default settings) or
fragmentation in XFS, it breaks quite horribly and cannot recover by itself.
Can I avoid XFS giving ENOSPC due to inode shortage even in worst case
situations? I would be fine preallocating 1 GB for inode storage if that
would fix the problem. ext4 with bytes-per-inode = blocksize does this fine.
You mentioned an aging problem with XFS. I guess you mean that an XFS
filesystem will get slower/more fragmented by time with abuse like this.
These mail submission above will happen in bursts, during normal times
it will go down to << 1000 files on the entire filesystem (empty
incoming directory). Is this enough for XFS to "fix itself"?
BTW, the software can hash the incoming directory in 16 or 16x16
subdirectories. Would that help XFS in any way with those filesizes? At
first glance I would have said yes, but due to the random access in
those directories it would still have the entire spool as workload.
Bernhard
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
