[ Not really adding any technical meat, but just some past experience with XFS, plus Ext3 experience ] I remember running into this a long time ago when I was first playing with XFS for /tmp and /var (I was still a Linux/XFS noob at the time, not that I'm an expert today). I ran into the same case where both free block and inodes were still available (although similarly well utilized), and the median file size was around 1KiB. It was also in the case of many small files being written out in a short period. In my case, I didn't use the XFS debugger to get into the allocation of the extents (would have if I wasn't such a noob, good, discrete command to know, thanx!). Extents are outstanding for data and similar directories, ordering and placing large and small files to mitigate fragmentation. But in this case, and correct me if I'm wrong, it's really just a wasteful use for the extents approach, as the files typically fit in a single data block or two. I mean, I can still see some benefits in how inodes are allocated, but it seems small compared to the overhead. Then add in the delete aspect, being that the files are not going to be retained in the user's use case here (this is a spool, correct?), and I'm not seeing XFS make sense. The fact that the services "fell behind" does not surprise me, although that's just a subjective feel (and if anyone knows how to back that up with good tools and metrics, I'm all ears). I never got around to benchmarking it against Ext3 in such a use case, but I quickly adopted a complementary Ext3+XFS volume approach. I've used Ext3 with around 8 million files with a median size well under 4KiB (under 32GiB total). It works "well enough." I'm curious how Ext4 would do though. I think Ric Wheeler's team (at Red Hat) has done some benchmarks on 7+ figure file counts on Ext3 and Ext4. I think I remember a couple of info and related tidbits back and forth when I was doing some Ext3 (with GFS and GFS2) testing, on expectations of performance. Although can't say I've had 2 million files in a single directory, so YMMV. Then again, if it was extent overhead, it may not reach 2M to begin with. ----- Original Message ----- From: Dave Chinner <david@xxxxxxxxxxxxx> Sent: Thursday, October 6, 2011 9:37 PM On Thu, Oct 06, 2011 at 09:55:07PM +0200, Bernhard Schmidt wrote: > ... > It was bombarded with mails faster than it could send them on, > which eventually led to almost 2 million files of ~1.5kB in one > directory. Suddenly, this started to happen > lxmhs45:/var/spool/postfix-bulk/postfix-bulkinhss # touch a > touch: cannot touch `a': No space left on device > lxmhs45:/var/spool/postfix-bulk/postfix-bulkinhss # df . > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb 10475520 7471160 3004360 72% > So you have a 10GB filesystem, with about 3GB of free space. > /var/spool/postfix-bulk > lxmhs45:/var/spool/postfix-bulk/postfix-bulkinhss # df -i . > Filesystem Inodes IUsed IFree IUse% Mounted on > /dev/sdb 10485760 1742528 8743232 17% /var/spool/postfix-bulk And with 1.7 million inodes in it. That's a lot for a tiny filesystem, and not really a use case that XFS is well suited to. XFS will work, but it won't age gracefully under these conditions... As it is, your problem is most likely fragmented free space (an aging problem). Inodes are allocated in chunks of 64, so require an -aligned- contiguous 16k extent for the default 256 byte inode size. If you have no aligned contiguous 16k extents free then inode allocation will fail. Running 'xfs_db -r "-c freesp -s" /dev/sdb' will give you a histogram of free space extents in the filesystem, which will tell us if you are hitting this problem. _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
