On Tue, Oct 12, 2010 at 06:21:42PM -0500, Stan Hoeppner wrote: > Dave Chinner put forth on 10/12/2010 5:29 AM: > > On Mon, Oct 11, 2010 at 08:27:00PM -0500, Stan Hoeppner wrote: > >> Dave Chinner put forth on 10/11/2010 5:35 PM: > >>> On Mon, Oct 11, 2010 at 03:03:28PM +0100, James Braid wrote: > >>>> On Fri, Oct 8, 2010 at 23:51, Dave Chinner <david@xxxxxxxxxxxxx> wrote: > >>>>> Sounds like fragmented free space. What is the output of: > >>>>> > >>>>> # xfs_db -r -c "freesp -s" <device> > >>>> > >>>> # xfs_db -r -c "freesp -s" /dev/sdb > >>>> from to extents blocks pct > >>>> 1 1 2298052 2298052 40.52 > >>>> 2 3 1568338 3337017 58.84 > >>>> 4 7 8432 35716 0.63 > >>>> 8 15 50 423 0.01 > >>>> total free extents 3874872 > >>>> total free blocks 5671208 > >>>> average free extent size 1.46359 > >>>> > >>>> Which seems to say there are a few tiny pieces of free space > >>>> available? The files that were failing to be written were a few > >>>> hundred bytes in size. > >>> > >>> The error has nothing to do with the size of the files, but > >>> everything to do with being able to allocate more inodes. Inode > >>> allocation requires 4 contiguous blocks (for 256 byte inodes, more > >>> for larger inodes) with alignment constraints. That means when you > >>> run out of 8 block or larger free extents, inode allocation will > >>> start failing and you'll get ENOSPC being reported. > >>> > >>>> We haven't seen any errors so far today, but xfs_fsr ran over the > >>>> weekend, so perhaps I guess it's reorganized the filesystem. > >>> > >>> Only a little. xfs_fsr will not improve fragmented free space > >>> conditions (indeed, it normally fragments free space more). The only > >>> way to reduce the fragmentation of free space is to remove a > >>> significant amount of data and inodes from the filesystem... > >> > >> Hay Dave, would a "backup/reformat/restore" help with free space > >> fragmentation in this case? > > > > Of course. But that's the last resort.... > > > >> If so, could/should the OP specify anything > >> during the mkfs.xfs reformat that may help alleviate or mitigate his > >> problem in the future? > > > > No. These problems usually appear in filesystems that have run at > > greater than 85-90% full for extended periods of time without being > > emptied at all. Once you start to free up space, it naturally > > defragments itself, but if you never free up any significant amount > > of space in the filesytesm, this cannot occur and so fragmentation > > just keeps getting worse.... > > So, given that this problem is on a production IMAP server, and the OP > likely can't just willy nilly start deleting user files, would adding > more disk (and assuming he's using LVM or somesuch) and growing the > filesystem alleviate this inode issue? As long as you are unsing inode64 then growing the filesystem will alow more inodes to be allocated. > Or would he be better off adding more disk, creating a new filesystem, > and moving half or so of his mailboxen over to the new filesystem at the > Cyrus (application) level? I've never used Cyrus, though IIRC Dovecot > can do this "split mail store" setup. Sure, that'd work, too. Fundamentally, moving data and inodes around after a grow (or new filesystem is added) is the only way to reduce existing free space fragmentation. Achieving this data movement is left as an exercise for the reader. ;) Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
