On Sat, Jul 02, 2016 at 09:52:40AM +0100, Nick Fisk wrote: > Hi, hope someone can help me here. > > I'm exporting some XFS fs's to ESX via NFS with the sync option enabled. I'm > seeing really heavy fragmentation when multiple VM's are copied onto the > share at the same time. I'm also seeing kmem_alloc failures, which is > probably the biggest problem as this effectively takes everything down. (Probably a result of loading the millions of bmbt extents into memory?) > Underlying storage is a Ceph RBD, the server the FS is running on, is > running kernel 4.5.7. Mount options are currently default. I'm seeing > Millions of extents, where the ideal is listed as a couple of thousand when > running xfs_db, there is only a couple of 100 files on the FS. It looks > like roughly the extent sizes roughly match the IO size that the VM's were > written to XFS with. So it looks like each parallel IO thread is being > allocated next to each other rather than at spaced out regions of the disk. > > From what I understand, this is because each NFS write opens and closes the > file which throws off any chance that XFS will be able to use its allocation > features to stop parallel write streams from interleaving with each other. > > Is there anything I can tune to try and give each write to each file a > little bit of space, so that it at least gives readahead a chance when > reading, that it might hit at least a few MB of sequential data? /me wonders if setting an extent size hint on the rootdir before copying the files over would help here... --D > > I have read that inode32 allocates more randomly compared to inode64, so I'm > not sure if it's worth trying this as there will likely be less than a 1000 > files per FS. > > Or am I best just to run fsr after everything has been copied on? > > Thanks for any advice > Nick > _______________________________________________ > xfs mailing list > xfs@xxxxxxxxxxx > http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
