On Tue, Jul 3, 2012 at 9:53 AM, Christoph Hellwig <hch@xxxxxxxxxxxxx> wrote: > On Tue, Jul 03, 2012 at 09:44:38AM -0700, Tommi Virtanen wrote: >> We've seen similar issues with btrfs, and others have reported that >> the large metadata btrfs option helps. We're still compiling >> information, but as of right now I hear best performance tends to >> happen with xfs; however, the lead position tends to shift around a >> lot. > > Btw, does anyone know which part of the btrfs metadata is hit hard? > It's been a while that I looked at the OSD code, but IIRC it didn't > create too big directories, does it? For heavy directory operations > XFS filesystems created using large directorit blocks (mkfs.xfs -n > size=64k) will also provide additional benefits. I could be misremembering, but I believe the fragmentation had more to do with our async snapshots and frequent updates driving the allocator crazy than with directory sizes (which are limited to 255 entries or something by default). My guess is our xattrs have more of an impact on size, and the large metadata means everything gets copied together instead of in pieces. But Sage or Sam might want to correct me. > Also IIRC the OSDs have a directory per VDI image I think you misunderstood when you looked into that. There's a directory per placement group, but those are in no way tied to the RBD images. > - for that kind of > usage pattern the -o filestreams mount option of XFS should provide > even more performance advatages. Either way make sure to mount with > -o inode64, and for not so recent kernels -o delaylog. I'm not familiar enough with xfs' allocation mechanisms to know if filestreams is useful given the above correction. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
