I take it back. Things deteriorated pretty quickly after I began dumping data onto my volume from multiple clients. Initially my transfer rates were okay, not fast, but livable. However after about an hour of copying several terabytes from 3-4 client machines, the rates of transfer often dropped to lb/s. Sometimes I would see a couple second burst of good transfer rates. Anyone have ideas on how to address this effectively? I'm at a loss. -Jon On Nov 2, 2012 1:21 PM, "Jonathan Lefman" <jonathan.lefman at essess.com> wrote: > I should have also said that my volume is working well now and all is well. > > -Jon > > > On Fri, Nov 2, 2012 at 1:21 PM, Jonathan Lefman < > jonathan.lefman at essess.com> wrote: > >> Thank you Brian. I'm happy to hear that this behavior is not typical. I >> am now using xfs on all of my drives. I also wiped out the entire >> /etc/glusterd directory for good measure. I bet that there was residual >> information from a previous attempt at a gluster volume that must have >> caused problems. Or moving to xfs from ext4 is an amazing fix, but I think >> this is less likely. >> >> I appreciate your time responding to me. >> >> -Jon >> On Nov 2, 2012 4:44 AM, "Brian Candler" <B.Candler at pobox.com> wrote: >> >>> On Thu, Nov 01, 2012 at 08:03:21PM -0400, Jonathan Lefman wrote: >>> > Soon after loading up about 100 MB of small files (about 300kb >>> each), >>> > the drive usage is at 1.1T. >>> >>> That is very odd. What do you get if you run du and df on the individual >>> bricks themselves? 100MB is only ~330 files of 300KB each. >>> >>> Did you specify any special options to mkfs.ext4? Maybe -l 512 would >>> help, >>> as the xattrs are more likely to sit within the indoes themselves. >>> >>> If you start everything from scratch, it would be interesting to see df >>> stats when the filesystem is empty. It may be that a huge amount of >>> space >>> has been allocated to inodes. If you expect most of your files >16KB >>> then >>> you could add -i 16384 to mkfs.ext4 to reduce the space reserved for >>> inodes. >>> But using xfs would be better, as it doesn't reserve any space for >>> inodes, >>> it allocates it dynamically. >>> >>> Ignore the comment that glusterfs is "not designed for handling large >>> count >>> small files" - 300KB is not small. >>> >>> Regards, >>> >>> Brian. >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20121105/64dd239b/attachment.html>
