On 07/10/2013 03:18 PM, Joe Julian wrote: > The "small file" complaint is all about latency though. There's very > little disk overhead (all inode lookups) to doing a self-heal check. "ls > -l" on a 50k file directory and nearly all the delay is from network RTT > for self-heal checks (check that with wireshark). Try it with localhost. Build a small test gluster brick, take networking out of the loop, create 50k files, and launch the self heal. RTT is part of it, but not the majority (last I checked it wasn't a significant fraction relative to other metadata bits). I did an experiment with 3.3.x a while ago with 2x ramdisks I created a set of files, looped back with losetup, built xfs fs atop them, mirrored them with glusterfs, and then set about to doing metadata/small file heavy workloads. Performance was still abysmal. Pretty sure none of that was RTT. Definitely a stack traversal problem, but I didn't trace it far enough back to be definitively sure where it was. -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/siflash phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615
