I have a number of labs I test my glusterfs installs on. From Infinband 40G w/switch and also some cheap $800 boxes on a gig network. None of them exhibit the poor performance i'm seeing in your post - so I'm just throwing out the differences I'm seeing with your config vs mine. Another option you might want to try is increasing the max number of threads: gluster volume set <name> performance.io-thread-count 64 On Mon, Mar 19, 2012 at 2:34 AM, Chris Webb <chris at arachsys.com> wrote: > Bryan Whitehead <driver at megahappy.net> writes: > >> I didn't see any sync's after the tar/rm commands... > > By default, ext4 flushes both metadata and data every five seconds, so a > post-benchmark sync tends to make little difference on a reasonable large test, > but for completeness: > > ?# time bash -c 'tar xfz ~/linux-3.3-rc7.tgz; sync; rm -rf linux-3.3-rc7; sync' > ?real ? ?0m23.826s > ?user ? ?0m20.681s > ?sys ? ? 0m2.392s > > vs > > ?# time bash -c 'tar xfz ~/linux-3.3-rc7.tgz; sync; rm -rf linux-3.3-rc7; sync' > > ?real ? ?4m24.067s > ?user ? ?0m24.692s > ?sys ? ? 0m7.588s > > showing very similar timings and the same effect. > >> try using xfs instead of ext4. > > I'll build the xfs tooling, add kernel support, and give this a go, but I'm > surprised you think changing the underlying filesystem would eliminate the big > gap between native and gluster performance. I could imagine it improving both > somewhat, but if anything, I'd expect a higher performance filesystem to > amplify the differences. Do you think that glusterfs does something that's > particularly expensive on ext4, much more expensive than the operations proxied > through it? > > Best wishes, > > Chris.
