On Mon, 23 Mar 2009, Gennadiy Nerubayev wrote: > Hi all, > > I'm using 2.0.0rc4 on a pair of infiniband connected servers. Both servers > are running 2.6.28.7, and are using ext4 on a local raid set as the storage > for Gluster. The speeds that I get writing to the aforementioned ext4 mount, > depending on benchmark and method, vary around 400MB/s (with dd) to 550MB/s+ > (with iozone, bonnie, and dd directio). Next, I used the Gluster server and > client on both servers, with ib-verbs and replicate (see attached > configuration files). The best speeds that I get when writing to the Gluster > mount with this setup is about 150MB/s, going up to about 200MB/s when doing > simultaneous writes from both nodes. I tried a couple of configuration > tweaks such as writeback, but they don't seem to be having much effect. That is about what I am getting: Raw disk: 8589934592 bytes (8.6 GB) copied, 21.0523 seconds, 408 MB/s Gluster: 8589934592 bytes (8.6 GB) copied, 47.4356 seconds, 181 MB/s Gluster --disable-direct-io-mode 8589934592 bytes (8.6 GB) copied, 336.514 seconds, 25.5 MB/ The part that is most concerning to me is the --disable-direct-io-mode stuff since I am trying to use this with xen tap:aio. > Is there something I can do to bring the performance closer to the native > disk speed, or are there some inherent limitations somewhere that I should > be aware of? You can try some caching, but overall it did not help me out a lot. ><> Nathan Stratton CTO, BlinkMind, Inc. nathan at robotics.net nathan at blinkmind.com http://www.robotics.net http://www.blinkmind.com
