Configs are at:
http://www.robotics.net/glusterfs.vol
http://www.robotics.net/glusterfsd.vol
Testing with:
[root@cluster0 share]# dd if=/dev/zero of=/export/bar bs=1G count=8
Raw disk:
A 8589934592 bytes (8.6 GB) copied, 105.002 seconds, 81.8 MB/s
B 8589934592 bytes (8.6 GB) copied, 21.0523 seconds, 408 MB/s
Gluster:
A 8589934592 bytes (8.6 GB) copied, 116.116 seconds, 74.0 MB/s
B 8589934592 bytes (8.6 GB) copied, 47.4356 seconds, 181 MB/s
Gluster --disable-direct-io-mode
A 8589934592 bytes (8.6 GB) copied, 452.66 seconds, 19.0 MB/s
B 8589934592 bytes (8.6 GB) copied, 336.514 seconds, 25.5 MB/
A and B are both running distribute over replicate over ib-verbs, the only
difference is that A just has one 750 G disk in each box and B has 16 750G
disks in RAID6 on a 3ware 3650SE card in each.
I was happy with the just under 10% performance hit Gluster had over
direct disk on cluster A, however but a little taken by the over 55% hit
on cluster B. Since both clusters were connected over 10 Gbps Infiniband
using ib-verbs I expected to get about 360 MB/s on cluster B. I did better
then Gig E, but not much.
I also found it interesting how similar A and B were when using
--disable-direct-io-mode. Clearly, fast local disk play little in this
overhead.
<>
Nathan Stratton CTO, BlinkMind, Inc.
nathan at robotics.net nathan at blinkmind.com
http://www.robotics.net http://www.blinkmind.com
