----- Original Message ----- > From: "Lars Hanke" <debian@xxxxxxxxx> > To: "Ben Turner" <bturner@xxxxxxxxxx> > Cc: gluster-users@xxxxxxxxxxx > Sent: Wednesday, February 18, 2015 5:09:19 PM > Subject: Re: Poor Gluster performance > > Am 18.02.2015 um 22:05 schrieb Ben Turner: > > ----- Original Message ----- > >> From: "Lars Hanke" <debian@xxxxxxxxx> > >> To: gluster-users@xxxxxxxxxxx > >> Sent: Wednesday, February 18, 2015 3:01:54 PM > >> Subject: Poor Gluster performance > >> > >> I set up a distributed, replicated volume consisting of just 2 bricks on > >> two physical nodes. The nodes are peered using a dedicated GB ethernet > >> and can be accessed from the clients using a separate GB ethernet NIC. > >> > >> Doing a simple dd performance test I see about 11 MB/s for read and > >> write. Running a local setup, i.e. both bricks on the same machine and > >> local mount, I saw even 500 MB/s. So network sould be the limiting > >> factor. But using NFS or CIFS on the same network I see 110 MB/s. > >> > >> Is gluster 10 times slower than NFS? > > > > Something is going on there. On my gigabit setups I see 100-120 MB / sec > > writes for pure distribute and about 45-55 MB / sec with replica 2. What > > block size are you using? I could see that if you were writing something > > like 4k or under but 64k and up you should be getting about what I said. > > Can you tell me more about your test? > > Block size is 50M: > > root@gladsheim:/# mount -t glusterfs node2:/test ~/mnt > root@gladsheim:/# dd if=/dev/zero of=~/mnt/testfile.null bs=50M count=10 > 10+0 records in > 10+0 records out > 524288000 bytes (524 MB) copied, 46.6079 s, 11.2 MB/s > root@gladsheim:/# dd if=~/mnt/testfile.null of=/dev/null bs=50M count=10 > 10+0 records in > 10+0 records out > 524288000 bytes (524 MB) copied, 45.7487 s, 11.5 MB/s This looks like the NICs may only be negotiating to 100Mb(max theoretical of 12.5 MB / sec), can you check ethtool on all of your NICs? Also I like to run iperf between servers and clients and servers and servers before I do anything with gluster, if you aren't getting ~line speed with iperf gluster wont be able to either. Double check you NICs and your backend and see if you can spot the bottleneck at either of those layers. -b > It doesn't depend on whether I use node1 or node2 for the mount. > > BTW: does the cut of the bandwidth to half in replicated mode mean that > the client writes to both nodes, i.e. doubles the network load on the > client side network? I hoped that replication would be run on the server > side network. > > Regards, > - lars. > > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
