Ray, We will get back on this. Krishna On Wed, Feb 4, 2009 at 4:58 AM, Ray Speth <speth at mit.edu> wrote: > I am wondering if anyone knows how to achieve good performance when > writing to a volume using cluter/replicate. I am getting write speeds of > about 4 MB/s over Gigabit Ethernet when transferring (single, large) > files to such a volume. When I remove replication, I get transfer rates > using "cp" of 21 MB/s. I can get much higher rates using "dd" with large > block sizes (at least 128KB) -- 34 MB/s for the replicated volume and > 117 MB/s without replication. Read speeds are over 100 MB/s in all > cases. The transfer rates for cp are comparable to those using dd with a > block size between 4KB and 8KB. > > I am using Gluster 2.0.0rc1 on Ubuntu 8.10 (Intrepid) systems, running > the stock 2.6.27 kernel. I have tried a range of things with the various > performance translators, but have not seen much change in performance. I > tried applying the FUSE patch, but it appears that the patch for FUSE > 2.7.3 modifies sections of the kernel module that have changed in the > 2.6.27 kernel. > > Does anyone know how I can improve the write performance in this type of > setup? Even the 21 MB/s that I get without replicate would be > acceptable, even if it's not really making use of all the available > bandwidth. > > My server and client configuration files are below. These include > cluster/distribute as well, but I don't think that has any effect when > copying a single file. > > Thanks, > Ray > > *** glusterfs-server.vol *** > volume brick0 > type storage/posix > option directory /scratch/gluster-brick > end-volume > > volume brick1 > type features/locks > subvolumes brick0 > end-volume > > volume brick > type performance/io-threads > option thread-count 8 > subvolumes brick1 > end-volume > > volume server > type protocol/server > option transport-type tcp > option auth.addr.brick.allow * > subvolumes brick > end-volume > > *** glusterfs-client.vol *** > > volume node01 > type protocol/client > option transport-type tcp > option remote-host node01 > option remote-subvolume brick > end-volume > > <snip: etc. for node02...node08> > > volume rep1 > type cluster/replicate > subvolumes node01 node02 > end-volume > > < snip: etc. for rep2, rep3, rep4 on the remaining nodes> > > volume distribute > type cluster/distribute > subvolumes rep1 rep2 rep3 rep4 > end-volume > > volume writebehind > type performance/write-behind > option block-size 1MB > option cache-size 4MB > option flush-behind on > subvolumes distribute > end-volume > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users >
