With NFS server problems resolved, I'm seeing what I'd hoped for which is rsync processes being more or less CPU bound with no i/o wait. Will start doing some more in depth benchmarks and likely be back later with more questions. Thanks for your help. jbh On Wed, Jun 1, 2011 at 8:48 AM, John Hanks <john.hanks at colorado.edu> wrote: > The culprit appears to be my source NFS server, looks like some of > it's RAID sets decided they needed to rebuild at about the same time I > was doing my testing. One should never assume that because something > worked well yesterday it will work well today, will retry things with > a different source that works as expected. > > jbh > > On Wed, Jun 1, 2011 at 12:44 AM, Pavan <tcp at gluster.com> wrote: >> On Tuesday 31 May 2011 11:01 PM, John Hanks wrote: >>> Hi, >>> >>> I'm setting up gluster for the first time and have a single server >>> with two bricks set up for testing: >>> >>> [root at filer-jdn1bp1 etc]# gluster volume info >>> >>> Volume Name: projects >>> Type: Distribute >>> Status: Started >>> Number of Bricks: 2 >>> Transport-type: tcp >>> Bricks: >>> Brick1: filer-jdn1bp1:/glusterfs/0.0 >>> Brick2: filer-jdn1bp1:/glusterfs/0.1 >>> Options Reconfigured: >>> nfs.disable: on >>> >>> Mounting this volume on the server or on a separate client over a 10 >>> Gbps link is working, but I'm seeing weird performance patterns. If I >>> do this: >>> >>> time sh -c "dd if=/dev/zero of=./testfile count=32768 bs=1M; sync" >>> >>> in the glusterfs mounted directory on either client I see rates of >>> ~250 MB/s but if I try to copy files onto the volume with either rsync >>> or cpio pass-thru, the rates drop to< ?10 MB/s. The directories I am >>> attempting to copy are mostly large files (> ?10 GB) and as I watch >>> cpu, network and disk i/o stats, nothing seems to be stressed or >>> blocked, it's just slow. >> >> You cannot just look at the cpu, network, io stats and conclude that >> things are fine. You will have to look at these values in comparison >> with dthe dd case where things work according to your expectation. >> >> When you say you are "rsync'ing to the volume", *from where* do you rsync? >> >> Pavan >> >>> >>> I've been searching for documentation on how to possibly tune this or >>> track down the bottleneck but not having much luck. Most things that >>> turn up in searches seem to apply to older gluster versions and the >>> 3.2 documentation is pretty sparse with respect to tuning things. >>> >>> Any suggestions for what I am doing wrong with this setup or any >>> pointers to documentation on how to troubleshoot/tune this would be >>> appreciated, especially any "intro for dummies" type docs that could >>> fill in the big gaps in my understanding of how gluster works. I'm >>> open to an RTFM reply, just haven't found the right manual yet myself >>> and would appreciate a pointer :) >>> >>> Thanks, >>> >>> jbh >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> >> >
