The culprit appears to be my source NFS server, looks like some of it's RAID sets decided they needed to rebuild at about the same time I was doing my testing. One should never assume that because something worked well yesterday it will work well today, will retry things with a different source that works as expected. jbh On Wed, Jun 1, 2011 at 12:44 AM, Pavan <tcp at gluster.com> wrote: > On Tuesday 31 May 2011 11:01 PM, John Hanks wrote: >> Hi, >> >> I'm setting up gluster for the first time and have a single server >> with two bricks set up for testing: >> >> [root at filer-jdn1bp1 etc]# gluster volume info >> >> Volume Name: projects >> Type: Distribute >> Status: Started >> Number of Bricks: 2 >> Transport-type: tcp >> Bricks: >> Brick1: filer-jdn1bp1:/glusterfs/0.0 >> Brick2: filer-jdn1bp1:/glusterfs/0.1 >> Options Reconfigured: >> nfs.disable: on >> >> Mounting this volume on the server or on a separate client over a 10 >> Gbps link is working, but I'm seeing weird performance patterns. If I >> do this: >> >> time sh -c "dd if=/dev/zero of=./testfile count=32768 bs=1M; sync" >> >> in the glusterfs mounted directory on either client I see rates of >> ~250 MB/s but if I try to copy files onto the volume with either rsync >> or cpio pass-thru, the rates drop to< ?10 MB/s. The directories I am >> attempting to copy are mostly large files (> ?10 GB) and as I watch >> cpu, network and disk i/o stats, nothing seems to be stressed or >> blocked, it's just slow. > > You cannot just look at the cpu, network, io stats and conclude that > things are fine. You will have to look at these values in comparison > with dthe dd case where things work according to your expectation. > > When you say you are "rsync'ing to the volume", *from where* do you rsync? > > Pavan > >> >> I've been searching for documentation on how to possibly tune this or >> track down the bottleneck but not having much luck. Most things that >> turn up in searches seem to apply to older gluster versions and the >> 3.2 documentation is pretty sparse with respect to tuning things. >> >> Any suggestions for what I am doing wrong with this setup or any >> pointers to documentation on how to troubleshoot/tune this would be >> appreciated, especially any "intro for dummies" type docs that could >> fill in the big gaps in my understanding of how gluster works. I'm >> open to an RTFM reply, just haven't found the right manual yet myself >> and would appreciate a pointer :) >> >> Thanks, >> >> jbh >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users > >
