> -----Original Message----- > From: Mark Mielke [mailto:mark at mark.mielke.cc] > Sent: 12 August 2009 16:35 > To: Hiren Joshi > Cc: gluster-users at gluster.org > Subject: Re: Performance > > On 08/12/2009 11:24 AM, Hiren Joshi wrote: > > Hello all, > > > > I have 2 servers both exporting 36 partitions. The client > mirrors the 36 > > partitions on each server and puts them all into a DHT. > > > > dd if=/dev/zero of=/home/webspace_glust/zeros bs=1024 count=1024000 > > Takes 8 minutes, compared to 30 seconds vi NFS, now granted > I'm using > > mostly default settings but a transfer rate of 2.1Mb/s vs > about 40Mb/s > > seems low. > > > > What am I doing wrong? Where should I start with the > performance tuning? > > How about start with a 1:1 comparison? Unless is somehow doing > mirroring, you should remove the mirroring from your solution before > comparing. Fair point, I've put on a single brick (no distribution no mirror) and I get: time dd if=/dev/zero of=/home/webspace_glust/zeros bs=1024 count=1024000 1024000+0 records in 1024000+0 records out 1048576000 bytes (1.0 GB) copied, 109.289 seconds, 9.6 MB/s real 1m49.291s user 0m0.156s sys 0m2.701s Compared to: time dd if=/dev/zero of=/home/webspace_nfs/josh/zeros bs=1024 count=1024000 1024000+0 records in 1024000+0 records out 1048576000 bytes (1.0 GB) copied, 29.4801 seconds, 35.6 MB/s real 0m29.526s user 0m0.486s sys 0m1.927s For NFS. > > Are you using striping? If you are not using striping, then the file > "/home/webspace_glust" is going to be assigned to a single partition, > and you would only be using a mirror of one partition. No striping, thanks, I didn't consider this, so I tried: time for x in `seq 1 1 1000`; do dd if=/dev/zero of=/home/webspace_nfs/zeros$x bs=1024 count=1024; echo $x; done real 0m45.292s user 0m0.760s sys 0m3.209s (about 20-40MB/s) and time for x in `seq 1 1 1000`; do dd if=/dev/zero of=/home/webspace_glust/zeros$x bs=1024 count=1024; echo $x; done real 1m58.931s user 0m0.432s sys 0m6.412s (about 8-10MB/s) > > 36 partitions on each server - the word "partition" is ambiguous. Are > they 36 separate drives? Or multiple partitions on the same drive. If > multiple partitions on the same drive, this would be a bad > idea, as it > would require the disk head to move back and forth between the > partitions, significantly increasing the latency, and therefore > significantly reducing the performance. If each partition is > on its own > drive, you still won't see benefit unless you have many clients > concurrently changing many different files. In your above case, it's > touching a single file in sequence, and having a cluster is > costing you > rather than benefitting you. We went with 36 partitions (on a single raid 6 drive) incase we got file system corruption, it would take less time to fsck a 100G partition than a 3.6TB one. Would a 3.6TB single disk be better? > > As for the 2.1Mb/s vs 40Mb/s, I have no clue. I'm new to > Gluster myself, > and I have yet to install it on my node cluster (10 servers) > and perform > timing myself. > > Cheers, > mark > > -- > Mark Mielke<mark at mielke.cc> > >
