I'll post the results of dd soon. Meanwhile is it possible to check the stripe size for HW raid using some commands on linux? Yes it is PERC controller. On Tue, Apr 19, 2011 at 9:08 PM, Joe Landman <landman at scalableinformatics.com> wrote: > On 04/19/2011 10:43 PM, Mohit Anchlia wrote: >> >> It's HW RAID we are using. These are Dell C6100 server with RAID >> controller. We expect around 100MB/sec. When I run "dd" I get 20MB/sec >> and since I have 6 servers I at least expected 3 X 20 MB/sec since it >> replica of 2. We are in a subnet inside a LAB with only us doing the >> testing so network is not a issue for sure. >> >> File distribution are as follows: >> >> bytes % >> 130000 19.762% >> 70000 30.101% >> 100000 20.165% >> 230000 20.016% >> 1100000 0.447% >> 430000 5% >> 2000000 039% >> 630000 4.47% > > Hmmm ... ?90% of your io sizes seem to be between 70k and 230k bytes. This > is ok. ?What does your dd command look like? > > Are you running stripe as well as mirror in gluster (not that it matters for > performance). > > A single dd will be rate limited to the single mirror pair it is writing to > or reading from, unless you stripe. ?Not RAID0 stripe, but stripe in > gluster. > > However ... before we get there ... on the raw volume (e.g. outside of > gluster) what does > > ? ? ? ?dd if=/dev/sdX of=/dev/null bs=128k count=80k > > e.g. > > ? ? ? ?[root at jr5-lab ~]# dd if=/dev/sdg of=/dev/null bs=128k count=80k > ? ? ? ?81920+0 records in > ? ? ? ?81920+0 records out > ? ? ? ?10737418240 bytes (11 GB) copied, 14.7611 seconds, 727 MB/s > > > get you for each of the RAID0? ?This will read ~ 11GB of data from the > RAID0. ?No writing. ?You can write a file as well (would be very helpful) > > ? ? ? ?dd if=/dev/zero of=/path/to/gluster/file/system bs=128k \ > ? ? ? ? ? ? ? ?count=80k > > > e.g. > > ? ? ? ?[root at jr5-lab ~]# dd if=/dev/zero of=/data1/big.file bs=128k \ > ? ? ? ? ? ? ? ?count=80k > ? ? ? ?81920+0 records in > ? ? ? ?81920+0 records out > ? ? ? ?10737418240 bytes (11 GB) copied, 29.5108 seconds, 364 MB/s > > > I am assuming these are PERC RAID controllers. ?And assuming you configured > the RAID0 in hardware rather than software. ?What chunk/stripe size did you > use? > > > > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics, Inc. > email: landman at scalableinformatics.com > web ?: http://scalableinformatics.com > ? ? ? http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax ?: +1 866 888 3112 > cell : +1 734 612 4615 >
