On 04/19/2011 10:43 PM, Mohit Anchlia wrote: > It's HW RAID we are using. These are Dell C6100 server with RAID > controller. We expect around 100MB/sec. When I run "dd" I get 20MB/sec > and since I have 6 servers I at least expected 3 X 20 MB/sec since it > replica of 2. We are in a subnet inside a LAB with only us doing the > testing so network is not a issue for sure. > > File distribution are as follows: > > bytes % > 130000 19.762% > 70000 30.101% > 100000 20.165% > 230000 20.016% > 1100000 0.447% > 430000 5% > 2000000 039% > 630000 4.47% Hmmm ... 90% of your io sizes seem to be between 70k and 230k bytes. This is ok. What does your dd command look like? Are you running stripe as well as mirror in gluster (not that it matters for performance). A single dd will be rate limited to the single mirror pair it is writing to or reading from, unless you stripe. Not RAID0 stripe, but stripe in gluster. However ... before we get there ... on the raw volume (e.g. outside of gluster) what does dd if=/dev/sdX of=/dev/null bs=128k count=80k e.g. [root at jr5-lab ~]# dd if=/dev/sdg of=/dev/null bs=128k count=80k 81920+0 records in 81920+0 records out 10737418240 bytes (11 GB) copied, 14.7611 seconds, 727 MB/s get you for each of the RAID0? This will read ~ 11GB of data from the RAID0. No writing. You can write a file as well (would be very helpful) dd if=/dev/zero of=/path/to/gluster/file/system bs=128k \ count=80k e.g. [root at jr5-lab ~]# dd if=/dev/zero of=/data1/big.file bs=128k \ count=80k 81920+0 records in 81920+0 records out 10737418240 bytes (11 GB) copied, 29.5108 seconds, 364 MB/s I am assuming these are PERC RAID controllers. And assuming you configured the RAID0 in hardware rather than software. What chunk/stripe size did you use? -- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: landman at scalableinformatics.com web : http://scalableinformatics.com http://scalableinformatics.com/sicluster phone: +1 734 786 8423 x121 fax : +1 866 888 3112 cell : +1 734 612 4615
