The nice thing with mdadm is that the card version ( 7010, 7050, 7500, etc) matters little as we are using the card in JBOD mode. WE are not using the processor, per se.Hello Maurice,
Saturday, June 14, 2003, 7:57:23 PM, you wrote:
>>I am experiencing a poor sequental write performance with 3-disk RAID5
>>array on 3Ware 7500-8 - it is only 11-12 MB/s. Under Win2000 i get 30-35 MB/s
>>from the same array.
>>
>>Has anybody managed to get good (> 30 MB/s) raid 5 write performance
>>from 3ware 7500 on Linux and desktop (non-server) hardware - 32-bit
>>33MHz PCI ?
MH> Anyway, if you want a better solution, grab mdadm from Neil Brown. MH> Anyway, using that, a 3Ware card, on a 32 bit PCI slot on an NForce2 MH> motherboard with an AMD Athlon XP2500, I see for performance on a 4 disk MH> RAID5 set for sequential block writes between 33 and 40MB/sec, using MH> Seagate (33MB/sec) and WD drives (40MB/sec) Was it 3ware 7500? And what was CPU usage for these writes? And did
Instead we are using CPU in the host for the RAID5 write parity calculations.
As for CPU usage? Obviously it is higher.
OTOH, if you consider writing the same data in half the time, the net effect is that total CPU usage remains about the same as a function of percentage of CPU versus time spent on writes.
In other words the "cost" remains the same.
you try hardware raid5 on it?Sure. Works well. but:
1) Slower. and the more drives you add the worse it gets.
On the 4 drive example I discussed I got 45MB/sec. On an array of 8 drives over 2 cards I get nearly double that ( 75+MB/sec)
The main bottleneck starts to become the PCI bus performance. Perhaps one day, when 3Ware releases a card with a faster clock rate than the current 33MHz, we will see further improvements.
2) Flexibility.
On a software array setup I can do things like partition the drives, and create normal ext2, ext3, Reiser or other partitions on the drives, and then use up a designated amount for the RAID.
If you end up in a scenario where the OS is damaged or misconfigured, you are not counting on loading the 3Ware RAID elements just to boot the machine.
It makes troubleshooting and setup a lot more robust in the long run.
Things like initrd, lilo, grub, are a lot simpler in design and maintenance.
3) Swap performance. Simply put, swap on raw disks, where you set up multiple swap partitions on multiple drives, and stripe them with PRI= in /etc/fstab, will run circles in performance as compared to swap on a RAID5 array.
Of course, you could have additional drives for swap.. But why??
Again, per item 2, one can make the swap partitions on the same disks as the RAID 5, with little or no penalty.
4) Obsolete disks.
Many of us have seen this scenario:
Build a RAID5 with some disks. 2 years later a couple of disks fail.
No problem, they are under warranty.
Oh, darn, the manufacturer does not make that model any more. They send their "new and improved" model drive.
If you are lucky, the new ones are bigger, and you are OK< albeit that extra capacity is wasted.
If not so lucky, the drives are smaller, and you have to scrap and rebuild the array from scratch.
I could go on, and on, but given the above I think that the argument I make is fairly compelling.
Thanks for you suggestion but my opinion was that using software raid is a step back in comparison with true hardware raid card like 3ware, especially for raid 5 which is computationally-intensive.
If you say so. However that is not my experience. Simply put an Athlon, XEON, P4, Opteron, Itanium, etc. are a lot faster chip for doing XOR calculations for parity than the processor on the RAID cards. The only RAID cards I have seen that provide higher performance than that usually have much bigger, hairier and faster CPUs on them, and the costs usually start around $3,000 for the controller.
Also performance of soft-raid should be lower because data for all N drives (incl. parity) of array should be sent over PCI, and for hardware raid - only data for N-1 drives. Also 3ware cards have features like automatic array validation etc.
Uh-huh, If you say so.
3ware claims that their 7500 series gives up to 80 MB/s in 64-bit slot so i was expecting that i got about 40 MB/s in 32 bit slot. In practice i got this (35-40 MB/s) but on _windows_2000_! On Linux i am getting only poor 10-11 MB/s despite the fact that 3ware claims that they fully support Linux!
Again, I doubt you are actually getting 40MB/sec RAID5 WRITES in a 32 bit slot.
I believe the benchmark software you are using in Windows is giving you impressive looking results by doing things like using too small a block size for write tests.
Maybe a simple test with copying some large files, and using a stopwatch would be a better bet.
Kind of like Microsoft's claim that their Windows2003 server is a faster file server than Samba. Once we read into the test we found that they had put the file shares on a RAID0 array!
Not very useful.
I/O performance is not the issue.
I can easily hit 150MB/sec READS on RAID5 on software RAID5 on 3Ware in a 64 bit slot.
In a 32 bit slot, we get up to 120MB/sec.
That is where it hits the wall on PCI slot performance.
Because they are crap, their I/O ports are slow, the cards are 32 bit, and the drivers are not Open Source, and are a POS. And their support sucks, and so on.And why buy 3ware card for doing software raid - there are some 4-port software raid cards on the market (like Promise TX4)?
If you don't believe me sign on one of the RAID support lists, such as Red Hat's. All we hear, almost every day, are Promise woes.
And, here is your Promise driver for Red Hat 8.0
Hope you don't want to install a newer kernel, because the moment you do their driver will no longer work.
And I CERTAINLY hope you don't want to call them and discuss a Linux driver tech question..
I know, because I tried. more than once.
If you don't believe me, ask Andre Hedrick. He wrote a lot of the driver code used in their stuff at least at one point in time.
MH> I can email you the full report separately if you are interested. would appreciate it very much
Coming at you! ( in private mail)
With our best regards,
Maurice W. Hilarius Telephone: 01-780-456-9771 Hard Data Ltd. FAX: 01-780-456-9772 11060 - 166 Avenue mailto:maurice@xxxxxxxxxxxx Edmonton, AB, Canada http://www.harddata.com/ T5X 1Y3
