Thanks Justin, good to hear about some real world experience. ----- Original Message ---- From: Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx> To: Peter <thenephilim13@xxxxxxxxx> Cc: linux-raid@xxxxxxxxxxxxxxx Sent: Monday, October 22, 2007 9:58:16 AM Subject: Re: slow raid5 performance With SW RAID 5 on the PCI bus you are not going to see faster than 38-42 MiB/s. Especially with only three drives it may be slower than that. Forget / stop using the PCI bus and expect high transfer rates. For writes = 38-42 MiB/s sw raid5. For reads = you will get close to 120-122 MiB/s sw raid5. This is from a lot of testing going up to 400GB x 10 drives using PCI cards on a regular PCI bus. Then I went PCI-e and used faster disks to get 0.5gigabytes/sec SW raid5. Justin. On Mon, 22 Oct 2007, Peter wrote: > Does anyone have any insights here? How do I interpret the seemingly competing system & iowait numbers... is my system both CPU and PCI bus bound? > > ----- Original Message ---- > From: nefilim > To: linux-raid@xxxxxxxxxxxxxxx > Sent: Thursday, October 18, 2007 4:45:20 PM > Subject: slow raid5 performance > > > > Hi > > Pretty new to software raid, I have the following setup in a file > server: > > /dev/md0: > Version : 00.90.03 > Creation Time : Wed Oct 10 11:05:46 2007 > Raid Level : raid5 > Array Size : 976767872 (931.52 GiB 1000.21 GB) > Used Dev Size : 488383936 (465.76 GiB 500.11 GB) > Raid Devices : 3 > Total Devices : 3 > Preferred Minor : 0 > Persistence : Superblock is persistent > > Update Time : Thu Oct 18 15:02:16 2007 > State : active > Active Devices : 3 > Working Devices : 3 > Failed Devices : 0 > Spare Devices : 0 > > Layout : left-symmetric > Chunk Size : 64K > > UUID : 9dcbd480:c5ca0550:ca45cdab:f7c9f29d > Events : 0.9 > > Number Major Minor RaidDevice State > 0 8 33 0 active sync /dev/sdc1 > 1 8 49 1 active sync /dev/sdd1 > 2 8 65 2 active sync /dev/sde1 > > 3 x 500GB WD RE2 hard drives > AMD Athlon XP 2400 (2.0Ghz), 1GB RAM > /dev/sd[ab] are connected to Sil 3112 controller on PCI bus > /dev/sd[cde] are connected to Sil 3114 controller on PCI bus > > Transferring large media files from /dev/sdb to /dev/md0 I see the > following > with iostat: > > avg-cpu: %user %nice %system %iowait %steal %idle > 1.01 0.00 55.56 40.40 0.00 3.03 > > Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn > sda 0.00 0.00 0.00 0 0 > sdb 261.62 31.09 0.00 30 0 > sdc 148.48 0.15 16.40 0 16 > sdd 102.02 0.41 16.14 0 15 > sde 113.13 0.29 16.18 0 16 > md0 8263.64 0.00 32.28 0 31 > > which is pretty much what I see with hdparm etc. 32MB/s seems pretty > slow > for drives that can easily do 50MB/s each. Read performance is better > around > 85MB/s (although I expected somewhat higher). So it doesn't seem that > PCI > bus is limiting factor here (127MB/s theoretical throughput.. 100MB/s > real > world?) quite yet... I see a lot of time being spent in the kernel.. > and a > significant iowait time. The CPU is pretty old but where exactly is the > bottleneck? > > Any thoughts, insights or recommendations welcome! > > Cheers > Peter > -- > View this message in context: > http://www.nabble.com/slow-raid5-performance-tf4650085.html#a13284909 > Sent from the linux-raid mailing list archive at Nabble.com. > > - > To unsubscribe from this list: send the line "unsubscribe linux-raid" > in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > - > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
