On 2/3/2012 5:54 AM, Brian Candler wrote: > and the drives are Hitachi Deskstar 5K3000 HDS5C3030ALA630: 3TB, 32MB cache, 5940 RPM > The files have an average size of 0.65MB You stated you're writing 100,000 of these files across 1,000 directories, with Bonnie, then reading them back with dd in your custom script. You state this is similar to your production workload. You've hit the peak read rate of these Hitachi drives. As others pointed out, if you need more read performance than the dozen of these you plan to RAID stripe, then you'll need to swap them for units with a faster spindle: 7.2k 1.21x 10k 1.68x 15k 2.53x or with SSDs, which will yield an order of magnitude increase. Your stated need is 20M 500-800KB files, or 20GB if my math is correct. Four of these enterprise Intel SLC SSDs in a layered mdRAID0 over mdRAID1 will give you ~375 file reads/sec at 800KB per file, again if my math is correct: http://www.newegg.com/Product/Product.aspx?Item=N82E16820167062 for $480 USD total investment. You might get by with a mirrored pair depending on your actual space needs, but performance would be half. You're probably wondering why I didn't recommend an mdRAID10 instead, or a 3 SSD RAID5. All of the mdRAID striped RAID codes serialize on a single master thread, except for RAID0 and the linear concatenation (--linear). With storage devices capable of 35K IOPS each, that single thread, even running on a 3+GHz core, has trouble keeping up. The LSI 9201-16 you have is based on the SAS2116 chip, which isn't fast enough in RAID10 mode to keep up with the SSDs. In straight HBA mode it is. -- Stan _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
