2009/8/1 Keld Jørn Simonsen <keld@xxxxxxxx>: > On Fri, Jul 31, 2009 at 01:10:37PM -0700, David Rees wrote: >> Let's use some data from a real disk, the Velociraptor and a 2-disk >> array and streaming reads/writes. At the beginning of the disk you >> can read about 120MB/s. At the end of the disk, you can read about >> 80MB/s. > > This is not actual figures from some benchmarking you did, true? Those are actual numbers from a Velociraptor, but the numbers are just estimates. >> Data on the "beginning" of array, RAID0 = 240MB/s >> Data on the "end" of array, RAID0 = 160MB/s. >> Data on the "beginning" of array, RAID10,n2 = 120MB/s >> Data on the "end" of array, RAID10,n2 = 80MB/s. >> Data on the "beginning" of array, RAID10,f2 = 200MB/s > > Should be: > > Data on the "beginning" of array, RAID10,f2 = 230MB/s No - you're getting 120 MB/s from one disk and 80MB/s from another. How that would add up to 230MB/s defies logic... >> With a f2 setup you'll read at something less than 120+80 = 200MB/s. > > When? at the beginning or the end? The whole thing, on average. But the whole point of f2 is to even out performance from beginning of the array and let you stripe reads. > Random read performance on a single disk, in one test, was 34 MB/s while > seq read on same disk was 82. In raid10,f2 with 2 disks random read was > 79 MB/s. This is 235 % of the random read on one disk (34 MB/s). This is > a likely result as you should expect a doubling in speed from the 2 > disks, and then som additional speed from the faster sectors of the > outer disks, and then the shorter access times on the oyuter disk > sectors. Geometry says that on average the transfer speeds are 17 % > shorter on the outer half part of the disk, compared to the whole disk. > So that gives some 235 % speed improvement (2 * 1.17). The head > improvement should also give a little, but maybe the elevator algorithm > of the file system eliminates most of that factor. Sorry - I'm having a hard time wrapping my head around that you can simply ignore access to the slow half the disk in a multi-threaded random IO test. The only way I might believe that you can get 235% improvement is in a single threaded test with a queue depth of 1 which lets the f2 setup only use the fast half the disks. If that is your assumption, then, OK. But then getting 34MB/s isn't out of a rotating disk isn't random IO, either. Random IO on a rotating disk is normally an order of magnitude slower. >> Your benchmarks showed about a 13% performance hit for f2 compared to >> n2 and RAID1, so I wouldn't quite call it the same. Close, but still >> noticeably slower. > > Nah, for random write, MB/s: > > raid1 55 > raid10,n2 48 > raid10,f2 55 Sorry - must have read them wrong again. >> >> 1. The array mostly sees write activity, streaming reads aren't that common. >> >> 2. I can only get about 120 MB/s out of the external enclosure because >> >> of the PCIe card [1] , so being able to stripe reads wouldn't help get >> >> any extra performance out of those disks. >> >> [1] http://ata.wiki.kernel.org/index.php/Hardware,_driver_status#Silicon_Image_3124 >> > >> > Hmm, a pci-e x1 should be able to get 2.5 Mbit/s = about 300 MB/s. >> > Wikipedia says 250 MB/s. It is strange that you only can get 120 MB/s. >> > That is the speed of a PCI 32 bit bus. I looked at your reference [1] >> > for the 3132 model. Have you tried it out in practice? >> >> Yes, in practice, IO reached exactly 120MB/s out of the controller. I >> ran dd read/write tests on individual disks and found that overall >> throughput peaked exactly at 120MB/s. > > Hmm, get another controller, then. A cheap PCIe contoller should be able > to do about 300 MB/s on a x1 PCIe. Please read my reference again. It's a motherboard limitation. I already _have_ a good, cheap PCIe controller. >> But the primary reason I built it was to handle lots of small random >> writes/reads, so being limited to 120MB/s out of the enclosure isn't >> noticeable most of the time in practice as you say. > > Yes, for random read/write you only get something like 45 % out of the > max transfer bandwidth. So 120 MB/s would be close to the max that your > 5 disks on the PCIe controller can deliver. With a faster PCIe > controller you should be able to get better performance on random reads > with raid10,f2. Anyway 180 MB/s may be fast enough for your application. Again - your idea of "random" IO is completely different than mine. My random IO workloads can only get a couple MB/s out of a single disk. Here's a benchmark which tests SSDs and rotational disks. All the rotational disks are getting less than 1MB/s in the random IO test. http://www.anandtech.com/storage/showdoc.aspx?i=3531&p=25 It's a worst case scenario, but not far from my workloads which obviously read a bit more data on each read. -Dave -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
