You may also try the Sun's F5100 (flash storage array) - you may easily get 700 MB/s just with a single I/O stream (single process), so just with 2 streams you'll get your throughput.. - The array has 2TB total space and max throughput should be around 4GB/s.. Rgds, -Dimitri On 11/18/10, Greg Smith <greg@xxxxxxxxxxxxxxx> wrote: > Eric Comeau wrote: >> Ideally 1 large file, but it may have to be multiple. We find that if >> we send multiple files it just causes the disk to thrash more so we >> get better throughput by sending one large file. > > If it's really one disk, sure. The problem you're facing is that your > typical drive controller is going to top out at somewhere between 300 - > 500MB/s of sequential writes before it becomes the bottleneck. Above > somewhere between 6 and 10 drives attached to one controller on current > hardware, adding more to a RAID-0 volume only increases the ability to > handle seeks quickly. If you want to try and do this with traditional > hard drives, I'd guess you'd need 3 controllers with at least 4 > short-stroked drives attached to each to have any hope of hitting > 1.25GB/s. Once you do that, you'll run into CPU time as the next > bottleneck. At that point, you'll probably need one CPU per controller, > all writing out at once, to keep up with your target. > > The only popular hardware design that comes to mind aimed at this sort > of thing was Sun's "Thumper" design, most recently seen in the Sun Fire > X4540. That put 8 controllers with 6 disks attached to each, claiming > "demonstrated up to 2 GB/sec from disk to network". It will take a > design like that, running across multiple controllers, to get what > you're looking for on the disk side--presuming everything else keeps up. > > One of the big SSD-on-PCI-e designs mentioned here already may very well > end up being a better choice for you here though, as those aren't going > to require quite as much hardware all get wired up. > > -- > Greg Smith 2ndQuadrant US greg@xxxxxxxxxxxxxxx Baltimore, MD > PostgreSQL Training, Services and Support www.2ndQuadrant.us > "PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books > > -- Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
