On Thu, 2006-05-18 at 08:44 +0100, Bob Gautier wrote: > On Thu, 2006-05-18 at 02:25 -0500, Jonathan E Brassow wrote: > > The system bus isn't a limiting factor is it? 64-bit PCI-X will get > > 8.5 GB/s (plenty), but 32-bit PCI 33MHz got 133MB/s. > > > > Can your disks sustain that much bandwidth? 10 striped drives might get > > better than 200MB/s if done right, I suppose. > > This is an HDS Lightning - 64GB of mirrored write cache - I doubt if any of the writes even see disk :-) > > Don't the switches run at 2 Gbits/s? 2 Gbits/s / 10 (throw in 2 bits > > for protocol) ~= 200MB/s. > > > > Thanks for the fast responses: > > The card is a 64-bit PCI-X, so I don't think the bus is the bottleneck, > and anyway the vendor specifies a maximum throughput of 200Mbyte/s per > card. > > The disk array does not appear to be the bottleneck because we get > 200Mbyte/s when we use *two* HBAs in load-balanced mode. > > The question is really about why we only see O(100Mbyte/s) with one HBA > when we can achieve O(200MByte/s) with two cards, given that one card > should be able to achieve that throughput. > We've just done _exactly_ the same test against an EVA 8000 with 8 active paths - theoretically we should be able to get 4Gb/s via two HBAs but in fact we saw max 200MB/s with ext2, dropping to 160MB/s with ext3 - this was to a fairly slow RAID5 but that is irrelevant as we have a 16GB write cache and we were only writing 4GB files with bonnie++ I'm not sure where the overhead is. The fact that we see a 20% performance drop when we switch journalling on suggests that the overhead might be in the filesystem perhaps? It might make sense to test raw writes to a device with dd and see if that gets comparable performance figures - I'll just try that myself actually. Nick -- M: +44 (0)7736 665171 Skype: nstrug http://europe.redhat.com GPG FPR: 9C6C 093C 756A 6C57 49A1 E211 BBBA F5F5 C440 5DE0
Attachment:
signature.asc
Description: This is a digitally signed message part
-- dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
