Hi Stan Thanks for the time in composing your reply > I'm curious why you are convinced that you need BBWC, or even simply WC, > on an HBA used for md RAID. In the past I have used battery backed cards and where the write speed is "fsync constrained" the writeback cache makes the app performance fly at perhaps 10-100x the speed So for example postfix delivery speeds and mysql write performance are examples of applications which generate regular fsyncs. The whole app pauses for basically the seek time of the drive head and performance is bounded by seek time (assuming spinning media). If we add a writeback cache then it would appear that you take a couple of "green" 2TB drives and suddenly your desktop server acquires short term performance which matches a bunch of high end drives? (noted only in bursts, after some seconds you catch up with the drives IOPs). For my basically "small server" requirements this gives me a big boost in the feeling of interactivity for perhaps less than the price of a couple of those high end drives > I'm also curious as to why you are so > adamant about _not_ using the RAID ASIC on an HBA, given that it will > take much greater advantage of the BBWC than md RAID will. Only for a single reason: Its a small office server and I want the flexibility to move the drives to a different card (eg failed server, failed card or something else). Buying a spare card changes the dynamics quite a bit when the whole server (sans raid card) only costs £1,000 ish? You may be > interested to know: > > 1. When BBWC is enabled, all internal drive caches must be disabled. > Otherwise you eliminate the design benefit of the BBU, and may as > well not have one. Yes, I hadn't thought of that. Good point! > 2. w/md RAID on an HBA, if you have a good UPS and don't suffer > kernel panics, crashes, etc, you can disable barrier support in > your FS and you can use the drive caches. I don't buy this... Note we are discussing "long tail events" here. ie catastrophic events which occur very infrequently. At this point experience is everything and I concede limited experience, you likely have more, but I'm going to claim that these events are sufficiently rare that your experience probably still isn't sufficient to draw proper conclusions... In my limited experience hardware is pretty reliable and goes bad rarely. However, my estimate is that powercables fall out, PSUs fail and UPSs go bad at least as often as the power fails? Obviously it's application dependent, some may tolerate small dataloss in the event of powerdown, but I should think most people want a guarantee that the system is "recoverable" in the event of sudden powerdown. I think disabling barriers might not be the best way to avoid fsync delays, compared with the incremental cost of adding BBU writeback cache? (basically the same thing, but smaller chance of failure) > For a stable system with good UPS and auto shutdown configured, BBWC is > totally overrated. If the system never takes a nose dive from power > drop, and doesn't crash due to software or hardware failure, then BBWC > is a useless $200-1000 option. It depends on the application, but I claim that there is a fairly significant chance of hard unexpected powerdown even with a good UPS. You still are at risk from cables getting pulled, UPSs failing, etc I think in a properly setup datacenter (racked) environment then it's easier to control these accidents. Cables can be tied in, layers of power backup can be managed, it becomes efficient to add quality surge/lightning protection, etc. However, there is a large proportion of the market that have a few machines in an office and now it's much harder to stop the cleaner tripping over the UPS, or hiding it under boxes of paper until it melts due to overheating... > If your current reasoning for wanting write cache on the HBA is > performance, then forget about the write cache as you don't need it with > md RAID. If you want the BBWC combo for safety as your system isn't > stable or you have a crappy or no UPS, then forgo md RAID and use the > hardware RAID and BBWC combo. I want BB writeback cache purely to get the performance of effectively disabling fsync, but without the loss of protection which occurs if you do so. > One last point: If you're bargain hunting, especially if looking at > used gear on Ebay, that mindset is antithetical to proper system > integration, especially when talking about a RAID card BBU. I think there are few businesses who actually don't care about budget. Everything is about optimisation of cost vs performance vs reliability. Like everything else, my question is really about the tradeoff of a small incremental spend, which in turn might generate a substantial performance increase for certain classes of application. Largely I'm thinking about performance tradeoffs for small office servers priced in the £500-3,000 kind of range (not "proper" high end storage devices) I think at that kind of level it makes sense to look for bargains, especially if you are adding servers in small quantities, eg singles or pairs. > If you buy > a use card, the first thing you muse do is chuck the BBU and order a new > one, Agreed > Buy 12: > http://www.seagate.com/ww/v/index.jsp?name=st91000640ss-constellation2-6gbs-sas-1-tb-hd&vgnextoid=ff13c5b2933d9210VgnVCM1000001a48090aRCRD&vgnextchannel=f424072516d8c010VgnVCM100000dd04090aRCRD&locale=en-US&reqPage=Support#tTabContentSpecifications Out of curiosity I check the power consumption and reliability numbers of the 3.5" "Green" drives and it's not so clear cut that the 2.5" drives outperform? Thanks for your thoughts - I think this thread has been very constructive - still very interested to hear good/bad reports of specific cards - perhaps someone might archive it into some kind of list? Cheers Ed W -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
