On Mon, Dec 5, 2011 at 13:47, Steven Crothers <steven.crothers@xxxxxxxxx> wrote: > I have a quick question about the way to layout disks in the OSDs. I > can't really find the information I'm looking for on the Wiki or in > the existing mailing list archives that I have saved (4-5months > worth). > > In the even a large OSD is being build (say 20 drives, same make/model > 2TB). How would you build the OSD? Is it best to format each disk > individually or build them into a raid group? Is there performance > benefits to having each disk setup individually or perhaps having X > amount of raid groups on the system? You are entering a zone where answers depend on a lot of variables. There is no single "do it this way" answer, it's a series of tradeoffs. Which ones make most sense to you depend on hardware features such as RAM size, expected usage patterns, risk tolerance, performance requirements, etc. To simplify, it's a tradeoff between these two: 1. Each OSD is a failure containment mechanism. If you only run 1 OSD per server, that server will always fail as a whole, taking out a whole lot of data at once, and causing a lot of repair traffic. So more OSDs is better. 2. Lots of OSDs on a single server can mean they all spike in CPU&RAM consumption at once, e.g. after a restart of the whole server. 20 OSDs could consume quite of a lot of RAM at that point. You're probably best off with something in between those two. Group a handful of disks together, have multiple of these groups, run an OSD per each group. Whether you use RAID or btrfs pools is largely a question of reliability etc. The only possible full answer is this: Follow the requirements you have for reliability, then benchmark the alternatives. We're currently hiring sales engineers etc to help people build clusters, and we're hiring people to run benchmarks of various configurations so we can provide more concrete recommendations. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
