On 08/13/2013 09:23 AM, Jeffrey 'jf' Lim wrote: >>> Anyway, I thought what if instead of RAID-10 I use ceph? All 6 disks will be local, so I could simply create >>> 6 local OSDs + a monitor, right? Is there anything I need to watch out for in such configuration? >> >> You can do that. Although it's nice to play with and everything, I >> wouldn't recommend doing it. It will give you more pain than pleasure. > > How so? Care to elaborate? Ceph is a complex system, built for clusters. It does some stuff in software that is otherwhise done in hardware (raid controllers). The nature of the complexity of a cluster system is a lot of overhead compared to a local raid [whatever] system, and latency of disk i/o will naturally suffer a bit. An OSD needs about 300 MB of RAM (may vary on your PGs), times 6 is a "waste" of nearly 2 GB of RAM (compared to a local RAID). Also ceph is young, and it does indeed have some bugs. RAID is old, and very mature. Although I rely on ceph on a productive cluster, too, it is way harder to maintain than a simple local raid. When a disk fails in ceph you don't have to worry about your data, which is a good thing, but you have to worry about the rebuilding (which isn't too hard, but at least you need to know SOMETHING about ceph), with (hardware) RAID you simply replace the disk, and it will be rebuilt. Others will find more reasons why this is not the best idea for a production system. Don't get me wrong, I'm a big supporter of ceph, but only for clusters, not for single systems. wogri > -jf > > > -- > He who settles on the idea of the intelligent man as a static entity > only shows himself to be a fool. > > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." > --Richard Stallman > -- DI (FH) Wolfgang Hennerbichler Software Development Unit Advanced Computing Technologies RISC Software GmbH A company of the Johannes Kepler University Linz IT-Center Softwarepark 35 4232 Hagenberg Austria Phone: +43 7236 3343 245 Fax: +43 7236 3343 250 wolfgang.hennerbichler@xxxxxxxxxxxxxxxx http://www.risc-software.at _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |