On Mon, Apr 1, 2013 at 2:16 PM, Sam Lang <slang@xxxxxxxxxxx> wrote: > On Mon, Apr 1, 2013 at 5:59 AM, Papaspyrou, Alexander > <papaspyrou@xxxxxxxxxxxxxxxx> wrote: >> Folks, >> >> we are trying to setup a ceph cluster with about 40 or so OSDs on our >> hosting provider's infrastructure. Our rollout works with Opscode Chef, and >> I'm driving my people to automate away everything they can. >> >> I've worked my way through the documentation, but a few questions were left >> open: >> >> 1. So far, I understand that OSD ids have to be numeric, nothing else in >> there. What I couldn't find is whether they really have to start at 0 or 1, >> and whether I need to increase them stepwise? The background of my question >> is automation: It would make our life much simpler if I could use some >> calculated value for the OSD id (say, the node's decimal IP address and some >> local ID for the disk), because we could then statically assign them without >> having to know what other OSDs already exist. > > They need to start from 0. The max_osd value maintained by the > monitor bounds the number of possible osds, keeps the size of the > osdmap down, etc. Note that using the "ceph osd create" command gives you an ID to use these days, and that's where you should be deriving these from. There's no support at all for making up your own OSD IDs; sorry. >> 2. All examples more or less state that each participating node has an >> identical ceph.conf file. This does not scale for me, because I don't want >> to update all nodes when adding a new OSD/MON/MDS/<whatever>. From my >> understanding of ceph, this should not be necessary anyway, because >> everything metadata-related goes through the MONs anyway. So: >> a. Is it sufficient to have a minimal ceph.conf on the OSDs, basically >> describing the local OSD parameters and containing the MON data, but no >> information about other OSDs? > > Are you using the ceph chef recipes? > http://ceph.com/docs/master/rados/deployment/chef/ > > You should be able to get away with not putting all the osds in your > conf file, but this isn't recommended. With the above chef recipes, I > believe the ceph.conf is generated and updated as necessary. To be a little more clear, if you're not putting all your daemons in a conf file then you need something else keeping track of where you have daemons. If you're using Chef or a similar system this is just fine — our Crowbar recipes for instance include only the local nodes and the monitor IPs in each ceph.conf. If you're expecting to manage your whole Ceph cluster via the sysvinit scripts (not recommended for real systems!) then you need them all listed so it can tell who to talk to. If the docs are implying otherwise then that's a bit of a weakness. I've personally run example systems without a ceph.conf anywhere in the deployment. :) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |