On Wed, Apr 3, 2013 at 9:45 AM, John Nielsen <lists@xxxxxxxxxxxx> wrote: > On Apr 1, 2013, at 3:33 PM, Gregory Farnum <greg@xxxxxxxxxxx> wrote: > >>> On Mon, Apr 1, 2013 at 2:16 PM, Sam Lang <slang@xxxxxxxxxxx> wrote: >>>> On Mon, Apr 1, 2013 at 5:59 AM, Papaspyrou, Alexander >>>> <papaspyrou@xxxxxxxxxxxxxxxx> wrote: >>>> 1. So far, I understand that OSD ids have to be numeric, nothing else in >>>> there. What I couldn't find is whether they really have to start at 0 or 1, >>>> and whether I need to increase them stepwise? The background of my question >>>> is automation: It would make our life much simpler if I could use some >>>> calculated value for the OSD id (say, the node's decimal IP address and some >>>> local ID for the disk), because we could then statically assign them without >>>> having to know what other OSDs already exist. >>> >>> They need to start from 0. The max_osd value maintained by the >>> monitor bounds the number of possible osds, keeps the size of the >>> osdmap down, etc. >> >> Note that using the "ceph osd create" command gives you an ID to use >> these days, and that's where you should be deriving these from. >> There's no support at all for making up your own OSD IDs; sorry. > > NOW you tell me.. :) I have a small cluster where I am using the first digit of the OSD to indicate which server it is in and the second to indicate which disk in that server it uses. Aside from occasional spurious entries (for the missing digits) in the CRUSH map it seems to work fine. > > Can you elaborate on why this might be Bad? I'm planning and beginning to deploy a larger cluster and was going to use four-digit OSD id's (two for the server, two for the disk). What problems should I expect if I do that? > > How hard would it be to support making up one's own OSD ID's? > > I would also suggest that this be better documented on the website. The 5-minute quickstart says nothing about ID constraints and offers only 0 and 1 as examples. The "ceph-conf" page says only that "the instance ID for an OSD is always numeric". The "add-or-rm-osds" page mentions "ceph osd create" and its ability to set a "UUID" automatically, but only after several steps that require you to already know the "osd-number". The problem is that these IDs are used as indexes into arrays, and people creating their own tends to lead to large and sparse arrays which actually become expensive to handle for a variety of reasons. Divorcing the "name" and "id" has been a thing in the back of our minds for a while but has never made it all the way to something we want to work on right now. :/ Until we support that we're pretty unlikely to support making up one's own IDs — we used to do so and it got people into so much trouble that we turned it off and just rely on monitor allocation at this point. It's possible the docs are out of date on this subject but they need not to be (John :). -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |