On Sun, 15 Feb 2015, Gregory Farnum wrote: > On Sun, Feb 15, 2015 at 5:39 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > On Sun, 15 Feb 2015, Mykola Golub wrote: > >> On Thu, Feb 05, 2015 at 08:33:39AM -0700, Ron Allred wrote: > >> > Hello, > >> > > >> > The latest ceph-osd in Firefly v0.80.8, no longer auto creates its osd.X > >> > entry, in the osd map, which it was assigned via ceph.conf. > >> > > >> > I am very aware documentation states "ceph osd create", can do this job, > >> > but this command only assigns the next sequential osd.X number. This is > >> > highly undesirable. For _years_, we have assigned number ranges to each > >> > OSD server for an organized multi-tier (SSD / SAS / SATA) crush map. > >> > (leaving gaps in osd numbering, naturally.) Skipping 'ceph osd create' > >> > entirely. > >> > > >> > We are now facing a problem that an OSD remove+replace, now can't use > >> > it's former osd.X ID. Making a huge mess of documentation, number > >> > patterning, and disk labeling. > >> > > >> > Is there a work-around to forcefully create an osd.X number?? > >> > >> The "ceph osd create" could be extended to have OSD ID as a second > >> optional argument (the first is already used for uuid). > >> > >> ceph osd create <uuid> <id> > >> > >> The command would succeed only if the ID were not in use. > >> > >> Ron, would this work for you? > >> > >> I have a patch as a proof of concept: > >> > >> https://github.com/trociny/ceph/compare/wip-osd_create > > > > This looks reasonable to me! > > > > Do you mind adding a few test cases in qa/workunits/cephtool/test.sh to go > > along with it? > > > > Usual disclaimer: we discourage getting creative with the osd ids because > > they are allocated as an *array* in memory, so skipping entries consumes > > some extra memory.. this can become significant if there are large > > gaps and/or clusters are large. > > These options used to exist and were removed quite deliberately. I > don't remember the entire conversation at this point but we'll need to > find and address the concerns raised then before reintroducing the > ability to explicitly set OSD IDs. IIRC I was on the losing end of > this, because it's definitely behavior we should be offering to > admins, but the issues were significant enough we had to eliminate the > option. Methods of preserving the user-facing utility like adding OSD > names were deemed too difficult to implement. :( > > (I think it largely had to do with serious issues over the > availability and location of data when OSDs disappear, but new ones > with the same ID are present. And what you do when somebody then > resurrects the original OSDs. But there might have been other things > too.) The part I remember was just that 'ceph osd create <id>' wasn't a use and idempotent command. I don't think reusing ids is the problem, though if it is then it is still a problem since osd create will re-use the first available id. I think all this option lets us do that we didn't before is leave gaps in the id space? sage _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |