On Sun, Feb 15, 2015 at 5:39 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Sun, 15 Feb 2015, Mykola Golub wrote: >> On Thu, Feb 05, 2015 at 08:33:39AM -0700, Ron Allred wrote: >> > Hello, >> > >> > The latest ceph-osd in Firefly v0.80.8, no longer auto creates its osd.X >> > entry, in the osd map, which it was assigned via ceph.conf. >> > >> > I am very aware documentation states "ceph osd create", can do this job, >> > but this command only assigns the next sequential osd.X number. This is >> > highly undesirable. For _years_, we have assigned number ranges to each >> > OSD server for an organized multi-tier (SSD / SAS / SATA) crush map. >> > (leaving gaps in osd numbering, naturally.) Skipping 'ceph osd create' >> > entirely. >> > >> > We are now facing a problem that an OSD remove+replace, now can't use >> > it's former osd.X ID. Making a huge mess of documentation, number >> > patterning, and disk labeling. >> > >> > Is there a work-around to forcefully create an osd.X number?? >> >> The "ceph osd create" could be extended to have OSD ID as a second >> optional argument (the first is already used for uuid). >> >> ceph osd create <uuid> <id> >> >> The command would succeed only if the ID were not in use. >> >> Ron, would this work for you? >> >> I have a patch as a proof of concept: >> >> https://github.com/trociny/ceph/compare/wip-osd_create > > This looks reasonable to me! > > Do you mind adding a few test cases in qa/workunits/cephtool/test.sh to go > along with it? > > Usual disclaimer: we discourage getting creative with the osd ids because > they are allocated as an *array* in memory, so skipping entries consumes > some extra memory.. this can become significant if there are large > gaps and/or clusters are large. These options used to exist and were removed quite deliberately. I don't remember the entire conversation at this point but we'll need to find and address the concerns raised then before reintroducing the ability to explicitly set OSD IDs. IIRC I was on the losing end of this, because it's definitely behavior we should be offering to admins, but the issues were significant enough we had to eliminate the option. Methods of preserving the user-facing utility like adding OSD names were deemed too difficult to implement. :( (I think it largely had to do with serious issues over the availability and location of data when OSDs disappear, but new ones with the same ID are present. And what you do when somebody then resurrects the original OSDs. But there might have been other things too.) -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |