Ceph has the ability to us a script to figure out where in the crushmap this disk should go (on osd start): http://docs.ceph.com/docs/master/rados/operations/crush-map/#ceph-crush-location-hook -- Adam On Tue, Apr 18, 2017 at 7:53 AM, Matthew Vernon <mv3@xxxxxxxxxxxx> wrote: > On 17/04/17 21:16, Richard Hesse wrote: >> I'm just spitballing here, but what if you set osd crush update on start >> = false ? Ansible would activate the OSD's but not place them in any >> particular rack, working around the ceph.conf problem you mentioned. >> Then you could place them in your CRUSH map by hand. I know you wanted >> to avoid editing the CRUSH map by hand, but it's usually the safest route. > > It scales really badly - "edit CRUSH map by hand" isn't really something > that I can automate; presumably something could be lashed up with ceph > osd crush add-bucket and ceph osd set ... but that feels more like a > lash-up and less like a properly-engineered solution to what must be a > fairly common problem? > > Regards, > > Matthew > >> On Wed, Apr 12, 2017 at 4:46 PM, Matthew Vernon <mv3@xxxxxxxxxxxx >> <mailto:mv3@xxxxxxxxxxxx>> wrote: >> >> Hi, >> >> Our current (jewel) CRUSH map has rack / host / osd (and the default >> replication rule does step chooseleaf firstn 0 type rack). We're shortly >> going to be adding some new hosts in new racks, and I'm wondering what >> the least-painful way of getting the new osds associated with the >> correct (new) rack will be. >> >> We deploy with ceph-ansible, which can add bits of the form >> [osd.104] >> osd crush location = root=default rack=1 host=sto-1-1 >> >> to ceph.conf, but I think this doesn't help for new osds, since >> ceph-disk will activate them before ceph.conf is fully assembled (and >> trying to arrange it otherwise would be serious hassle). >> >> Would making a custom crush location hook be the way to go? then it'd >> say rack=4 host=sto-4-x and new osds would end up allocated to rack 4? >> And would I need to have done ceph osd crush add-bucket rack4 rack >> first, presumably? >> >> I am planning on adding osds to the cluster one box at a time, rather >> than going with the add-everything-at-crush-weight-0 route; if nothing >> else it seems easier to automate. And I'd rather avoid having to edit >> the crush map directly... >> >> Any pointers welcomed :) >> >> Regards, >> >> Matthew >> >> >> -- >> The Wellcome Trust Sanger Institute is operated by Genome Research >> Limited, a charity registered in England with number 1021457 and a >> company registered in England with number 2742969, whose registered >> office is 215 Euston Road, London, NW1 2BE. >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >> >> > > > > -- > The Wellcome Trust Sanger Institute is operated by Genome Research > Limited, a charity registered in England with number 1021457 and a > company registered in England with number 2742969, whose registered > office is 215 Euston Road, London, NW1 2BE. > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |