On 17/04/17 21:16, Richard Hesse wrote: > I'm just spitballing here, but what if you set osd crush update on start > = false ? Ansible would activate the OSD's but not place them in any > particular rack, working around the ceph.conf problem you mentioned. > Then you could place them in your CRUSH map by hand. I know you wanted > to avoid editing the CRUSH map by hand, but it's usually the safest route. It scales really badly - "edit CRUSH map by hand" isn't really something that I can automate; presumably something could be lashed up with ceph osd crush add-bucket and ceph osd set ... but that feels more like a lash-up and less like a properly-engineered solution to what must be a fairly common problem? Regards, Matthew > On Wed, Apr 12, 2017 at 4:46 PM, Matthew Vernon <mv3@xxxxxxxxxxxx > <mailto:mv3@xxxxxxxxxxxx>> wrote: > > Hi, > > Our current (jewel) CRUSH map has rack / host / osd (and the default > replication rule does step chooseleaf firstn 0 type rack). We're shortly > going to be adding some new hosts in new racks, and I'm wondering what > the least-painful way of getting the new osds associated with the > correct (new) rack will be. > > We deploy with ceph-ansible, which can add bits of the form > [osd.104] > osd crush location = root=default rack=1 host=sto-1-1 > > to ceph.conf, but I think this doesn't help for new osds, since > ceph-disk will activate them before ceph.conf is fully assembled (and > trying to arrange it otherwise would be serious hassle). > > Would making a custom crush location hook be the way to go? then it'd > say rack=4 host=sto-4-x and new osds would end up allocated to rack 4? > And would I need to have done ceph osd crush add-bucket rack4 rack > first, presumably? > > I am planning on adding osds to the cluster one box at a time, rather > than going with the add-everything-at-crush-weight-0 route; if nothing > else it seems easier to automate. And I'd rather avoid having to edit > the crush map directly... > > Any pointers welcomed :) > > Regards, > > Matthew > > > -- > The Wellcome Trust Sanger Institute is operated by Genome Research > Limited, a charity registered in England with number 1021457 and a > company registered in England with number 2742969, whose registered > office is 215 Euston Road, London, NW1 2BE. > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> > > -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |