Most ceph clusters are setup once and then maintained. Even if new OSD nodes are added, it's not a frequent enough operation to warrant automation.
Yes, ceph does provide hooks for automatically updating the CRUSH map (crush location hook), but it's up to you to properly write, debug, and maintain that script. How often are you going to use it? If you're a massive company, sure by all means go for it.
How many OSD's are we talking about? We're about 500 now, and even adding another 2000-3000 is a 5 minute cut/paste job of editing the CRUSH map. If you really are adding racks and racks of OSD's every week, you should have found the crush location hook a long time ago.
On Tue, Apr 18, 2017 at 12:53 PM, Matthew Vernon <mv3@xxxxxxxxxxxx> wrote:
On 17/04/17 21:16, Richard Hesse wrote:
> I'm just spitballing here, but what if you set osd crush update on start
> = false ? Ansible would activate the OSD's but not place them in any
> particular rack, working around the ceph.conf problem you mentioned.
> Then you could place them in your CRUSH map by hand. I know you wanted
> to avoid editing the CRUSH map by hand, but it's usually the safest route.
It scales really badly - "edit CRUSH map by hand" isn't really something
that I can automate; presumably something could be lashed up with ceph
osd crush add-bucket and ceph osd set ... but that feels more like a
lash-up and less like a properly-engineered solution to what must be a
fairly common problem?
Regards,
Matthew
> On Wed, Apr 12, 2017 at 4:46 PM, Matthew Vernon <mv3@xxxxxxxxxxxx
> ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxx.> <mailto:mv3@xxxxxxxxxxxx>> wrote:
>
> Hi,
>
> Our current (jewel) CRUSH map has rack / host / osd (and the default
> replication rule does step chooseleaf firstn 0 type rack). We're shortly
> going to be adding some new hosts in new racks, and I'm wondering what
> the least-painful way of getting the new osds associated with the
> correct (new) rack will be.
>
> We deploy with ceph-ansible, which can add bits of the form
> [osd.104]
> osd crush location = root=default rack=1 host=sto-1-1
>
> to ceph.conf, but I think this doesn't help for new osds, since
> ceph-disk will activate them before ceph.conf is fully assembled (and
> trying to arrange it otherwise would be serious hassle).
>
> Would making a custom crush location hook be the way to go? then it'd
> say rack=4 host=sto-4-x and new osds would end up allocated to rack 4?
> And would I need to have done ceph osd crush add-bucket rack4 rack
> first, presumably?
>
> I am planning on adding osds to the cluster one box at a time, rather
> than going with the add-everything-at-crush-weight-0 route; if nothing
> else it seems easier to automate. And I'd rather avoid having to edit
> the crush map directly...
>
> Any pointers welcomed :)
>
> Regards,
>
> Matthew
>
>
> --
> The Wellcome Trust Sanger Institute is operated by Genome Research
> Limited, a charity registered in England with number 1021457 and a
> company registered in England with number 2742969, whose registered
> office is 215 Euston Road, London, NW1 2BE.
> _______________________________________________
> ceph-users mailing list
com >
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. com
> <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. >com
>
>
--
The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
