yes, I think so. I think the context in which this originally came up was somebody trying to replace the tiebreaker mon. On Fri, Dec 2, 2022 at 9:08 AM Sake Paulusma <sake1989@xxxxxxxxxxx> wrote: > That isn't a great solution indeed, but I'll try the solution. Would this > also be necessary to replace the Tiebreaker? > ------------------------------ > *From:* Adam King <adking@xxxxxxxxxx> > *Sent:* Friday, December 2, 2022 2:48:19 PM > *To:* Sake Paulusma <sake1989@xxxxxxxxxxx> > *Cc:* ceph-users@xxxxxxx <ceph-users@xxxxxxx> > *Subject:* Re: How to replace or add a monitor in stretch > cluster? > > This can't be done in a very nice way currently. There's actually an open > PR against main to allow setting the crush location for mons in the service > spec specifically because others found that this was annoying as well. What > I think should work as a workaround is, go to the host where the mon that > failed to join the quorum but failed due to lack of crush location is, open > the /var/lib/ceph/<fsid>/mon.<mon-id>/unit.run file, then at the very end > of the last line (the last line should be a long podman/docker run command) > append "--set-crush-location <crush-loc>". Then, still on that host, a > "systemctl restart <mon-service>" where mon-service is the systemd unit > listed for the mon in "cephadm ls --no-detail". That should allow the > monitor to at least join the quorum as it now has a crush location, and > then you should be able to make other alterations a bit easier. > > On Fri, Dec 2, 2022 at 7:40 AM Sake Paulusma <sake1989@xxxxxxxxxxx> wrote: > > I succesfully setup a stretched cluster, except the CRUSH rule mentioned > in the docs wasn't correct. The parameters for "min_size" and "max_size" > should be removed, or else the rule can't be imported. > Second there should be a mention about setting the monitor crush location > takes sometime and know other ceph command can be used. > > But now I need to replace a few monitors (it's virtualized and machines > need to be replaced). I use cephadm and have a label "mon" which is > assigned to the monitor services. > With the command "ceph orch host add <hostname> <ip-address> --labels=mon" > I add normally a new monitor to the cluster. > > Only this results in the following error in de logs: > 12/2/22 1:27:19 PM [INF] <hostname> attempted to join from <hostname> > [v2:<ip-address>:3300/0,v1:<ip-address>:6789/0]; but lacks a crush_location > for stretch mode > > Next I tried to set the CRUSH location like used in the docs for stretch > mode with the command "ceph mon set_location <hostname> datacenter=<dc1>". > This only results in the following error: > Error ENOENT: mon.oqsoel11437 does not exist > > So how can I add/replace a monitor in a stretched cluster? > > Best regards, > Sake > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |