I'm testing the change (actually re-starting the monitors after the monitor removal), but this brings up the issue with why we didn't want to do this in the first place: When reducing the number of monitors from 5 to 3, we are guaranteed to have a service outage for the time it takes to restart at least one of the monitors (and, possibly, for two of the restarts, now that I think on it). In theory, the stop/start cycle is very short and should complete in a reasonable time. What I'm concerned about, however, is the case that something is wrong with our re-written config file. In that case, the outage is immediate and will last until the problem is corrected on the first server to have the monitor restarted. On Mon, Jun 24, 2013 at 10:07 AM, John Nielsen <lists@xxxxxxxxxxxx> wrote: > On Jun 21, 2013, at 5:00 PM, Mandell Degerness <mandell@xxxxxxxxxxxxxxx> wrote: > >> There is a scenario where we would want to remove a monitor and, at a >> later date, re-add the monitor (using the same IP address). Is there >> a supported way to do this? I tried deleting the monitor directory >> and rebuilding from scratch following the add monitor procedures from >> the web, but the monitor still suicide's when started. > > > I assume you're already referencing this: > http://ceph.com/docs/master/rados/operations/add-or-rm-mons/ > > I have done what you describe before. There were a couple hiccups, let's see if I remember the specifics: > > Remove: > Follow the first two steps under "removing a monitor (manual) at the link above: > service ceph stop mon.N > ceph mon remove N > Comment out the monitor entry in ceph.conf on ALL mon, osd and client hosts. > Restart services as required to make everyone happy with the smaller set of monitors > > Re-add: > Wipe the old monitor's directory and re-create it > Follow the steps for "adding a monitor (manual) at the link above. Instead of adding a new entry you can just un-comment your old ones in ceph.conf. You can also start the monitor with "service ceph start mon N" on the appropriate host instead of running yourself (step 8). Note that you DO need to run ceph-mon as specified in step 5. I was initially confused about the '--mkfs' flag there--it doesn't refer to the OS's filesystem, you should use a directory or mountpoint that is already prepared/mounted. > > HTH. If you run into trouble post exactly the steps you followed and additional details about your setup. > > JN > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |