MONs are down, the quorum is unable to resolve.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi everyone,

Because of unfortunate events, I’ve a containers based ceph cluster
(nautilus) in a bad shape.

One of the lab cluster which is only made of 2 nodes as control plane (I
know it’s bad :-)) each of these nodes run a mon, a mgr and a rados-gw
containerized ceph_daemon.

They were installed using ceph-ansible if relevant for anyone.

However, when I was performing an upgrade on one of the first nodes, the
second went down too (electrical power outage).

As soon as I saw that I stopped all current process within the upgrading
node.

For now, if I try to restart my second node, as the quorum is looking for
two node the cluster isn’t available.

The container start, the node elect itself as the master but all ceph
commands are stuck forever, which is perfectly normal as the quorum still
wait for one member to achieve the election process etc.

So, my question is, as I can’t (to my knowledge) extract the monmap with
this intermediary state, and as my first node will still be considered as a
known mon and try to join back if started properly, can I just copy the
/etc/ceph.conf and /var/lib/mon/<host>/keyring from the last living node
(the second one) and copy everything at its own place within the first
node? My mon keys were the same for both mon initially and if I’m not
making any mistakes my first node being blank will try to create a default
store, join the existing cluster and try to retrieve the appropriate monmap
from the remaining node right?

If not, is there a process to be able to save/extract the monmap when using
a container based ceph ? I can perfectly exec on the remaining node if it
make any difference.

Thanks a lot!
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux