Hi Robert, After removing the dead monitors with the monmaptool the mon container has vanished from podman. So this somehow made things worse. Is it possible to create and add new monitors? Re-bootstrap the cluster in lack of better terms? //Jens -----Oprindelig meddelelse----- Fra: Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx> Sendt: Thursday, January 26, 2023 1:25 PM Til: ceph-users@xxxxxxx Emne: Re: ceph 16.2.10 cluster down Hi, On 26.01.23 12:46, Jens Galsgaard wrote: > > Setup is: > 3 hosts with each 12 disks (osd/mon) > 3 vm's with mon/mds/mgr > > The vm's are unavailable at the moment and one of the hosts is online with osd/mon running. You have only one out of six MONs running. This MON is unable to form a quorum. Best would be to start the other MONs so that you have at least 4 running. They could form a quorum and then the cluster will respond again. If that is not possible and you want to recover from a catastrophic failure you need to manually edit the MON map (with monmaptool) and remove all but the running MON from it. Then this MON will only see itself as active in the cluster and form the quorum. https://docs.ceph.com/en/quincy/rados/troubleshooting/troubleshooting-mon https://docs.ceph.com/en/quincy/rados/operations/add-or-rm-mons/#removing-monitors Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin https://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Amtsgericht Berlin-Charlottenburg - HRB 220009 B Geschäftsführer: Peer Heinlein - Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |