Had the time to debug it a little bit further today and i think i found a solution ;) The last logline i saw after container start was "Existing mon, trying to rejoin cluster..." https://github.com/ceph/ceph-container/blob/main/src/daemon/start_mon.sh#L154 So i modified the script and added some more debug output. Then i figured out that my monitor was not listed in ceph.conf so the line which evaluates the entry / ip https://github.com/ceph/ceph-container/blob/main/src/daemon/start_mon.sh#L173 cannot work. I modified my ceph.conf (also added v2 / v1 syntax) and now it works without deleting keyring under /var/lib/ceph/mon... and forcing a new bootstrap on every container start. Hope that helps someone ;) Regards, Kilian ________________________________ Von: Stefan Kooman <stefan@xxxxxx> Gesendet: Montag, 20. Juni 2022 16:29:22 An: Kilian Ries; ceph-users@xxxxxxx Betreff: Re: AW: ceph-container: docker restart, mon's unable to join On 6/20/22 16:11, Kilian Ries wrote: > This was related to an older post back from 2020: > > > https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/thread/GBUBEGTZTAMNEQNKUGS47M6W6B4AEVVS/ Yeah, I can remember that. Not sure if I ever got that fixed though. I tried to remember the details past weekend but to no avail. I replaced that setup with a "ceph-ansible" setup. And later on to a cephadm based setup (which it currently is). > > ------------------------------------------------------------------------ > *Von:* Kilian Ries <mail@xxxxxxxxxxxxxx> > *Gesendet:* Freitag, 17. Juni 2022 17:23:03 > *An:* ceph-users@xxxxxxx; stefan@xxxxxx > *Betreff:* ceph-container: docker restart, mon's unable to > join > @Stefan > > Did you find any solution to your problem? I just got the same error ... > I have a running pacific cluster with 4x monitor servers and wanted to > join a fifth monitor via docker container. My container start command is: > > docker run --name ceph-mon-arbiter01 --net=host -v /etc/ceph:/etc/ceph > -v /var/lib/ceph/:/var/lib/ceph/ -e MON_IP=192.168.2.5 -e > CEPH_PUBLIC_NETWORK=192.168.2.0/24 quay.io/ceph/daemon:latest-pacific mon Hmm, I might remember it now ... can you add a "--privileged" to that docker command? This is what I currently have for one of the mons: docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph-mon --privileged --group-add=disk --name ceph-UUID-here-mon.host1 -e CONTAINER_IMAGE=quay.io/ceph/ceph:v15.2.16 Gr. Stefan _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |