Le 17/07/2024 à 09:40:59+0200, David C. a écrit
Hi everyone.
>
> The curiosity of Albert's cluster is that (msgr) v1 and v2 are present on the
> mons, as well as on the osds backend.
>
> But v2 is absent on the public OSD and MDS network
>
> The specific point is that the public network has been changed.
>
> At first, I thought it was the order of declaration of my_host (v1 before v2)
> but apparently, that's not it.
>
>
> Le mer. 17 juil. 2024 à 09:21, Frédéric Nass <frederic.nass@xxxxxxxxxxxxxxxx> a
> écrit :
>
> Hi David,
>
> Redeploying 2 out of 3 MONs a few weeks back (to have them using RocksDB to
> be ready for Quincy) prevented some clients from connecting to the cluster
> and mounting cephfs volumes.
>
> Before the redeploy, these clients were using port 6789 (v1) explicitly as
> connections wouldn't work with port 3300 (v2).
> After the redeploy, removing port 6789 from mon_ips fixed the situation.
>
> Seems like msgr v2 activation did only occur after all 3 MONs were
> redeployed and used RocksDB. Not sure why this happened though.
>
> @Albert, if this cluster has been upgrade several times, you might want to
> check /var/lib/ceph/$(ceph fsid)/kv_backend, redeploy the MONS if leveldb,
> make sure all clients use the new mon_host syntax in ceph.conf ([v2:
> <cthulhu1_ip>:3300,v1:<cthulhu1_ip>:6789],etc.]) and check their ability to
> connect to port 3300.
So it's working now, I can mount from all my clients the cephfs.
Because I'm not sure what really happens and where was the issue here what
have been done on the cluster (in that timeline) :
When I change the IP address of the server I make a maybe mistake and put
ceph mon set-addrs cthulhu1 [v1:cthulhu1_new_ip:6789/0,v2:cthulhu1_new_ip:3300/0]
Yesterday David change it to the right way
ceph mon set-addrs cthulhu1 [v2:cthulhu1_new_ip:3300/0,v1:cthulhu1_new_ip:6789/0]
but it's was not enough. Even after restarting all the OSD.
Then have been try some redeploying the mds --> no joy.
This morning I restart a osd and notice the restarted osd listen on v2
and v1, so I restart all osd.
After that, every osd listen on v2 and v1.
But still unable to mount the cephfs.
I try the option ms_mode=prefer-crc but nothing.
So I end up by rebooting the all cluster and now everything work fine.
Thanks for your help.
Regards
--
Albert SHIH 🦫 🐸
Observatoire de Paris
France
Heure locale/Local time:
mer. 17 juil. 2024 15:42:00 CEST
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
