On 3/2/21 5:42 PM, Ilya Dryomov wrote:
On Tue, Mar 2, 2021 at 9:26 AM Stefan Kooman <stefan@xxxxxx> wrote:
Hi,
On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with
Ceph Octopus 15.2.9 packages installed. The MDS server is running
Nautilus 14.2.16. Messenger v2 has been enabled. Poort 3300 of the
monitors is reachable from the client. At mount time we get the following:
Mar 2 09:01:14 kernel: Key type ceph registered
Mar 2 09:01:14 kernel: libceph: loaded (mon/osd proto 15/24)
Mar 2 09:01:14 kernel: FS-Cache: Netfs 'ceph' registered for caching
Mar 2 09:01:14 kernel: ceph: loaded (mds proto 32)
Mar 2 09:01:14 kernel: libceph: mon4 (1)[mond addr]:6789 session established
Mar 2 09:01:14 kernel: libceph: another match of type 1 in addrvec
Mar 2 09:01:14 kernel: ceph: corrupt mdsmap
Mar 2 09:01:14 kernel: ceph: error decoding mdsmap -22
Mar 2 09:01:14 kernel: libceph: another match of type 1 in addrvec
Mar 2 09:01:14 kernel: libceph: corrupt full osdmap (-22) epoch 98764 off 6357 (0000000027a57a75 of 00000000d3075952-00000000e307797f)
Mar 2 09:02:15 kernel: ceph: No mds server is up or the cluster is laggy
The /etc/ceph/ceph.conf has been adjusted to reflect the messenger v2
changes. ms_bind_ipv6=trie, ms_bind_ipv4=false. The kernel client still
seems to be use the v1 port though (although since 5.11 v2 should be
supported).
Has anyone seen this before? Just guessing here, but could it that the
client tries to speak v2 protocol on v1 port?
Hi Stefan,
Those "another match of type 1" errors suggest that you have two
different v1 addresses for some of or all OSDs and MDSes in osdmap
and mdsmap respectively.
What is the output of "ceph osd dump" and "ceph fs dump"?
That's a lot of output, so I trimmed it:
--- snip ---
osd.0 up in weight 1 up_from 98071 up_thru 98719 down_at 98068
last_clean_interval [96047,98067)
[v2:[2001:7b8:80:1:0:1:2:1]:6848/505534,v1:[2001:7b8:80:1:0:1:2:1]:6854/505534,v2:0.0.0.0:6860/505534,v1:0.0.0.0:6866/505534]
[v2:[2001:7b8:80:1:0:1:2:1]:6872/505534,v1:[2001:7b8:80:1:0:1:2:1]:6878/505534,v2:0.0.0.0:6886/505534,v1:0.0.0.0:6892/505534]
exists,up 93e7d17f-2c7a-4acd-93c0-586dbb7cc6d7
-- snap ---
-- snip ---
[mds.mds1{0:229930080} state up:active seq 144042 addr
[v2:[2001:7b8:80:1:0:1:3:1]:6800/2234186180,v1:[2001:7b8:80:1:0:1:3:1]:6801/2234186180,v2:0.0.0.0:6802/2234186180,v1:0.0.0.0:6803/2234186180]]
Standby daemons:
[mds.mds2{-1:229977514} state up:standby seq 2 addr
[v2:[2001:7b8:80:3:0:2c:3:2]:6800/2983725953,v1:[2001:7b8:80:3:0:2c:3:2]:6801/2983725953,v2:0.0.0.0:6802/2983725953,v1:0.0.0.0:6803/2983725953]]
--- snap ---
We only have a public address network in use, and IPv6 only, So not sure
how we could have multiple v1 addresses for OSDs and MDSes.
Thanks,
Stefan
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
