On Tue, 2021-03-02 at 17:44 +0100, Stefan Kooman wrote: > On 3/2/21 5:16 PM, Jeff Layton wrote: > > On Tue, 2021-03-02 at 09:25 +0100, Stefan Kooman wrote: > > > Hi, > > > > > > On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP > > > Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with > > > > I'm guessing this is a stable series kernel > > It's a kernel from 'elrepo', so I'm not sure. I would guess so too. > > > > > > Ceph Octopus 15.2.9 packages installed. The MDS server is running > > > Nautilus 14.2.16. Messenger v2 has been enabled. Poort 3300 of the > > > monitors is reachable from the client. At mount time we get the following: > > > > > > > Mar 2 09:01:14 kernel: Key type ceph registered > > > > Mar 2 09:01:14 kernel: libceph: loaded (mon/osd proto 15/24) > > > > Mar 2 09:01:14 kernel: FS-Cache: Netfs 'ceph' registered for caching > > > > Mar 2 09:01:14 kernel: ceph: loaded (mds proto 32) > > > > Mar 2 09:01:14 kernel: libceph: mon4 (1)[mond addr]:6789 session established > > > > Mar 2 09:01:14 kernel: libceph: another match of type 1 in addrvec > > > > Mar 2 09:01:14 kernel: ceph: corrupt mdsmap > > > > Mar 2 09:01:14 kernel: ceph: error decoding mdsmap -22 > > > > -22 == -EINVAL > > > > Looks like a an osdmap parsing error? > > Indeed, that is kinda weird isn't it. > > > > > > > Mar 2 09:01:14 kernel: libceph: another match of type 1 in addrvec > > > > Mar 2 09:01:14 kernel: libceph: corrupt full osdmap (-22) epoch 98764 off 6357 (0000000027a57a75 of 00000000d3075952-00000000e307797f) > > > > Mar 2 09:02:15 kernel: ceph: No mds server is up or the cluster is laggy > > > > > > The /etc/ceph/ceph.conf has been adjusted to reflect the messenger v2 > > > changes. ms_bind_ipv6=trie, ms_bind_ipv4=false. The kernel client still > > > seems to be use the v1 port though (although since 5.11 v2 should be > > > supported). > > > > > > > The mount helper only recently got v2 support, and that hasn't trickled > > out into the distros yet. See: > > > > https://github.com/ceph/ceph/pull/38788 > > Ah, good to know. > > > > > > Has anyone seen this before? Just guessing here, but could it that the > > > client tries to speak v2 protocol on v1 port? > > > > > > > What mount options are you passing in? Are you using mon autodiscovery? > > We provide the mon address explictly in the /etc/fstab, So something > like this: > > mon1,mon2,mon3,mon4,mon5:/cephfs/dir /client_mountpoint ceph > name=client-id,secretfile=/etc/ceph/ceph.client.client-id.cephfs.key,noatime,_netdev > 0 2 > > We are not using any dns based discovery of monitors if that is what you > mean. > > Note: We tried with nautilus packages before (14.2.16) and got the same > result. > > > > > v2 support in the kernel is keyed on the ms_mode= mount option, so that > > has to be passed in if you're connecting to a v2 port. Until the mount > > helpers get support for that option you'll need to specify the address > > and port manually if you want to use v2. > > I've tried feeding it ms_mode=v2 but I get a "mount error 22 = Invalid > argument", the ms_mode=legacy does work, but fails with the same errors. > That needs different values. See: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=00498b994113a871a556f7ff24a4cf8a00611700 You can try passing in a specific mon address and port, like: 192.168.21.22:3300:/cephfs/dir/ ...and then pass in ms_mode=crc or something similar. That said, what you're doing should be working, so this sounds like a regression. I presume you're able to mount with earlier kernels? What's the latest kernel version that you have that works? -- Jeff Layton <jlayton@xxxxxxxxxx> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |