On 3/3/21 1:16 PM, Ilya Dryomov wrote:
I have tested with 5.11 kernel (5.11.2-arch1-1 #1 SMP PREEMPT Fri, 26
Feb 2021 18:26:41 +0000 x86_64 GNU/Linux) port 3300 and ms_mode=crc as
well as ms_mode=prefer-crc and that works when cluster is running with
ms_bind_ipv4=false. So the "fix" is to have this config option set: ceph
config set global ms_bind_ipv4 false
Right. According to your original post that was already the case:
"ms_bind_ipv6=trie, ms_bind_ipv4=false".
Indeed, I wrote that. That was not correct. We *did* have set that on a
test cluster, but those changes have never propagated to production.
And from this documentation:
https://docs.ceph.com/en/latest/rados/configuration/network-config-ref/#ipv4-ipv6-dual-stack-mode
we learned that dual stack is not possible for any current stable
release, but might be possible with latest code. So the takeaway is that
the linux kernel client needs fixing to be able to support dual stack
clusters in the future (multiple v1 / v2 address families), and, that
until then you should run with ms_bind_ipv4=false for IPv6 only clusters.
I don't think we do any dual stack testing, whether in userspace or
(certainly!) with the kernel client.
Yeah, there are also quite some combinations possible, especially with
dual stack and separate public / cluster networks, even with different
IP stack maybe? Would be good to know (as in the broader community) what
is being tested / supported with regards to networking. Nowadays
separate public/cluster is not advised anymore (which is a good thing,
thanks Wido) but maybe this should also be made clear for IP families.
IMHO it would be good to test both IPv4 and IPv6.
I'll make a PR to clear up the documenation. Do you want me to create a
tracker for the kernel client? I will happily test your changes.
Sure. You are correct that the kernel client needs a bit a work as we
haven't considered dual stack configurations there at all.
Check, I'll do that and come back with a tracker ID.
Thanks,
Stefan
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
