Hi all, I'm setting up an experimental cluster at home, consisting of 3 nodes which run combined ceph-osd and ceph-mon daemons, and a pair of nodes that run virtual machines. All 5 nodes run Intel Atom C2750s with 8GB RAM and quad-ethernet, with Gentoo Linux being the host OS. The gory details on the kit concerned is here if anyone is interested: https://hackaday.io/project/10529-solar-powered-cloud-computing The machines are dual-stack, with public IPv6 addresses and private IPv4 addresses. Consequently the DNS records reflect this. I note though that ceph-mon does not listen on the IPv6 port. As a result, I see messages like this from clients: > oneadmin@opennebula:~$ rados df --id libvirt > 2017-07-30 09:58:32.389376 7f4f611b4700 0 -- :/3981532287 >> [2001:44b8:21ac:70fc::2]:6789/0 pipe(0x560bbfd3eb80 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x560bbfd3fe60).fault It may bounce around a few times before it gives up on IPv6 and reverts back to IPv4. I tried setting up xinetd to proxy the V6 connection to V4, but then ceph-mon refuses the connection from xinetd. My only work-around has been to change the clients to use IPv4 addresses instead of DNS hostnames, which is distasteful at best. The cluster otherwise works fine, and I'm getting good performance. Easily better than a similar one I did for my workplace beforehand (based on Intel Core i3 hosts). This may be down to improvements in Ceph since I last did that deployment, or it might be down to hardware (using proper server-grade kit) … I'm not sure. IPv6 connectivity is handy with regard to VPNs since I'm guaranteed to be able to access my hosts without address clashes, irrespective of the local IPv4 subnet in use. (e.g. years ago I used to use 10.0.0.0/24; then hit a problem with my uni using 10.0.0.0/16 for their WiFi.) I've read the documentation, and while there appears to be a flag to make OSDs bind to IPv6, there doesn't seem to be an option for monitors. Given that it all seems to be TCP communications, I'm surprised we can't just bind a socket using AF_INET6 and support both simultaneously. Done that in the past, and it has worked well. Anyway, is there something I missed or has this been overlooked? -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
