> Op 30 juli 2017 om 2:42 schreef Stuart Longland <stuartl@xxxxxxxxxxxxxxxxxx>: > > > Hi all, > > I'm setting up an experimental cluster at home, consisting of 3 nodes > which run combined ceph-osd and ceph-mon daemons, and a pair of nodes > that run virtual machines. > > All 5 nodes run Intel Atom C2750s with 8GB RAM and quad-ethernet, with > Gentoo Linux being the host OS. The gory details on the kit concerned > is here if anyone is interested: > https://hackaday.io/project/10529-solar-powered-cloud-computing > > The machines are dual-stack, with public IPv6 addresses and private IPv4 > addresses. Consequently the DNS records reflect this. I note though > that ceph-mon does not listen on the IPv6 port. > > As a result, I see messages like this from clients: > > oneadmin@opennebula:~$ rados df --id libvirt > > 2017-07-30 09:58:32.389376 7f4f611b4700 0 -- :/3981532287 >> [2001:44b8:21ac:70fc::2]:6789/0 pipe(0x560bbfd3eb80 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x560bbfd3fe60).fault > > It may bounce around a few times before it gives up on IPv6 and reverts > back to IPv4. I tried setting up xinetd to proxy the V6 connection to > V4, but then ceph-mon refuses the connection from xinetd. > > My only work-around has been to change the clients to use IPv4 addresses > instead of DNS hostnames, which is distasteful at best. See my information below, but you should create a DNS record with only A or AAAA-records, not mixed. > > The cluster otherwise works fine, and I'm getting good performance. > Easily better than a similar one I did for my workplace beforehand > (based on Intel Core i3 hosts). This may be down to improvements in > Ceph since I last did that deployment, or it might be down to hardware > (using proper server-grade kit) … I'm not sure. > > IPv6 connectivity is handy with regard to VPNs since I'm guaranteed to > be able to access my hosts without address clashes, irrespective of the > local IPv4 subnet in use. (e.g. years ago I used to use 10.0.0.0/24; > then hit a problem with my uni using 10.0.0.0/16 for their WiFi.) > > I've read the documentation, and while there appears to be a flag to > make OSDs bind to IPv6, there doesn't seem to be an option for monitors. > ms_bind_ipv6 = true But keep in mind, that MONs will ONLY bind the the address specified on the MONMap. A Ceph cluster can't run Dual-Stack, so it has to run either IPv4 OR IPv6. > Given that it all seems to be TCP communications, I'm surprised we can't > just bind a socket using AF_INET6 and support both simultaneously. Done > that in the past, and it has worked well. > > Anyway, is there something I missed or has this been overlooked? No, this is on purpose. Ceph relies heavily on OSDMaps and MONMaps which contain IPs internally. You'd have to choose if you want to run Ceph over v4 or v6, but it's not possible to mix this. Wido > -- > Stuart Longland (aka Redhatter, VK4MSL) > > I haven't lost my mind... > ...it's backed up on a tape somewhere. > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |