This is simply not true. We run quite a few ceph clusters with rack-level layer2 domains (thus routing between racks) and everything works great. On Wed, Aug 30, 2017 at 10:52 AM, David Turner <drakonstein@xxxxxxxxx> wrote: > ALL OSDs need to be running the same private network at the same time. ALL > clients, RGW, OSD, MON, MGR, MDS, etc, etc need to be running on the same > public network at the same time. You cannot do this as a one at a time > migration to the new IP space. Even if all of the servers can still > communicate via routing, it just won't work. Changing the public/private > network addresses for a cluster requires full cluster down time. > > On Wed, Aug 30, 2017 at 11:09 AM Ben Morrice <ben.morrice@xxxxxxx> wrote: >> >> Hello >> >> We have a small cluster that we need to move to a different network in >> the same datacentre. >> >> My workflow was the following (for a single OSD host), but I failed >> (further details below) >> >> 1) ceph osd set noout >> 2) stop ceph-osd processes >> 3) change IP, gateway, domain (short hostname is the same), VLAN >> 4) change references of OLD IP (cluster and public network) in >> /etc/ceph/ceph.conf with NEW IP (see [1]) >> 5) start a single OSD process >> >> This seems to work as the NEW IP can communicate with mon hosts and osd >> hosts on the OLD network, the OSD is booted and is visible via 'ceph -w' >> however after a few seconds the OSD drops with messages such as the >> below in it's log file >> >> heartbeat_check: no reply from 10.1.1.100:6818 osd.14 ever on either >> front or back, first ping sent 2017-08-30 16:42:14.692210 (cutoff >> 2017-08-30 16:42:24.962245) >> >> There are logs like the above for every OSD server/process >> >> and then eventually a >> >> 2017-08-30 16:42:14.486275 7f6d2c966700 0 log_channel(cluster) log >> [WRN] : map e85351 wrongly marked me down >> >> >> Am I missing something obvious to reconfigure the network on a OSD host? >> >> >> >> [1] >> >> OLD >> [osd.0] >> host = sn01 >> devs = /dev/sdi >> cluster addr = 10.1.1.101 >> public addr = 10.1.1.101 >> NEW >> [osd.0] >> host = sn01 >> devs = /dev/sdi >> cluster addr = 10.1.2.101 >> public addr = 10.1.2.101 >> >> -- >> Kind regards, >> >> Ben Morrice >> >> ______________________________________________________________________ >> Ben Morrice | e: ben.morrice@xxxxxxx | t: +41-21-693-9670 >> EPFL / BBP >> Biotech Campus >> Chemin des Mines 9 >> 1202 Geneva >> Switzerland >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |