The IP that an OSD (or other non-monitor daemon) uses normally depends on what IP is used by the local host to reach the monitor(s). If you want your OSDs to be on a different network, generally the way to do that is to move the monitors to that network too. You can also try the public_network option, which lets you specify a CIDR network (e.g., "10.0.0.0/8"). The OSD (or other daemon) will look at all IPs bound to the local host and pick one that is contained by that network (or by one of the networks in the list--you can list multiple networks). Recreating the OSD isn't necessary or related; the daemon picks a new IP and port each time it is started. HTH, sage On Fri, 8 Feb 2019, Ashley Merrick wrote: > I just tried that, nothing showing in ceph osd ls or ceph osd tree. > > Run the purge command, wiped the disk. > > However after re-creating the OSD it's still trying to connect via the > external IP, I've looked to see if there is an option to specify the osd ID > in ceph-deploy to try and use another ID but does not seem to be an option. > > Any other ideas? > > Ashley > > On Fri, Feb 8, 2019 at 8:14 PM Hector Martin <hector@xxxxxxxxxxxxxx> wrote: > > > On 08/02/2019 20.54, Ashley Merrick wrote: > > > Yes that is all fine, the other 3 OSD's on the node work fine as > > expected, > > > > > > When I did the orginal OSD via ceph-deploy i used the external hostname > > > at the end of the command instead of the internal hostname, I then > > > deleted the OSD and zap'd the disk and re-added using the internal > > > hostname + the other 3 OSD's. > > > > > > The other 3 are using the internal IP fine, the first OSD is not. > > > > > > The config and everything else is fine as I can reboot any of the other > > > 3 OSD's and they work fine, just somewhere the osd.22 is still storing > > > the orginal hostname/ip it was given via ceph-deploy even after a rm / > > > disk zap > > > > The OSDMap stores the OSD IP, though I *think* it's supposed to update > > itself when the OSD's IP changes. > > > > If this is a new OSD and you don't care about the data (or can just let > > it rebalance away), just follow the instructions for "add/remove OSDs" > > here: > > > > http://docs.ceph.com/docs/mimic/rados/operations/add-or-rm-osds/ > > > > Make sure when the OSD is gone it really is gone (nothing in 'ceph osd > > tree' or 'ceph osd ls'), e.g. 'ceph osd purge <id> > > --yes-i-really-mean-it' and make sure there isn't a spurious entry for > > it in ceph.conf, then re-deploy. Once you do that there is no possible > > other place for the OSD to somehow remember its old IP. > > > > > > -- > > Hector Martin (hector@xxxxxxxxxxxxxx) > > Public Key: https://mrcn.st/pub > > > e _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |