Hi Don,
I reconfigured the Monitors network recently.
My environment is ceph 0.80.7; Openstack Icehouse ; nova, glance, cinder
using ceph RBD ; RHEL7.0 nodes.
The first thing to do is to check that your new network config will
allow communications between your MONs (I assume you have 3 mons), with
the OSDs and with the Ceph clients. You have to make sure that basic
network connectivity works, that port 6789 is open, and that you don't
have any MTU issues.
Basically I followed this proc:
http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address-the-messy-way
I did the update in one shot, which is disruptive, acceptable in my
non-production environment :
- update ceph.conf so that it includes the new subnet(s), and remove the
old mon subnets. In my case I have three public subnets in the target
config, with routing in between
I updated as well "mon initial members" accordingly.
- deploy ceph.conf to the Ceph nodes and to ceph client nodes
- build a new monmap
- stop all mons
- inject the new monmap to each node
- start the mon on each node
This should establish the quorum, and all mons should be online.
While ceph was happy with the new config, I then hit a pretty bad issue
with OpenStack and RBD, which prevented any VM from starting.
The reason is that /var/lib/nova/instances/<instance UUID>/libvirt.xml
and /etc/libvirt/qemu/<VM name>.xml keep the old IP definition of
the monitors. Rebooting hard the VM does not solve the problem. I solved
this by migrating all VMs. This rebuilds the XML file.
"virsh edit" is maybe the best solution. I haven't tried it yet.
I will reconfigure the OSD network in a couple of days.
HTH
Francois
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
