Hi Jason, that's been very helpful, but it got me thinking and looking. The pool name is both inside the libvirt.xml (and running KVM config) and it's cached in the Nova database. For it to change would require a detach/attach which may not be viable or easy, specially for boot volumes. What about: 1 - upgrade all binaries to Nautilus and ensure Ceph daemons are restarted and all are running Nautilus 2 - stop the OpenStack instances that are on that pool 3 - rename "openstack" pool to "old.openstack" 4 - create new pool "openstack" 5 - "rbd migration" prepare all the RBD volumes in the "old.openstack" 6 - start all instances again. This should ensure all instances are running with the Nautilius binaries 7 - run the "rbd migration execute" and "rbd migration commit" as performance allows when all is finished delete the "old.openstack" pool. As we're running an EC+CT environment will this still apply? Should we remove the CT before doing this or will Ceph be aware of the Cache Tier and do it transparently? I will test all this a few times before doing it in any significant environment. I will do my best to share the experience in the mailing list after... Is there anything we should be aware, and is there anything I could report back to the community to test/experiment/debug the solution? and thanks for all your help. On Fri, Feb 8, 2019 at 5:20 PM Jason Dillaman <jdillama@xxxxxxxxxx> wrote: > > On Fri, Feb 8, 2019 at 11:43 AM Luis Periquito <periquito@xxxxxxxxx> wrote: > > > > This is indeed for an OpenStack cloud - it didn't require any level of > > performance (so was created on an EC pool) and now it does :( > > > > So the idea would be: > > 0 - upgrade OSDs and librbd clients to Nautilus > > > 1- create a new pool > > Are you using EC via cache tier over a replicated pool or an RBD image > with an EC data pool? > > > 2- change cinder to use the new pool > > > > for each volume > > 3- stop the usage of the volume (stop the instance?) > > 4- "live migrate" the volume to the new pool > > Yes, execute the "rbd migration prepare" step here and manually update > the Cinder database to point the instance to the new pool (if the pool > name changed). I cannot remember if Nova caches the Cinder volume > connector details, so you might also need to detach/re-attach the > volume if that's the case (or tweak the Nova database entries as > well). > > > 5- start up the instance again > > 6 - run "rbd migration execute" and "rbd migration commit" at your convenience. > > > > > > > Does that sound right? > > > > thanks, > > > > On Fri, Feb 8, 2019 at 4:25 PM Jason Dillaman <jdillama@xxxxxxxxxx> wrote: > > > > > > Correction: at least for the initial version of live-migration, you > > > need to temporarily stop clients that are using the image, execute > > > "rbd migration prepare", and then restart the clients against the new > > > destination image. The "prepare" step will fail if it detects that the > > > source image is in-use. > > > > > > On Fri, Feb 8, 2019 at 9:00 AM Jason Dillaman <jdillama@xxxxxxxxxx> wrote: > > > > > > > > Indeed, it is forthcoming in the Nautilus release. > > > > > > > > You would initiate a "rbd migration prepare <src-image-spec> > > > > <dst-image-spec>" to transparently link the dst-image-spec to the > > > > src-image-spec. Any active Nautilus clients against the image will > > > > then re-open the dst-image-spec for all IO operations. Read requests > > > > that cannot be fulfilled by the new dst-image-spec will be forwarded > > > > to the original src-image-spec (similar to how parent/child cloning > > > > behaves). Write requests to the dst-image-spec will force a deep-copy > > > > of all impacted src-image-spec backing data objects (including > > > > snapshot history) to the associated dst-image-spec backing data > > > > object. At any point a storage admin can run "rbd migration execute" > > > > to deep-copy all src-image-spec data blocks to the dst-image-spec. > > > > Once the migration is complete, you would just run "rbd migration > > > > commit" to remove src-image-spec. > > > > > > > > Note: at some point prior to "rbd migration commit", you will need to > > > > take minimal downtime to switch OpenStack volume registration from the > > > > old image to the new image if you are changing pools. > > > > > > > > On Fri, Feb 8, 2019 at 5:33 AM Caspar Smit <casparsmit@xxxxxxxxxxx> wrote: > > > > > > > > > > Hi Luis, > > > > > > > > > > According to slide 21 of Sage's presentation at FOSDEM it is coming in Nautilus: > > > > > > > > > > https://fosdem.org/2019/schedule/event/ceph_project_status_update/attachments/slides/3251/export/events/attachments/ceph_project_status_update/slides/3251/ceph_new_in_nautilus.pdf > > > > > > > > > > Kind regards, > > > > > Caspar > > > > > > > > > > Op vr 8 feb. 2019 om 11:24 schreef Luis Periquito <periquito@xxxxxxxxx>: > > > > >> > > > > >> Hi, > > > > >> > > > > >> a recurring topic is live migration and pool type change (moving from > > > > >> EC to replicated or vice versa). > > > > >> > > > > >> When I went to the OpenStack open infrastructure (aka summit) Sage > > > > >> mentioned about support of live migration of volumes (and as a result > > > > >> of pools) in Nautilus. Is this still the case and is expected to have > > > > >> live migration working by then? > > > > >> > > > > >> thanks, > > > > >> _______________________________________________ > > > > >> ceph-users mailing list > > > > >> ceph-users@xxxxxxxxxxxxxx > > > > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > _______________________________________________ > > > > > ceph-users mailing list > > > > > ceph-users@xxxxxxxxxxxxxx > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > > > > > > > > > -- > > > > Jason > > > > > > > > > > > > -- > > > Jason > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@xxxxxxxxxxxxxx > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > -- > Jason _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |