Florian; Forgive my lack of knowledge of OpenStack, and your environment / use case. Why would you need / want to snapshot an ephemeral disk? Isn't the point of ephemeral storage to not be persistent? Thank you, Dominic L. Hilsbos, MBA Director - Information Technology Perform Air International Inc. DHilsbos@xxxxxxxxxxxxxx www.PerformAir.com -----Original Message----- From: Florian Haas [mailto:florian@xxxxxxxxxxxxxx] Sent: Wednesday, August 21, 2019 8:41 AM To: dillaman@xxxxxxxxxx Cc: ceph-users Subject: Re: RBD, OpenStack Nova, libvirt, qemu-guest-agent, and FIFREEZE: is this working as intended? Hi Jason! Thanks for the quick reply. On 21/08/2019 16:51, Jason Dillaman wrote:> > It just looks like this was an oversight from the OpenStack developers > when Nova RBD "direct" ephemeral image snapshot support was added [1]. > I would open a bug ticket against Nova for the issue.> > [1] https://opendev.org/openstack/nova/commit/824c3706a3ea691781f4fcc4453881517a9e1c55 OK, wow... that was 4 years ago, does that mean that quiesce/freeze/thaw for RBD-backed Nova instances has probably been non-functional throughout that time? Looking at the reno for that commit I had an idea for a workaround: > features: > - When RBD is used for ephemeral disks and image storage, make > snapshot use Ceph directly, and update Glance with the new location. > In case of failure, it will gracefully fallback to the "generic" > snapshot method. This requires changing the typical permissions > for the Nova Ceph user (if using authx) to allow writing to > the pool where vm images are stored, and it also requires > configuring Glance to provide a v2 endpoint with direct_url > support enabled (there are security implications to doing this). > See http://docs.ceph.com/docs/master/rbd/rbd-openstack/ for more > information on configuring OpenStack with RBD. So, suppose that deployers running Nova with ephemeral disks on RBD prefer snapshot consistency over this shortcut. Until Nova fixes the direct_snapshot() call, I figured that such deployers could tweak the caps for the Nova CephX identity such that that user were no longer allowed to write to the Glance pool. Under those circumstances, the snapshot creation (in the ephemeral pool) would work, but then the clone() call in this line should throw nova.exception.Forbidden from an rbd.PermissionError: https://opendev.org/openstack/nova/src/commit/7bf75976016aae5d458eca9f6ddac92bfe75dc59/nova/virt/libvirt/imagebackend.py#L1075 Which should then trigger this except block: https://opendev.org/openstack/nova/src/commit/7bf75976016aae5d458eca9f6ddac92bfe75dc59/nova/virt/libvirt/driver.py#L2065 ... and Nova/libvirt should go back to the (arguably more correct) fallback. Would you agree with that assessment, or am I missing something? (Just trying to make sure that I don't give the Nova folks the wrong facts.) Thanks again! Cheers, Florian _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |