----- On Sep 7, 2018, at 9:26 PM, Eric Blake eblake@xxxxxxxxxx wrote: > On 09/07/2018 12:06 PM, Lentes, Bernd wrote: >> Hi, >> >> currently i'm following >> https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit. I 'm >> playing around with it and it seems to be quite nice. >> What i want is a daily consistent backup of my image file of the guest. >> I have the idea of the following procedure: >> >> - Shutdown the guest (i can live with a downtime of a few minutes, it will >> happen in the night). >> And i think it's the only way to have a real clean snapshot > > Not the only way - you can also use qemu-guest-agent with a trusted > guest to quiesce all filesystem I/O after freezing database operations > at a consistent point, for a clean snapshot of a live guest. But > shutting down is indeed safe, and easier to reason about than worrying > whether your qga interaction is properly hooked into all necessary > places for a live quiesce. > >> - create a snapshot with snapshot-create-as: snapshot-create-as guest testsn >> --disk-only >> - start the guest again. Changes will now go into the overlay, as e.g. inserts >> in a database >> - rsync the base file to a cifs server. With rsync not the complete, likely big >> file is transferred but just the delta > > We're also trying to add support for incremental backups into a future > version of libvirt on top of the qemu 3.0 feature of persistent bitmaps > in qcow2 images, which could indeed guarantee that you transfer only the > portions of the guest disk that were touched since the last backup. But > as that's still something I'm trying to code up, your solution of using > rsync to pick out the deltas is as good as anything you can get right now. > >> - blockcommit the overlay: blockcommit guest /path/to/testsn --active --wait >> --verbose --pivot >> - delete the snapshot: snapshot-delete guest --snapshotname testsn --metadata >> - remove the overlay >> >> Is that ok ? How "safe" is blockcommit on a running guest ? > > Yep, that's the right way to do it! It's perfectly safe - the guest > doesn't care whether it is reading/writing from the backing file or the > overlay, and even if the blockcommit action is aborted, you can restart > it for the same effects. > > (Okay, if you want to get technical, you need to know that committing > from 'Base <- mid <- top' down to 'Base' leaves 'mid' in an inconsistent > state - but that's not something the guest can see through 'top'; and > your specific case is just committing to the immediate backing layer > rather than skipping layers like that). > >> It's possible that during the rsync, when the guest is running, some inserts are >> done in a database. > > As long as the backing file is read-only during the rsync (which it is, > since all your guest writes are going to the overlay), then nothing the > guest can do will interfere with what rsync can see. Just be sure that > you don't start the blockcommit until after rsync is done. > >> Is it safe to copy the new sectors (i assume that's what blockcommit does) under >> a running database ? >> Or is it only safe doing blockcommit on a stopped guest ? > > Live blockcommit is safe, and exists so you don't have to stop the guest. > > For a bit more insight into what is going on under the hood, the slides > from my KVM Forum talk from a few years back may give some nice insights: > http://events17.linuxfoundation.org/sites/events/files/slides/2015-qcow2-expanded.pdf > >> >> Thanks for any answer. >> >> Bernd >> >> P.S. Is the same procedure possible when the guest disk(s) reside directly in a >> plain logical volume, without a file system in-between ? > > Live blockcommit works onto any host storage protocol (whether > filesystem, block device via LVM, or even remote access such as NBD or > ceph). The key is that your overlay is a qcow2 file that is tracking > the deltas during the time in which you are capturing your backup of the > backing file, and then blockcommit safely writes those deltas back into > the backing file prior to no longer needing the overlay. > > -- > Eric Blake, Principal Software Engineer > Red Hat, Inc. +1-919-301-3266 > Virtualization: qemu.org | libvirt.org Hi Eric, a big thanks to this outstanding clear and thorough answer. It's a pleasure to get such information from the developers themselves, and so quick ! Bernd Helmholtz Zentrum Muenchen Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH) Ingolstaedter Landstr. 1 85764 Neuherberg www.helmholtz-muenchen.de Aufsichtsratsvorsitzende: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrer: Prof. Dr. med. Dr. h.c. Matthias H. Tschoep, Heinrich Bassler, Dr. rer. nat. Alfons Enhsen Registergericht: Amtsgericht Muenchen HRB 6466 USt-IdNr: DE 129521671 _______________________________________________ libvirt-users mailing list libvirt-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvirt-users
