On Fri, Oct 12, 2012 at 3:02 PM, Lukas Laukamp <lukas@xxxxxxxxxx> wrote: > Am 12.10.2012 14:59, schrieb Stefan Hajnoczi: > >> On Fri, Oct 12, 2012 at 1:51 PM, Lukas Laukamp <lukas@xxxxxxxxxx> wrote: >>> >>> Am 12.10.2012 13:36, schrieb Stefan Hajnoczi: >>> >>>> On Fri, Oct 12, 2012 at 12:50 PM, Lukas Laukamp <lukas@xxxxxxxxxx> >>>> wrote: >>>>> >>>>> Am 12.10.2012 12:47, schrieb Stefan Hajnoczi: >>>>> >>>>>> On Fri, Oct 12, 2012 at 12:16 PM, Lukas Laukamp <lukas@xxxxxxxxxx> >>>>>> wrote: >>>>>>> >>>>>>> Am 12.10.2012 12:11, schrieb Stefan Hajnoczi: >>>>>>> >>>>>>>> On Fri, Oct 12, 2012 at 11:17 AM, Lukas Laukamp <lukas@xxxxxxxxxx> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Am 12.10.2012 10:58, schrieb Lukas Laukamp: >>>>>>>>> >>>>>>>>>> Am 12.10.2012 10:42, schrieb Stefan Hajnoczi: >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 12, 2012 at 08:52:32AM +0200, Lukas Laukamp wrote: >>>>>>>>>>>> >>>>>>>>>>>> I have a simple user question. I have a few LVM based KVM guests >>>>>>>>>>>> and >>>>>>>>>>>> wan't to backup them to files. The simple and nasty way would be >>>>>>>>>>>> to >>>>>>>>>>>> create a complete output file with dd, which wastes very much >>>>>>>>>>>> space. >>>>>>>>>>>> So I would like to create a backup of the LVM to a file which >>>>>>>>>>>> only >>>>>>>>>>>> locates the space which is used on the LVM. Would be create when >>>>>>>>>>>> the >>>>>>>>>>>> output file would be something like a qcow2 file which could be >>>>>>>>>>>> also >>>>>>>>>>>> simply startet with KVM. >>>>>>>>>>> >>>>>>>>>>> If the VM is not running you can use "qemu-img convert": >>>>>>>>>>> >>>>>>>>>>> qemu-img convert -f raw -O qcow2 /dev/vg/vm001 >>>>>>>>>>> vm001-backup.qcow2 >>>>>>>>>>> >>>>>>>>>>> Note that cp(1) tries to make the destination file sparse (see >>>>>>>>>>> the >>>>>>>>>>> --sparse option in the man page). So you don't need to use >>>>>>>>>>> qcow2, >>>>>>>>>>> you >>>>>>>>>>> can use cp(1) to copy the LVM volume to a raw file. It will not >>>>>>>>>>> use >>>>>>>>>>> disk space for zero regions. >>>>>>>>>>> >>>>>>>>>>> If the VM is running you need to use LVM snapshots or stop the VM >>>>>>>>>>> temporarily so a crash-consistent backup can be taken. >>>>>>>>>>> >>>>>>>>>>> Stefan >>>>>>>>>>> -- >>>>>>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>>>>>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hello Stefano, >>>>>>>>>> >>>>>>>>>> thanks for the fast reply. I will test this later. In my case now >>>>>>>>>> it >>>>>>>>>> would >>>>>>>>>> be a offline backup. For the online backup I think about a >>>>>>>>>> seperated >>>>>>>>>> system >>>>>>>>>> which every day makes incremental backups and once a week a full >>>>>>>>>> backup. >>>>>>>>>> The >>>>>>>>>> main problem is, that the systems are in a WAN network and I need >>>>>>>>>> encryption >>>>>>>>>> between the systems. Would it be possible to do something like >>>>>>>>>> this: >>>>>>>>>> create >>>>>>>>>> the LVM snapshot for the backup, read this LVM snapshot with the >>>>>>>>>> remote >>>>>>>>>> backup system via ssh tunnel and save the output of this to qcow2 >>>>>>>>>> files >>>>>>>>>> on >>>>>>>>>> the backup system? And in which format could be the incremental >>>>>>>>>> backups >>>>>>>>>> be >>>>>>>>>> stored? >>>>>>>> >>>>>>>> Since there is a WAN link it's important to use a compact image >>>>>>>> representation before hitting the network. I would use qemu-img >>>>>>>> convert -O qcow2 on the host and only transfer the qcow2 output. >>>>>>>> The >>>>>>>> qcow2 file does not contain zero regions and will therefore save a >>>>>>>> lot >>>>>>>> of network bandwidth compared to accessing the LVM volume over the >>>>>>>> WAN. >>>>>>>> >>>>>>>> If you are using rsync or another tool it's a different story. You >>>>>>>> could rsync the current LVM volume on the host over the last full >>>>>>>> backup, it should avoid transferring image data which is already >>>>>>>> present in the last full backup - the result is that you only >>>>>>>> transfer >>>>>>>> changed data plus the rsync metadata. >>>>>>>> >>>>>>>> Stefan >>>>>>>> -- >>>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>> >>>>>>> >>>>>>> Hello Stefan, >>>>>>> >>>>>>> the rsync part I don't have understood fully. So to create a qcow2 on >>>>>>> the >>>>>>> host, transfer this to the backup server will result in the weekly >>>>>>> full >>>>>>> backup. So do you mean I could use rsync to read the LVM from the >>>>>>> host, >>>>>>> compare the LVM data with the data in the qcow2 on the backup server >>>>>>> and >>>>>>> simply transfer the differences to the file? Or does it work on >>>>>>> another >>>>>>> way? >>>>>> >>>>>> When using rsync you can skip qcow2. Only two objects are needed: >>>>>> 1. The LVM volume on the host. >>>>>> 2. The last full backup on the backup client. >>>>>> >>>>>> rsync compares #1 and #2 efficiently over the network and only >>>>>> transfers data from #1 which has changed. >>>>>> >>>>>> After rsync completes your full backup image is identical to the LVM >>>>>> volume. Next week you can use it as the "last" image to rsync >>>>>> against. >>>>>> >>>>>> Stefan >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>>> >>>>> So I simply update the full backup, which is simply a raw file which >>>>> get >>>>> mounted while the backup? >>>> >>>> The image file does not need to be mounted. Just rsync the raw image >>>> file. >>>> >>>> Stefan >>> >>> >>> Ah, thats great so to have a complete task in mind: >>> >>> 1. Create a qcow2 of a snapshot of an running VM >>> 2. Transfer the qcow2 to the backup node >>> 3. For incremental backup sync a daily LVM snapshot with the image on the >>> backup node via rsync >> >> Yes. Only make sure not to rsync the raw LVM image onto the qcow2 >> image - the backup client needs to have a raw image if you're syncing >> directly against the raw LVM volume. >> >> Stefan > > > So I should use a raw file for the syncing and convert it on the backup node > to a qcow2 to have a good file format? I would leave them raw as long as they are sparse (zero regions do not take up space). If you need to copy them you can either convert to qcow2 or use tools that preserve sparseness (BTW compression tools are good at this). But it's up to you. Stefan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
