Re: [User Question] How to create a backup of an LVM based maschine without wasting space

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Oct 12, 2012 at 8:14 PM, Lukas Laukamp <lukas@xxxxxxxxxx> wrote:
> Am 12.10.2012 13:36, schrieb Stefan Hajnoczi:
>
>> On Fri, Oct 12, 2012 at 12:50 PM, Lukas Laukamp <lukas@xxxxxxxxxx> wrote:
>>>
>>> Am 12.10.2012 12:47, schrieb Stefan Hajnoczi:
>>>
>>>> On Fri, Oct 12, 2012 at 12:16 PM, Lukas Laukamp <lukas@xxxxxxxxxx>
>>>> wrote:
>>>>>
>>>>> Am 12.10.2012 12:11, schrieb Stefan Hajnoczi:
>>>>>
>>>>>> On Fri, Oct 12, 2012 at 11:17 AM, Lukas Laukamp <lukas@xxxxxxxxxx>
>>>>>> wrote:
>>>>>>>
>>>>>>> Am 12.10.2012 10:58, schrieb Lukas Laukamp:
>>>>>>>
>>>>>>>> Am 12.10.2012 10:42, schrieb Stefan Hajnoczi:
>>>>>>>>>
>>>>>>>>> On Fri, Oct 12, 2012 at 08:52:32AM +0200, Lukas Laukamp wrote:
>>>>>>>>>>
>>>>>>>>>> I have a simple user question. I have a few LVM based KVM guests
>>>>>>>>>> and
>>>>>>>>>> wan't to backup them to files. The simple and nasty way would be
>>>>>>>>>> to
>>>>>>>>>> create a complete output file with dd, which wastes very much
>>>>>>>>>> space.
>>>>>>>>>> So I would like to create a backup of the LVM to a file which only
>>>>>>>>>> locates the space which is used on the LVM. Would be create when
>>>>>>>>>> the
>>>>>>>>>> output file would be something like a qcow2 file which could be
>>>>>>>>>> also
>>>>>>>>>> simply startet with KVM.
>>>>>>>>>
>>>>>>>>> If the VM is not running you can use "qemu-img convert":
>>>>>>>>>
>>>>>>>>>       qemu-img convert -f raw -O qcow2 /dev/vg/vm001
>>>>>>>>> vm001-backup.qcow2
>>>>>>>>>
>>>>>>>>> Note that cp(1) tries to make the destination file sparse (see the
>>>>>>>>> --sparse option in the man page).  So you don't need to use qcow2,
>>>>>>>>> you
>>>>>>>>> can use cp(1) to copy the LVM volume to a raw file.  It will not
>>>>>>>>> use
>>>>>>>>> disk space for zero regions.
>>>>>>>>>
>>>>>>>>> If the VM is running you need to use LVM snapshots or stop the VM
>>>>>>>>> temporarily so a crash-consistent backup can be taken.
>>>>>>>>>
>>>>>>>>> Stefan
>>>>>>>>> --
>>>>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>>>
>>>>>>>>
>>>>>>>> Hello Stefano,
>>>>>>>>
>>>>>>>> thanks for the fast reply. I will test this later. In my case now it
>>>>>>>> would
>>>>>>>> be a offline backup. For the online backup I think about a seperated
>>>>>>>> system
>>>>>>>> which every day makes incremental backups and once a week a full
>>>>>>>> backup.
>>>>>>>> The
>>>>>>>> main problem is, that the systems are in a WAN network and I need
>>>>>>>> encryption
>>>>>>>> between the systems. Would it be possible to do something like this:
>>>>>>>> create
>>>>>>>> the LVM snapshot for the backup, read this LVM snapshot with the
>>>>>>>> remote
>>>>>>>> backup system via ssh tunnel and save the output of this to qcow2
>>>>>>>> files
>>>>>>>> on
>>>>>>>> the backup system? And in which format could be the incremental
>>>>>>>> backups
>>>>>>>> be
>>>>>>>> stored?
>>>>>>
>>>>>> Since there is a WAN link it's important to use a compact image
>>>>>> representation before hitting the network. I would use qemu-img
>>>>>> convert -O qcow2 on the host and only transfer the qcow2 output.  The
>>>>>> qcow2 file does not contain zero regions and will therefore save a lot
>>>>>> of network bandwidth compared to accessing the LVM volume over the
>>>>>> WAN.
>>>>>>
>>>>>> If you are using rsync or another tool it's a different story.  You
>>>>>> could rsync the current LVM volume on the host over the last full
>>>>>> backup, it should avoid transferring image data which is already
>>>>>> present in the last full backup - the result is that you only transfer
>>>>>> changed data plus the rsync metadata.
>>>>>>
>>>>>> Stefan
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>>
>>>>> Hello Stefan,
>>>>>
>>>>> the rsync part I don't have understood fully. So to create a qcow2 on
>>>>> the
>>>>> host, transfer this to the backup server will result in the weekly full
>>>>> backup. So do you mean I could use rsync to read the LVM from the host,
>>>>> compare the LVM data with the data in the qcow2 on the backup server
>>>>> and
>>>>> simply transfer the differences to the file? Or does it work on another
>>>>> way?
>>>>
>>>> When using rsync you can skip qcow2.  Only two objects are needed:
>>>> 1. The LVM volume on the host.
>>>> 2. The last full backup on the backup client.
>>>>
>>>> rsync compares #1 and #2 efficiently over the network and only
>>>> transfers data from #1 which has changed.
>>>>
>>>> After rsync completes your full backup image is identical to the LVM
>>>> volume.  Next week you can use it as the "last" image to rsync
>>>> against.
>>>>
>>>> Stefan
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>> So I simply update the full backup, which is simply a raw file which get
>>> mounted while the backup?
>>
>> The image file does not need to be mounted.  Just rsync the raw image
>> file.
>>
>> Stefan
>
>
> I tested the qemu-img command now, but it does not do that what I want. I
> have a VM with a 5GB disk, this disk is not allocated with 1GB of data. When
> I do the convert command the output is a 5GB qcow2 disk. What do I have to
> do to get a qcow2 file with only the allocated space/data from the LVM? I
> also tried the -c option of qemu-img convert but the result was nearly the
> same.

Please show the exact command-lines you are using and the qemu-img
info <filename> output afterwards.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux