Re: [User Question] How to create a backup of an LVM based maschine without wasting space

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Am 12.10.2012 22:43, schrieb Stefan Hajnoczi:
On Fri, Oct 12, 2012 at 9:26 PM, Lukas Laukamp <lukas@xxxxxxxxxx> wrote:
Am 12.10.2012 21:13, schrieb Stefan Hajnoczi:

On Fri, Oct 12, 2012 at 8:14 PM, Lukas Laukamp <lukas@xxxxxxxxxx> wrote:
Am 12.10.2012 13:36, schrieb Stefan Hajnoczi:

On Fri, Oct 12, 2012 at 12:50 PM, Lukas Laukamp <lukas@xxxxxxxxxx>
wrote:
Am 12.10.2012 12:47, schrieb Stefan Hajnoczi:

On Fri, Oct 12, 2012 at 12:16 PM, Lukas Laukamp <lukas@xxxxxxxxxx>
wrote:
Am 12.10.2012 12:11, schrieb Stefan Hajnoczi:

On Fri, Oct 12, 2012 at 11:17 AM, Lukas Laukamp <lukas@xxxxxxxxxx>
wrote:
Am 12.10.2012 10:58, schrieb Lukas Laukamp:

Am 12.10.2012 10:42, schrieb Stefan Hajnoczi:
On Fri, Oct 12, 2012 at 08:52:32AM +0200, Lukas Laukamp wrote:
I have a simple user question. I have a few LVM based KVM guests
and
wan't to backup them to files. The simple and nasty way would be
to
create a complete output file with dd, which wastes very much
space.
So I would like to create a backup of the LVM to a file which
only
locates the space which is used on the LVM. Would be create when
the
output file would be something like a qcow2 file which could be
also
simply startet with KVM.
If the VM is not running you can use "qemu-img convert":

        qemu-img convert -f raw -O qcow2 /dev/vg/vm001
vm001-backup.qcow2

Note that cp(1) tries to make the destination file sparse (see
the
--sparse option in the man page).  So you don't need to use
qcow2,
you
can use cp(1) to copy the LVM volume to a raw file.  It will not
use
disk space for zero regions.

If the VM is running you need to use LVM snapshots or stop the VM
temporarily so a crash-consistent backup can be taken.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hello Stefano,

thanks for the fast reply. I will test this later. In my case now
it
would
be a offline backup. For the online backup I think about a
seperated
system
which every day makes incremental backups and once a week a full
backup.
The
main problem is, that the systems are in a WAN network and I need
encryption
between the systems. Would it be possible to do something like
this:
create
the LVM snapshot for the backup, read this LVM snapshot with the
remote
backup system via ssh tunnel and save the output of this to qcow2
files
on
the backup system? And in which format could be the incremental
backups
be
stored?
Since there is a WAN link it's important to use a compact image
representation before hitting the network. I would use qemu-img
convert -O qcow2 on the host and only transfer the qcow2 output.
The
qcow2 file does not contain zero regions and will therefore save a
lot
of network bandwidth compared to accessing the LVM volume over the
WAN.

If you are using rsync or another tool it's a different story.  You
could rsync the current LVM volume on the host over the last full
backup, it should avoid transferring image data which is already
present in the last full backup - the result is that you only
transfer
changed data plus the rsync metadata.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hello Stefan,

the rsync part I don't have understood fully. So to create a qcow2 on
the
host, transfer this to the backup server will result in the weekly
full
backup. So do you mean I could use rsync to read the LVM from the
host,
compare the LVM data with the data in the qcow2 on the backup server
and
simply transfer the differences to the file? Or does it work on
another
way?
When using rsync you can skip qcow2.  Only two objects are needed:
1. The LVM volume on the host.
2. The last full backup on the backup client.

rsync compares #1 and #2 efficiently over the network and only
transfers data from #1 which has changed.

After rsync completes your full backup image is identical to the LVM
volume.  Next week you can use it as the "last" image to rsync
against.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

So I simply update the full backup, which is simply a raw file which
get
mounted while the backup?
The image file does not need to be mounted.  Just rsync the raw image
file.

Stefan

I tested the qemu-img command now, but it does not do that what I want. I
have a VM with a 5GB disk, this disk is not allocated with 1GB of data.
When
I do the convert command the output is a 5GB qcow2 disk. What do I have
to
do to get a qcow2 file with only the allocated space/data from the LVM? I
also tried the -c option of qemu-img convert but the result was nearly
the
same.
Please show the exact command-lines you are using and the qemu-img
info <filename> output afterwards.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hello,

the VM is called mailer and I used this command: qemu-img convert -f raw -c
-O qcow2 /dev/vmdisks/mailer ./vmachines/mailer.qcow2

Im on the /mnt dir because it's a seperate HDD with the folder vmachines.

Thats the output of qemu-img info:

image: ./vmachines/mailer.qcow2
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 4.1G
cluster_size: 65536

I checked the info of the fs inside the guest. The / partition is 4,7GB big
and 1,2GB are used. The fs is ext3.
I wonder if you see better results without the -c option.

Stefan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

So I would say that the results are very similar. So with -c option the file is 650-800MB smaller but thats not very much. I have a few bigger VM disks with sizes like 100GB when we calculat that in % so we see that the compression is very ecactly 20% so I would have an 80GB image of a disk where only 3GBs are used inside the VM.

I think that it must be possible to create an image with a size like the used space + a few hundret MB with metadata or something like that.

Best Regards
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux