[PATCH RFC 00/40] qemu: Add support for incremental backups

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This series is my attempt to clean up and finish incremental backup as
it was last posted in v10 here:

https://www.redhat.com/archives/libvir-list/2019-August/msg01018.html

Out of that series, patch 1 was dropped for now and I'll revisit it
later. The patches adding docs, xml parsing and job plumbing were
partially modified and stripped of currently unimplemented
functionality.

The qemu implementation was completely rewritten.

Of the new series first 28 patches are cleanups and can be reviewed for
pushing.

The main incremental backup functionality is not complete yet and should
not be pushed right now not even in experimental (disabled [1]) state
due to possible API changes:

- figure out whether we want our own event for the backup job or want to
deal with the domain job event

- if we go with the domain job event everything must be rewritten as a
single domain job

- otherwise plumbing for the event must be added

- documentation must reflect the above


after that  will be sorted out the following needs to be done until we
can finally enable it:

- more testing. I merely verified that qemu generates files/exports NBD
- blockjob support for bitmaps (for commit/pull/copy)
- snapshot support for bitmaps
- merging of bitmaps accross backing chain entries
- figuring out a more elegant way to do the bitmap for the backup than
creating two, one for the 'store' (for NBD server) and one for the disk
itself.
- ...

Until then the following XML can be used to enable the functionality for
testing without any guarantee [1]:

  <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    ...
    <qemu:capabilities>
      <qemu:add capability='blockdev'/>
      <qemu:add capability='incremental-backup'/>
    </qemu:capabilities>
  </domain>

Eric Blake (5):
  backup: Document new XML for backups
  backup: Introduce virDomainBackup APIs
  backup: Implement backup APIs for remote driver
  backup: Parse and output backup XML
  backup: Implement virsh support for backup

Peter Krempa (35):
  util: hash: Add possibility to use simpler data free function in
    virHash
  util: hash: Add new constructor 'virHashNew'
  util: hash: Introduce virHashHasEntry
  qemu: domain: Split out setup of virStorageSource from qemu driver
    config
  qemu: domain: Remove pointless return value in
    qemuDomainPrepareDiskSourceData
  qemu: domain: clarify sematics of qemuDomainPrepareDiskSourceData
  qemu: domain: Tolerate NULL @disk in qemuDomainPrepareDiskSourceData
  conf: Remove virDomainDiskPathByName
  conf: Introduce virDomainDiskByTarget
  qemu: Replace use of virDomainDiskFindByBusAndDst with
    virDomainDiskByTarget
  conf: Remove unused virDomainDiskFindByBusAndDst
  Replace virDomainDiskByName by virDomainDiskByTarget in appropriate
    cases
  qemu: monitor: Introduce new interface to query-named-block-nodes
  qemu: block: Don't query monitor in
    qemuBlockStorageSourceCreateDetectSize
  qemu: checkpoint: Fix rollback and access to unlocked 'vm' when
    deleting checkpoints
  qemu: monitor: Remove non-transaction based dirty bitmap APIs
  conf: snapshot: Don't clear current snapshot when redefining an
    existing one
  conf: snapshot: Remove 'update_current' parameter from
    virDomainSnapshotRedefinePrep
  conf: Don't reuse variable for different object in
    virDomainCheckpointRedefinePrep
  conf: checkpoint: Don't clear current checkpoint when redefining an
    existing one
  conf: checkpoint: Don't clear current checkpoint when redefining
  qemu: checkpoint: Enforce that 'bitmap' name must match checkpoint
    name
  qemu: checkpoint: Split out checkpoint creation code
  qemu: checkpoint: Extract finalizing steps of checkpoint creation
  qemu: monitor: Add helper for generating data for block bitmap merging
  qemu: checkpoint: Use qemuMonitorTransactionBitmapMergeSourceAddBitmap
  qemu: blockjob: Refactor qemuBlockJobEventProcessConcludedTransition
  qemu: blockjob: Use 'g_free' in qemuBlockJobDataDispose
  Add 'backup' block job type
  qemu: monitor: Add support for blockdev-backup via 'transaction'
  qemu: domain: Track backup job data in the status XML
  qemu: blockjob: Track internal data for 'backup' blockjob
  tests: qemustatusxml2xml: Add test for 'pull' type backup job
  qemu: Implement backup job APIs and qemu handling
  qemu: blockjob: Implement concluded blockjob handler for backup
    blockjobs

 docs/docs.html.in                             |   3 +-
 docs/format.html.in                           |   1 +
 docs/formatbackup.html.in                     | 168 ++++
 docs/formatcheckpoint.html.in                 |  12 +-
 docs/index.html.in                            |   3 +-
 docs/schemas/domainbackup.rng                 | 219 +++++
 examples/c/misc/event-test.c                  |   3 +
 include/libvirt/libvirt-domain.h              |  29 +-
 libvirt.spec.in                               |   1 +
 mingw-libvirt.spec.in                         |   2 +
 src/conf/Makefile.inc.am                      |   2 +
 src/conf/backup_conf.c                        | 518 ++++++++++
 src/conf/backup_conf.h                        | 102 ++
 src/conf/checkpoint_conf.c                    |  17 +-
 src/conf/domain_addr.c                        |   4 +-
 src/conf/domain_conf.c                        |  50 +-
 src/conf/domain_conf.h                        |   8 +-
 src/conf/snapshot_conf.c                      |   6 -
 src/conf/snapshot_conf.h                      |   1 -
 src/conf/virconftypes.h                       |   3 +
 src/driver-hypervisor.h                       |  20 +
 src/libvirt-domain-checkpoint.c               |   7 +-
 src/libvirt-domain.c                          | 191 ++++
 src/libvirt_private.syms                      |  13 +-
 src/libvirt_public.syms                       |   8 +
 src/libxl/libxl_driver.c                      |   2 +-
 src/qemu/Makefile.inc.am                      |   2 +
 src/qemu/qemu_backup.c                        | 895 ++++++++++++++++++
 src/qemu/qemu_backup.h                        |  41 +
 src/qemu/qemu_block.c                         |  26 +-
 src/qemu/qemu_block.h                         |   5 +-
 src/qemu/qemu_blockjob.c                      | 171 ++--
 src/qemu/qemu_blockjob.h                      |  19 +
 src/qemu/qemu_checkpoint.c                    | 232 +++--
 src/qemu/qemu_checkpoint.h                    |  15 +
 src/qemu/qemu_domain.c                        | 161 +++-
 src/qemu/qemu_domain.h                        |  11 +-
 src/qemu/qemu_driver.c                        | 116 ++-
 src/qemu/qemu_hotplug.c                       |   2 +-
 src/qemu/qemu_migration.c                     |   3 +-
 src/qemu/qemu_monitor.c                       |  91 +-
 src/qemu/qemu_monitor.h                       |  49 +-
 src/qemu/qemu_monitor_json.c                  | 242 ++---
 src/qemu/qemu_monitor_json.h                  |  33 +-
 src/remote/remote_driver.c                    |   3 +
 src/remote/remote_protocol.x                  |  53 +-
 src/remote_protocol-structs                   |  28 +
 src/test/test_driver.c                        |   2 +-
 src/util/vircgroup.c                          |   2 +-
 src/util/virhash.c                            |  85 +-
 src/util/virhash.h                            |  12 +
 tests/Makefile.am                             |   2 +
 tests/domainbackupxml2xmlin/backup-pull.xml   |   9 +
 tests/domainbackupxml2xmlin/backup-push.xml   |   9 +
 tests/domainbackupxml2xmlin/empty.xml         |   1 +
 tests/domainbackupxml2xmlout/backup-pull.xml  |   9 +
 tests/domainbackupxml2xmlout/backup-push.xml  |   9 +
 tests/domainbackupxml2xmlout/empty.xml        |   7 +
 tests/qemublocktest.c                         |   3 +-
 tests/qemumonitorjsontest.c                   |  53 +-
 .../qemustatusxml2xmldata/backup-pull-in.xml  | 607 ++++++++++++
 .../qemustatusxml2xmldata/backup-pull-out.xml |   1 +
 tests/qemuxml2xmltest.c                       |   2 +
 tests/virschematest.c                         |   2 +
 tools/Makefile.am                             |   1 +
 tools/virsh-backup.c                          | 209 ++++
 tools/virsh-backup.h                          |  21 +
 tools/virsh-domain.c                          |   8 +-
 tools/virsh.c                                 |   2 +
 tools/virsh.h                                 |   1 +
 tools/virsh.pod                               |  37 +
 71 files changed, 4133 insertions(+), 552 deletions(-)
 create mode 100644 docs/formatbackup.html.in
 create mode 100644 docs/schemas/domainbackup.rng
 create mode 100644 src/conf/backup_conf.c
 create mode 100644 src/conf/backup_conf.h
 create mode 100644 src/qemu/qemu_backup.c
 create mode 100644 src/qemu/qemu_backup.h
 create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlin/empty.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml
 create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml
 create mode 100644 tests/domainbackupxml2xmlout/empty.xml
 create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml
 create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml
 create mode 100644 tools/virsh-backup.c
 create mode 100644 tools/virsh-backup.h

-- 
2.21.0

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list



[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux