The following is the latest version of my API proposal for incremental backups, and follows along from the demo I presented as part of my KVM Forum 2018 talk: https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat The patches are also available via https://repo.or.cz/libvirt/ericb.git at the tag backup-v5. Depends on my earlier bulk redefine for snapshots, which is pending review at v3: https://www.redhat.com/archives/libvir-list/2019-March/msg00144.html I haven't finished addressing all of John Ferlan's review comments from v4 (including the fact that several of these larger patches need to be split for easier reviews, and the fact that I want virDomainSnapshotObj and virDomainCheckpointObj to share a common base class); but I'm posting this now because I _have_ completed: - push mode backups mostly work (event notification is still not polished, but the fact that I can now do both push and pull modes is nice) - bulk mode list/redefine, accessible via 'virsh dumpxml --checkpoints' and 'virsh snapshot-create --redefine-list' - move virDomainBackup() into libvirt-domain.h (libvirt-domain-checkpoint.h is now JUST about standalone checkpoint objects; then both virDomainBackup() and virDomainSnapshotCreateXML() will be able to create a checkpoint in parallel) - rebase to master (5.2 now instead of 5.1, start using VIR_AUTOFREE in more places, ...) I'm still debating about making <domaincheckpoint> a sub-element of <domainbackup> (in which case I can reuse virDomainSnapshotCreateXML() to create a backup) or keeping it as a separate XML argument (in which case I need to add an API for virDomainSnapshotCreateXMLWithSnapshot()). I also need to work out an API for listing all currently running jobs (even if the current backup code can only run one job at a time, we know we want support in the future for parallel backup jobs). So there WILL be a v6. Hopefully, though, we can start trimming off the front of the series, and at least now the tail of the series currently compiles (John gave up review on v4 when he ran into rebase issues - although with our current rate of VIR_AUTOFREE conversions, there may again be rebase issues soon...) since v4: 001/20:[----] [-C] 'domain_conf: Expose virDomainStorageNetworkParseHost' 002/20:[down] 'qemu: clean up qemuDomainRemoveInactiveCommon' 003/20:[----] [--] 'backup: Document nuances between different state capture APIs' 004/20:[0020] [FC] 'backup: Introduce virDomainCheckpointPtr' 005/20:[down] 'wip: backup: Document new XML for backups' 006/20:[0054] [FC] 'backup: Introduce virDomainCheckpoint APIs' 007/20:[0053] [FC] 'backup: Introduce virDomainBackup APIs' 008/20:[0002] [FC] 'backup: Add new domain:checkpoint access control' 009/20:[0093] [FC] 'backup: Implement backup APIs for remote driver' 010/20:[0381] [FC] 'wip: backup: Parse and output checkpoint XML' 011/20:[0001] [FC] 'wip: backup: Parse and output backup XML' 012/20:[0393] [FC] 'backup: Implement virsh support for checkpoints' 013/20:[0058] [FC] 'wip: backup: virsh support for backup' 014/20:[0019] [FC] 'backup: Add new qemu monitor interactions' 015/20:[0161] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs' 016/20:[----] [-C] 'wip: backup: Wire up qemu checkpoint commands over QMP' 017/20:[0006] [FC] 'wip: backup: qemu: Implement framework for backup job APIs' 018/20:[0046] [FC] 'backup: Wire up qemu full pull backup commands over QMP' 019/20:[down] 'wip: backup: qemu: Wire up qemu full push backup commands over QMP' 020/20:[0022] [FC] 'backup: implement qemu incremental pull backup' Eric Blake (20): domain_conf: Expose virDomainStorageNetworkParseHost qemu: clean up qemuDomainRemoveInactiveCommon backup: Document nuances between different state capture APIs backup: Introduce virDomainCheckpointPtr wip: backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver wip: backup: Parse and output checkpoint XML wip: backup: Parse and output backup XML backup: Implement virsh support for checkpoints wip: backup: virsh support for backup backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs wip: backup: Wire up qemu checkpoint commands over QMP wip: backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP wip: backup: qemu: Wire up qemu full push backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/virterror.h | 6 +- src/util/virerror.c | 12 +- include/libvirt/libvirt-domain-checkpoint.h | 161 ++ include/libvirt/libvirt-domain.h | 48 +- include/libvirt/libvirt.h | 3 +- src/access/viraccessperm.h | 8 +- src/conf/checkpoint_conf.h | 237 +++ src/conf/domain_conf.h | 17 + src/conf/virdomainobjlist.h | 9 +- src/datatypes.h | 31 +- src/driver-hypervisor.h | 74 +- src/qemu/qemu_block.h | 3 + src/qemu/qemu_blockjob.h | 1 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.h | 32 +- src/qemu/qemu_monitor.h | 21 +- src/qemu/qemu_monitor_json.h | 19 +- tools/virsh-checkpoint.h | 29 + tools/virsh-completer.h | 4 + tools/virsh-util.h | 3 + tools/virsh.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 10 +- docs/domainstatecapture.html.in | 314 +++ docs/format.html.in | 2 + docs/formatbackup.html.in | 123 ++ docs/formatcheckpoint.html.in | 188 ++ docs/formatsnapshot.html.in | 2 + docs/index.html.in | 4 +- docs/schemas/domainbackup.rng | 185 ++ docs/schemas/domaincheckpoint.rng | 94 + examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 3 + src/Makefile.am | 2 + src/access/viraccessperm.c | 5 +- src/conf/Makefile.inc.am | 2 + src/conf/checkpoint_conf.c | 1743 +++++++++++++++++ src/conf/domain_conf.c | 73 +- src/conf/virdomainobjlist.c | 12 +- src/datatypes.c | 63 +- src/libvirt-domain-checkpoint.c | 741 +++++++ src/libvirt-domain.c | 217 +- src/libvirt_private.syms | 33 + src/libvirt_public.syms | 19 + src/qemu/qemu_block.c | 12 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_conf.c | 5 + src/qemu/qemu_domain.c | 276 ++- src/qemu/qemu_driver.c | 1661 +++++++++++++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_monitor.c | 65 +- src/qemu/qemu_monitor_json.c | 199 +- src/qemu/qemu_process.c | 7 + src/remote/remote_daemon_dispatch.c | 22 +- src/remote/remote_driver.c | 33 +- src/remote/remote_protocol.x | 239 ++- src/remote_protocol-structs | 129 ++ src/rpc/gendispatch.pl | 32 +- tests/Makefile.am | 15 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 + tests/domaincheckpointxml2xmltest.c | 233 +++ .../caps_4.0.0.riscv32.xml | 2 + .../caps_4.0.0.riscv64.xml | 2 + tests/qemumonitorjsontest.c | 2 +- tests/virschematest.c | 4 + tools/Makefile.am | 3 +- tools/virsh-checkpoint.c | 1377 +++++++++++++ tools/virsh-completer.c | 52 +- tools/virsh-domain-monitor.c | 25 +- tools/virsh-domain.c | 277 ++- tools/virsh-util.c | 11 + tools/virsh.c | 2 + tools/virsh.pod | 272 ++- 86 files changed, 9510 insertions(+), 110 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/conf/checkpoint_conf.h create mode 100644 tools/virsh-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatbackup.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/conf/checkpoint_conf.c create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml create mode 100644 tests/domaincheckpointxml2xmltest.c create mode 100644 tools/virsh-checkpoint.c -- 2.20.1 -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
