Hi Daniel, will repost a v7 today with many more improvements, in terms of split and in terms of multifd-compression. Thanks! Claudio On 5/5/22 2:52 PM, Claudio Fontana wrote: > This a respin of the multifd save prototype, > rebased on post 8.3.0 release. > > There are still quite a few rough edges: > > KNOWN ISSUES: > > 1) this applies only to virsh save and virsh restore for now > (no managed save etc). > > 2) error handling is not good yet, especially during resume, > errors may leave behind a qemu process and such. > May need some help find all of these cases > > 3) the compression part is demonstrative only, there needs > to be more attention to compression options, and detecting > the compression used to store the multifd saves. > > ... > > changes from v4: > > * runIO renamed to virFileDiskCopy and rethought arguments > > * renamed new APIs from ...ParametersFlags to ...Params > > * introduce the new virDomainSaveParams and virDomainRestoreParams > without any additional parameters, so they can be upstreamed first. > > * solved the issue in the gendispatch.pl script generating code that > was missing the conn parameter. > > --- > > changes from v3: > > * reordered series to have all helper-related change at the start > > * solved all reported issues from ninja test, including documentation > > * fixed most broken migration capabilities code (still imperfect likely) > > * added G_GNUC_UNUSED as needed > > * after multifd restore, added what I think were the missing operations: > > qemuProcessRefreshState(), > qemuProcessStartCPUs() - most importantly, > virDomainObjSave() > > The domain now starts running after restore without further encouragement > > * removed the sleep(10) from the multifd-helper > > > changes from v2: > > * added ability to restore the VM from disk using multifd > > * fixed the multifd-helper to work in both directions, > assuming the need to listen for save, and connect for restore. > > * fixed a large number of bugs, and probably introduced some :-) > > > Thanks for your thoughts, > > Claudio > > > Claudio Fontana (27): > iohelper: introduce new struct to carry copy operation parameters > iohelper: refactor copy operation as a separate function > iohelper: move runIO function to virfile.c > virfile: rename runIO to virFileDiskCopy > virfile: change virFileDiskCopy arguments to extend beyond stdin, > stdout > virfile: add comment about the use of SEEK_END in virFileDiskCopy > multifd-helper: new helper for parallel save/restore > libvirt: introduce virDomainSaveParams public API > libvirt: introduce virDomainRestoreParams public API > remote: Add RPC support for the virDomainSaveParams API > gendispatch: add DomainRestoreParams as requiring conn argument > remote: Add RPC support for the virDomainRestoreParams API > qemu: add implementation for virDomainSaveParams API > qemu: add implementation for virDomainRestoreParams API > libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter > qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save > qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore > qemu: saveimage: introduce virQEMUSaveFd > qemu: wire up saveimage code with the multifd helper > qemu: capabilities: add multifd to the probed migration capabilities > qemu: implement qemuMigrationSrcToFilesMultiFd > qemu: add parameter to qemuMigrationDstRun to skip waiting > qemu: implement qemuSaveImageLoadMultiFd > tools: add parallel parameter to virsh save command > tools: add parallel parameter to virsh restore command > docs: update refs to virDomainSaveParams and virDomainRestoreParams > qemu: add migration parameter multifd-compression > > docs/formatsnapshot.rst | 5 +- > docs/manpages/virsh.rst | 34 +- > include/libvirt/libvirt-domain.h | 49 ++ > po/POTFILES.in | 1 + > src/driver-hypervisor.h | 14 + > src/libvirt-domain.c | 99 +++- > src/libvirt_private.syms | 1 + > src/libvirt_public.syms | 6 + > src/qemu/qemu_capabilities.c | 6 + > src/qemu/qemu_capabilities.h | 4 + > src/qemu/qemu_driver.c | 239 +++++++-- > src/qemu/qemu_migration.c | 155 ++++-- > src/qemu/qemu_migration.h | 16 +- > src/qemu/qemu_migration_params.c | 71 ++- > src/qemu/qemu_migration_params.h | 15 + > src/qemu/qemu_process.c | 3 +- > src/qemu/qemu_process.h | 5 +- > src/qemu/qemu_saveimage.c | 496 ++++++++++++++---- > src/qemu/qemu_saveimage.h | 49 +- > src/qemu/qemu_snapshot.c | 6 +- > src/remote/remote_driver.c | 2 + > src/remote/remote_protocol.x | 29 +- > src/remote_protocol-structs | 17 + > src/rpc/gendispatch.pl | 5 +- > src/util/iohelper.c | 162 +----- > src/util/meson.build | 19 + > src/util/multifd-helper.c | 249 +++++++++ > src/util/virfile.c | 218 ++++++++ > src/util/virfile.h | 2 + > src/util/virthread.c | 5 + > src/util/virthread.h | 1 + > .../caps_4.0.0.aarch64.xml | 1 + > .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + > .../caps_4.0.0.riscv32.xml | 1 + > .../caps_4.0.0.riscv64.xml | 1 + > .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 1 + > .../caps_4.0.0.x86_64.xml | 1 + > .../caps_4.1.0.x86_64.xml | 1 + > .../caps_4.2.0.aarch64.xml | 1 + > .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + > .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + > .../caps_4.2.0.x86_64.xml | 1 + > .../caps_5.0.0.aarch64.xml | 2 + > .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 + > .../caps_5.0.0.riscv64.xml | 2 + > .../caps_5.0.0.x86_64.xml | 2 + > .../qemucapabilitiesdata/caps_5.1.0.sparc.xml | 2 + > .../caps_5.1.0.x86_64.xml | 2 + > .../caps_5.2.0.aarch64.xml | 2 + > .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml | 2 + > .../caps_5.2.0.riscv64.xml | 2 + > .../qemucapabilitiesdata/caps_5.2.0.s390x.xml | 2 + > .../caps_5.2.0.x86_64.xml | 2 + > .../caps_6.0.0.aarch64.xml | 2 + > .../qemucapabilitiesdata/caps_6.0.0.s390x.xml | 2 + > .../caps_6.0.0.x86_64.xml | 2 + > .../caps_6.1.0.x86_64.xml | 2 + > .../caps_6.2.0.aarch64.xml | 2 + > .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml | 2 + > .../caps_6.2.0.x86_64.xml | 2 + > .../caps_7.0.0.aarch64.xml | 2 + > .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml | 2 + > .../caps_7.0.0.x86_64.xml | 2 + > tools/virsh-domain.c | 96 +++- > 64 files changed, 1686 insertions(+), 446 deletions(-) > create mode 100644 src/util/multifd-helper.c >
