[libvirt RFCv10 00/32] multifd save restore prototype

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This is v10 of the multifd save prototype, which improves the
DIRECT I/O APIs and makes changes to the on disk format
backward and forward compatible, avoiding the need to alter
the QEMU_SAVE_VERSION.

This series demonstrates the DIRECT I/O friendly read/write code
working by using the block-friently multifd streams supplied by
the multifd helper, but could also work with block-aware QEMU streams
if ever available in the future.

KNOWN ISSUES:

a) still applies only to save/restore (no managed save etc)
b) multifd: saves to multiple files instead of just one file

---

changes from v9:

* exposed virFileDirectAlign

* separated the >= 2 QEMU_SAVE_VERSION change in own patch

* reworked the write code to add the alignment padding to the
  data_len, making the on disk format compatible when loaded
  from an older libvirt.

* reworked the read code to use direct I/O APIs only for actual
  direct I/O file descriptors, so as to make old images work
  with newer libvirt.

---

changes from v8:

* rebased on master

* reordered patches to add more upstreamable content at the start

* split introduction of virQEMUSaveFd, so the first part is multifd-free

* new virQEMUSaveDataRead as a mirror of virQEMUSaveDataWrite

* introduced virFileDirect API, using it in virFileDisk operations and
  for virQEMUSaveRead and virQEMUSaveWrite

---

changes from v7:

* [ base params API and iohelper refactoring upstreamed ]

* extended the QEMU save image format more, to record the nr
  of multifd channels on save. Made the data header struct packed.

* removed --parallel-connections from the restore command, as now
  it is useless due to QEMU save image format extension.

* separate out patches to expose migration_params APIs to saveimage,
  including qemuMigrationParamsSetString, SetCap, SetInt.

* fixed bugs in the ImageOpen patch (missing saveFd init), removed
  some whitespace, and fixed some convoluted code paths for return
  value -3.

---

changes from v6:

* improved error path handling, with error messages and especially
  cancellation of qemu process on error during restore.

* split patches more and reordered them to keep general refactoring
  at the beginning before the --parallel stuff is introduced.

* improved multifd compression support, including adding an enum
  and extending the QEMU save image format to record the compression
  used on save, and pick it up automatically on restore.

---

changes from v4:

* runIO renamed to virFileDiskCopy and rethought arguments

* renamed new APIs from ...ParametersFlags to ...Params

* introduce the new virDomainSaveParams and virDomainRestoreParams
  without any additional parameters, so they can be upstreamed first.

* solved the issue in the gendispatch.pl script generating code that
  was missing the conn parameter.

---

changes from v3:

* reordered series to have all helper-related change at the start

* solved all reported issues from ninja test, including documentation

* fixed most broken migration capabilities code (still imperfect likely)

* added G_GNUC_UNUSED as needed

* after multifd restore, added what I think were the missing operations:

  qemuProcessRefreshState(),
  qemuProcessStartCPUs() - most importantly,
  virDomainObjSave()

  The domain now starts running after restore without further encouragement

* removed the sleep(10) from the multifd-helper

---

changes from v2:

* added ability to restore the VM from disk using multifd

* fixed the multifd-helper to work in both directions,
  assuming the need to listen for save, and connect for restore.

* fixed a large number of bugs, and probably introduced some :-)

---

Claudio Fontana (32):
  virfile: introduce virFileDirect APIs
  virfile: use virFileDirect API in runIOCopy
  qemu: saveimage: rework image read/write to be O_DIRECT friendly
  qemu: saveimage: assume future formats will also support compression
  virfile: virFileDiskCopy: prepare for O_DIRECT files without wrapper
  qemu: saveimage: introduce virQEMUSaveFd
  qemu: saveimage: convert qemuSaveImageCreate to use virQEMUSaveFd
  qemu: saveimage: convert qemuSaveImageOpen to use virQEMUSaveFd
  tools: prepare doSave to use parameters
  tools: prepare cmdRestore to use parameters
  libvirt: add new VIR_DOMAIN_SAVE_PARALLEL flag and parameter
  qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in save
  qemu: add stub support for VIR_DOMAIN_SAVE_PARALLEL in restore
  multifd-helper: new helper for parallel save/restore
  qemu: saveimage: add virQEMUSaveFd APIs for multifd
  qemu: saveimage: wire up saveimage code with the multifd helper
  qemu: capabilities: add multifd to the probed migration capabilities
  qemu: saveimage: add multifd related fields to save format
  qemu: migration_params: add APIs to set Int and Cap
  qemu: migration: implement qemuMigrationSrcToFilesMultiFd for save
  qemu: add parameter to qemuMigrationDstRun to skip waiting
  qemu: implement qemuSaveImageLoadMultiFd for restore
  tools: add parallel parameter to virsh save command
  tools: add parallel parameter to virsh restore command
  qemu: add migration parameter multifd-compression
  libvirt: add new VIR_DOMAIN_SAVE_PARAM_PARALLEL_COMPRESSION
  qemu: saveimage: add parallel compression argument to ImageCreate
  qemu: saveimage: add stub support for multifd compression parameter
  qemu: migration: expose qemuMigrationParamsSetString
  qemu: saveimage: implement multifd-compression in parallel save
  qemu: saveimage: restore compressed parallel images
  tools: add parallel-compression parameter to virsh save command

 docs/manpages/virsh.rst                       |  39 +-
 include/libvirt/libvirt-domain.h              |  29 +
 po/POTFILES.in                                |   1 +
 src/libvirt_private.syms                      |   9 +
 src/qemu/qemu_capabilities.c                  |   4 +
 src/qemu/qemu_capabilities.h                  |   2 +
 src/qemu/qemu_driver.c                        | 146 ++--
 src/qemu/qemu_migration.c                     | 160 ++--
 src/qemu/qemu_migration.h                     |  16 +-
 src/qemu/qemu_migration_params.c              |  71 +-
 src/qemu/qemu_migration_params.h              |  15 +
 src/qemu/qemu_process.c                       |   3 +-
 src/qemu/qemu_process.h                       |   5 +-
 src/qemu/qemu_saveimage.c                     | 793 ++++++++++++++----
 src/qemu/qemu_saveimage.h                     |  71 +-
 src/qemu/qemu_snapshot.c                      |   6 +-
 src/util/meson.build                          |  16 +
 src/util/multifd-helper.c                     | 247 ++++++
 src/util/virfile.c                            | 316 ++++++-
 src/util/virfile.h                            |  11 +
 src/util/virthread.c                          |   5 +
 src/util/virthread.h                          |   1 +
 .../caps_4.0.0.aarch64.xml                    |   1 +
 .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml |   1 +
 .../caps_4.0.0.riscv32.xml                    |   1 +
 .../caps_4.0.0.riscv64.xml                    |   1 +
 .../qemucapabilitiesdata/caps_4.0.0.s390x.xml |   1 +
 .../caps_4.0.0.x86_64.xml                     |   1 +
 .../caps_4.1.0.x86_64.xml                     |   1 +
 .../caps_4.2.0.aarch64.xml                    |   1 +
 .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml |   1 +
 .../qemucapabilitiesdata/caps_4.2.0.s390x.xml |   1 +
 .../caps_4.2.0.x86_64.xml                     |   1 +
 .../caps_5.0.0.aarch64.xml                    |   2 +
 .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml |   2 +
 .../caps_5.0.0.riscv64.xml                    |   2 +
 .../caps_5.0.0.x86_64.xml                     |   2 +
 .../qemucapabilitiesdata/caps_5.1.0.sparc.xml |   2 +
 .../caps_5.1.0.x86_64.xml                     |   2 +
 .../caps_5.2.0.aarch64.xml                    |   2 +
 .../qemucapabilitiesdata/caps_5.2.0.ppc64.xml |   2 +
 .../caps_5.2.0.riscv64.xml                    |   2 +
 .../qemucapabilitiesdata/caps_5.2.0.s390x.xml |   2 +
 .../caps_5.2.0.x86_64.xml                     |   2 +
 .../caps_6.0.0.aarch64.xml                    |   2 +
 .../qemucapabilitiesdata/caps_6.0.0.s390x.xml |   2 +
 .../caps_6.0.0.x86_64.xml                     |   2 +
 .../caps_6.1.0.x86_64.xml                     |   2 +
 .../caps_6.2.0.aarch64.xml                    |   2 +
 .../qemucapabilitiesdata/caps_6.2.0.ppc64.xml |   2 +
 .../caps_6.2.0.x86_64.xml                     |   2 +
 .../caps_7.0.0.aarch64.xml                    |   2 +
 .../qemucapabilitiesdata/caps_7.0.0.ppc64.xml |   2 +
 .../caps_7.0.0.x86_64.xml                     |   2 +
 .../caps_7.1.0.x86_64.xml                     |   2 +
 tools/virsh-domain.c                          | 101 ++-
 56 files changed, 1718 insertions(+), 404 deletions(-)
 create mode 100644 src/util/multifd-helper.c

-- 
2.35.3





[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux