Hi guys, I've been working on adding support for nbdkit to libvirt for network storage sources like http and ssh. See https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information, but the summary is that RHEL does not want to ship the qemu storage plugins for curl and ssh. Handling them outside of the qemu process provides several advantages such as reduced attack surface and stability. I have something that works for me, but as I have not dealt with the storage stuff much before, I have a feeling that I'm missing some things. A quick summary of the code: - at startup I query to see whether nbdkit exists on the host and if so, I query which plugins/filters are installed. This is stored as qemuNbdkitCaps on the qemu driver - When the driver prepares the domain, we go through each disk source and determine whether the nbdkit capabilities allow us to support this disk via nbdkit, and if so, we allocate a qemuNbdkitProcess object and stash it in the private data of the virStorageSource. - The presence or absence of this qemuNbdkitProcess data then indicates whether this disk will be served to qemu indirectly via nbdkit or not - When we launch the qemuProcess, as part of the "external device start" step, I launch a ndkit process for each disk that is supported by nbdkit. I also optionally fork a child process to communicate authentication details and cookies to the nbdkit process via a unix socket. - for devices which are served by the ndkit process, I change the qemu commandline in the following ways: - I no longer pass auth/cookie secrets to qemu (those are handled by nbdkit) - I replace the actual network URL of the remote disk source with the path to the nbdkit unix socket Known shortcomings - I don't yet re-query for nbdkit / nbdkit caps, so need to restart libvirt to pick up newly-installed nbdkit or additional capabilities - testing is pretty limited at the moment - selinux not working yet - creating disks isn't supported, though Rich has added some support for that upstream in the nbdkit ssh plugin. I'd appreciate feedback on what i've got so far. Jonathon Jongsma (3): docs: clarify 'readahead' and 'timeout' for disks schema: Be more flexible for diskSourceNetworkProtocolPropsCommon WIP: use nbdkit for remote disk sources docs/formatdomain.rst | 10 +- include/libvirt/virterror.h | 1 + po/POTFILES | 1 + src/conf/schemas/domaincommon.rng | 34 +- src/qemu/meson.build | 1 + src/qemu/qemu_block.c | 64 +- src/qemu/qemu_block.h | 1 + src/qemu/qemu_command.c | 26 +- src/qemu/qemu_conf.c | 19 + src/qemu/qemu_conf.h | 5 + src/qemu/qemu_domain.c | 110 ++- src/qemu/qemu_domain.h | 5 + src/qemu/qemu_driver.c | 4 +- src/qemu/qemu_extdevice.c | 25 + src/qemu/qemu_nbdkit.c | 629 ++++++++++++++++++ src/qemu/qemu_nbdkit.h | 89 +++ src/qemu/qemu_validate.c | 22 +- src/qemu/qemu_validate.h | 4 +- src/util/virerror.c | 1 + tests/qemublocktest.c | 8 +- tests/qemustatusxml2xmldata/modern-in.xml | 1 - ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 ++ .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 45 ++ .../disk-network-http-nbdkit.xml | 1 + ...work-source-curl-nbdkit.x86_64-latest.args | 49 ++ .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 53 ++ .../disk-network-source-curl.xml | 71 ++ tests/qemuxml2argvtest.c | 12 + tests/testutilsqemu.c | 16 + tests/testutilsqemu.h | 4 + 32 files changed, 1302 insertions(+), 53 deletions(-) create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml -- 2.35.3
