After a bit of a lengthy delay, this is the second version of this patch series. See https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information about the goal, but the summary is that RHEL does not want to ship the qemu storage plugins for curl and ssh. Handling them outside of the qemu process provides several advantages such as reduced attack surface and stability. A quick summary of the code: - at startup I query to see whether nbdkit exists on the host and if so, I query which plugins/filters are installed. These capabilities are cached and stored in the qemu driver - When the driver prepares the domain, we go through each disk source and determine whether the nbdkit capabilities allow us to support this disk via nbdkit, and if so, we allocate a qemuNbdkitProcess object and stash it in the private data of the virStorageSource. - The presence or absence of this qemuNbdkitProcess data then indicates whether this disk will be served to qemu indirectly via nbdkit or directly - When we launch the qemuProcess, as part of the "external device start" step, I launch a ndkit process for each disk that is supported by nbdkit. - for devices which are served by an intermediate ndkit process, I change the qemu commandline in the following ways: - I no longer pass auth/cookie secrets to qemu (those are handled by nbdkit) - I replace the actual network URL of the remote disk source with the path to the nbdkit unix socket Open questions - selinux: I need some help from people more familiar with selinux to figure out what is needed here. When selinux is enforcing, I get a failure to launch nbdkit to serve the disks. I suspect we need a new context and policy for /usr/sbin/nbdkit that allows it to transition to the appropriate selinux context. The current context (on fedora) is "system_u:object_r:bin_t:s0". When I (temporarily) change the context to something like qemu_exec_t, I am able to start nbdkit and the domain launches. Known shortcomings - creating disks (in ssh) still isn't supported. I wanted to send out the patch series anyway since it's been delayed too long already. Changes since v1: - split into multiple patches - added a build option for nbdkit_moddir - don't instantiate any secret / cookie props for disks that are being served by nbdkit since we don't send secrets to qemu anymore - ensure that nbdkit processes are started/stopped for the entire backing chain - switch to virFileCache-based capabilities for nbdkit so that we don't need to requery every time - switch to using pipes for communicating sensitive data to nbdkit - use pidfile support built into virCommand rather than nbdkit's --pidfile argument - added significantly more tests Jonathon Jongsma (16): schema: allow 'ssh' as a protocol for network disks qemu: Add qemuNbdkitCaps to qemu driver qemu: expand nbdkit capabilities util: Allow virFileCache data to be any GObject qemu: implement basic virFileCache for nbdkit caps qemu: implement persistent file cache for nbdkit caps qemu: use file cache for nbdkit caps qemu: Add qemuNbdkitProcess qemu: add functions to start and stop nbdkit tests: add ability to test various nbdkit capabilities qemu: split qemuDomainSecretStorageSourcePrepare qemu: use nbdkit to serve network disks if available qemu: include nbdkit state in private xml tests: add tests for nbdkit invocation qemu: pass sensitive data to nbdkit via pipe qemu: add test for authenticating a https network disk build-aux/syntax-check.mk | 4 +- docs/formatdomain.rst | 2 +- meson.build | 6 + meson_options.txt | 1 + po/POTFILES | 1 + src/conf/schemas/domaincommon.rng | 1 + src/qemu/meson.build | 1 + src/qemu/qemu_block.c | 168 ++- src/qemu/qemu_command.c | 4 +- src/qemu/qemu_conf.c | 22 + src/qemu/qemu_conf.h | 6 + src/qemu/qemu_domain.c | 176 ++- src/qemu/qemu_domain.h | 4 + src/qemu/qemu_driver.c | 3 + src/qemu/qemu_extdevice.c | 84 ++ src/qemu/qemu_nbdkit.c | 1051 +++++++++++++++++ src/qemu/qemu_nbdkit.h | 90 ++ src/qemu/qemu_nbdkitpriv.h | 46 + src/util/virfilecache.c | 15 +- src/util/virfilecache.h | 2 +- src/util/virutil.h | 2 +- tests/meson.build | 1 + .../disk-cdrom-network.args.disk0 | 7 + .../disk-cdrom-network.args.disk1 | 9 + .../disk-cdrom-network.args.disk1.pipe.45 | 1 + .../disk-cdrom-network.args.disk2 | 9 + .../disk-cdrom-network.args.disk2.pipe.47 | 1 + .../disk-network-http.args.disk0 | 7 + .../disk-network-http.args.disk1 | 6 + .../disk-network-http.args.disk2 | 7 + .../disk-network-http.args.disk2.pipe.45 | 1 + .../disk-network-http.args.disk3 | 8 + .../disk-network-http.args.disk3.pipe.47 | 1 + ...work-source-curl-nbdkit-backing.args.disk0 | 8 + ...rce-curl-nbdkit-backing.args.disk0.pipe.45 | 1 + .../disk-network-source-curl.args.1.pipe.1 | 1 + .../disk-network-source-curl.args.disk0 | 8 + ...isk-network-source-curl.args.disk0.pipe.45 | 1 + .../disk-network-source-curl.args.disk1 | 10 + ...isk-network-source-curl.args.disk1.pipe.47 | 1 + ...isk-network-source-curl.args.disk1.pipe.49 | 1 + .../disk-network-source-curl.args.disk2 | 8 + ...isk-network-source-curl.args.disk2.pipe.49 | 1 + ...isk-network-source-curl.args.disk2.pipe.51 | 1 + .../disk-network-source-curl.args.disk3 | 7 + .../disk-network-source-curl.args.disk4 | 7 + .../disk-network-ssh.args.disk0 | 7 + tests/qemunbdkittest.c | 271 +++++ ...sk-cdrom-network-nbdkit.x86_64-latest.args | 42 + .../disk-cdrom-network-nbdkit.xml | 1 + ...isk-network-http-nbdkit.x86_64-latest.args | 45 + .../disk-network-http-nbdkit.xml | 1 + ...rce-curl-nbdkit-backing.x86_64-latest.args | 38 + ...isk-network-source-curl-nbdkit-backing.xml | 45 + ...work-source-curl-nbdkit.x86_64-latest.args | 50 + .../disk-network-source-curl-nbdkit.xml | 1 + ...isk-network-source-curl.x86_64-latest.args | 54 + .../disk-network-source-curl.xml | 74 ++ ...disk-network-ssh-nbdkit.x86_64-latest.args | 36 + .../disk-network-ssh-nbdkit.xml | 1 + .../disk-network-ssh.x86_64-latest.args | 36 + tests/qemuxml2argvdata/disk-network-ssh.xml | 31 + tests/qemuxml2argvtest.c | 18 + tests/testutilsqemu.c | 27 + tests/testutilsqemu.h | 5 + 65 files changed, 2474 insertions(+), 111 deletions(-) create mode 100644 src/qemu/qemu_nbdkit.c create mode 100644 src/qemu/qemu_nbdkit.h create mode 100644 src/qemu/qemu_nbdkitpriv.h create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.45 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.47 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.45 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.47 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.45 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.45 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.47 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.49 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.49 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.51 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0 create mode 100644 tests/qemunbdkittest.c create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml -- 2.37.1
