[libvirt PATCH v3 00/18] Use nbdkit for http/ftp/ssh network drives in libvirt

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This is the third version of this patch series. See
https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information about
the goal, but the summary is that RHEL does not want to ship the qemu storage
plugins for curl and ssh.  Handling them outside of the qemu process provides
several advantages such as reduced attack surface and stability.

A quick summary of the code:
 - at startup I query to see whether nbdkit exists on the host and if
   so, I query which plugins/filters are installed. These capabilities
   are cached and stored in the qemu driver
 - When the driver prepares the domain, we go through each disk source
   and determine whether the nbdkit capabilities allow us to support
   this disk via nbdkit, and if so, we allocate a qemuNbdkitProcess
   object and stash it in the private data of the virStorageSource.
 - The presence or absence of this qemuNbdkitProcess data then indicates
   whether this disk will be served to qemu indirectly via nbdkit or
   directly
 - When we launch the qemuProcess, as part of the "external device
   start" step, I launch a ndkit process for each disk that is supported
   by nbdkit.
 - for devices which are served by an intermediate ndkit process, I
   change the qemu commandline in the following ways:
   - I no longer pass auth/cookie secrets to qemu (those are handled by
     nbdkit)
   - I replace the actual network URL of the remote disk source with the
     path to the nbdkit unix socket
 - We create a 'monitor' for the nbdkit process that watches to see whether the
   process exits. If it does, we pause the domain, attempt to restart nbdkit,
   and then resume the domain.

Open questions
 - I think selinux will work once we add a policy for the /usr/sbin/nbdkit
   binary to allow it to be executed by libvirt, but for now it fails to
   execute nbdkit in enforcing mode. The current context for nbdkit (on fedora)
   is "system_u:object_r:bin_t:s0". When I temporarily change the context to
   something like qemu_exec_t, I am able to start nbdkit and the domain
   launches.

Known shortcomings
 - creating disks (in ssh) still isn't supported.

Changes in v3:
 - Various formatting fixes
 - Don't kill process in qemuNbdkitProcessFree() since we want the nbdkit
   daemon to continue running even if libvirt restarts.
 - Better detection and error reporting when starting the nbdkit process
 - Add monitoring for nbdkit process so that it can be restarted if if ever
   exits unexpectedly.

Changes in v2:
 - split into multiple patches
 - added a build option for nbdkit_moddir
 - don't instantiate any secret / cookie props for disks that are being served
   by nbdkit since we don't send secrets to qemu anymore
 - ensure that nbdkit processes are started/stopped for the entire backing
   chain
 - switch to virFileCache-based capabilities for nbdkit so that we don't need
   to requery every time
 - switch to using pipes for communicating sensitive data to nbdkit
 - use pidfile support built into virCommand rather than nbdkit's --pidfile
   argument
 - added significantly more tests

Jonathon Jongsma (18):
  schema: allow 'ssh' as a protocol for network disks
  qemu: Add functions for determining nbdkit availability
  qemu: expand nbdkit capabilities
  util: Allow virFileCache data to be any GObject
  qemu: implement basic virFileCache for nbdkit caps
  qemu: implement persistent file cache for nbdkit caps
  qemu: use file cache for nbdkit caps
  qemu: Add qemuNbdkitProcess
  qemu: add functions to start and stop nbdkit
  tests: add ability to test various nbdkit capabilities
  qemu: split qemuDomainSecretStorageSourcePrepare
  qemu: include nbdkit state in private xml
  qemu: use nbdkit to serve network disks if available
  tests: add tests for nbdkit invocation
  util: make virCommandSetSendBuffer testable
  qemu: pass sensitive data to nbdkit via pipe
  qemu: add test for authenticating a https network disk
  qemu: Monitor nbdkit process for exit

 build-aux/syntax-check.mk                     |    4 +-
 meson.build                                   |    9 +
 meson_options.txt                             |    1 +
 po/POTFILES                                   |    1 +
 src/conf/schemas/domaincommon.rng             |    1 +
 src/libvirt_private.syms                      |    1 +
 src/qemu/meson.build                          |    1 +
 src/qemu/qemu_block.c                         |  162 ++-
 src/qemu/qemu_conf.c                          |   23 +
 src/qemu/qemu_conf.h                          |    7 +
 src/qemu/qemu_domain.c                        |  180 ++-
 src/qemu/qemu_domain.h                        |    4 +
 src/qemu/qemu_driver.c                        |    4 +
 src/qemu/qemu_extdevice.c                     |   48 +
 src/qemu/qemu_nbdkit.c                        | 1243 +++++++++++++++++
 src/qemu/qemu_nbdkit.h                        |  120 ++
 src/qemu/qemu_nbdkitpriv.h                    |   31 +
 src/qemu/qemu_process.c                       |   13 +
 src/util/vircommand.c                         |   19 +-
 src/util/vircommand.h                         |    8 +
 src/util/vircommandpriv.h                     |    4 +
 src/util/virfilecache.c                       |   14 +-
 src/util/virfilecache.h                       |    2 +-
 src/util/virutil.h                            |    2 +-
 tests/meson.build                             |    1 +
 .../disk-cdrom-network.args.disk0             |    7 +
 .../disk-cdrom-network.args.disk1             |    9 +
 .../disk-cdrom-network.args.disk1.pipe.1778   |    1 +
 .../disk-cdrom-network.args.disk2             |    9 +
 .../disk-cdrom-network.args.disk2.pipe.1780   |    1 +
 .../disk-network-http.args.disk0              |    7 +
 .../disk-network-http.args.disk1              |    6 +
 .../disk-network-http.args.disk2              |    7 +
 .../disk-network-http.args.disk2.pipe.1778    |    1 +
 .../disk-network-http.args.disk3              |    8 +
 .../disk-network-http.args.disk3.pipe.1780    |    1 +
 ...work-source-curl-nbdkit-backing.args.disk0 |    8 +
 ...e-curl-nbdkit-backing.args.disk0.pipe.1778 |    1 +
 .../disk-network-source-curl.args.1.pipe.1    |    1 +
 .../disk-network-source-curl.args.disk0       |    8 +
 ...k-network-source-curl.args.disk0.pipe.1778 |    1 +
 .../disk-network-source-curl.args.disk1       |   10 +
 ...k-network-source-curl.args.disk1.pipe.1780 |    1 +
 ...k-network-source-curl.args.disk1.pipe.1782 |    1 +
 ...isk-network-source-curl.args.disk1.pipe.49 |    1 +
 .../disk-network-source-curl.args.disk2       |    8 +
 ...k-network-source-curl.args.disk2.pipe.1782 |    1 +
 ...k-network-source-curl.args.disk2.pipe.1784 |    1 +
 ...isk-network-source-curl.args.disk2.pipe.51 |    1 +
 .../disk-network-source-curl.args.disk3       |    7 +
 .../disk-network-source-curl.args.disk4       |    7 +
 .../disk-network-ssh.args.disk0               |    7 +
 tests/qemunbdkittest.c                        |  294 ++++
 ...sk-cdrom-network-nbdkit.x86_64-latest.args |   42 +
 .../disk-cdrom-network-nbdkit.xml             |    1 +
 ...isk-network-http-nbdkit.x86_64-latest.args |   45 +
 .../disk-network-http-nbdkit.xml              |    1 +
 ...rce-curl-nbdkit-backing.x86_64-latest.args |   38 +
 ...isk-network-source-curl-nbdkit-backing.xml |   45 +
 ...work-source-curl-nbdkit.x86_64-latest.args |   50 +
 .../disk-network-source-curl-nbdkit.xml       |    1 +
 ...isk-network-source-curl.x86_64-latest.args |   54 +
 .../disk-network-source-curl.xml              |   74 +
 ...disk-network-ssh-nbdkit.x86_64-latest.args |   36 +
 .../disk-network-ssh-nbdkit.xml               |    1 +
 .../disk-network-ssh.x86_64-latest.args       |   36 +
 tests/qemuxml2argvdata/disk-network-ssh.xml   |   31 +
 tests/qemuxml2argvtest.c                      |   18 +
 tests/testutilsqemu.c                         |   27 +
 tests/testutilsqemu.h                         |    5 +
 70 files changed, 2707 insertions(+), 116 deletions(-)
 create mode 100644 src/qemu/qemu_nbdkit.c
 create mode 100644 src/qemu/qemu_nbdkit.h
 create mode 100644 src/qemu/qemu_nbdkitpriv.h
 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk0
 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1
 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk1.pipe.1778
 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2
 create mode 100644 tests/qemunbdkitdata/disk-cdrom-network.args.disk2.pipe.1780
 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk0
 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk1
 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2
 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk2.pipe.1778
 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3
 create mode 100644 tests/qemunbdkitdata/disk-network-http.args.disk3.pipe.1780
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl-nbdkit-backing.args.disk0.pipe.1778
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.1.pipe.1
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk0.pipe.1778
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1780
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.1782
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk1.pipe.49
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.1782
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.1784
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk2.pipe.51
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk3
 create mode 100644 tests/qemunbdkitdata/disk-network-source-curl.args.disk4
 create mode 100644 tests/qemunbdkitdata/disk-network-ssh.args.disk0
 create mode 100644 tests/qemunbdkittest.c
 create mode 100644 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.x86_64-latest.args
 create mode 120000 tests/qemuxml2argvdata/disk-cdrom-network-nbdkit.xml
 create mode 100644 tests/qemuxml2argvdata/disk-network-http-nbdkit.x86_64-latest.args
 create mode 120000 tests/qemuxml2argvdata/disk-network-http-nbdkit.xml
 create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit-backing.xml
 create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.x86_64-latest.args
 create mode 120000 tests/qemuxml2argvdata/disk-network-source-curl-nbdkit.xml
 create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/disk-network-source-curl.xml
 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.x86_64-latest.args
 create mode 120000 tests/qemuxml2argvdata/disk-network-ssh-nbdkit.xml
 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/disk-network-ssh.xml

-- 
2.37.3




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux