From: Marc-André Lureau <marcandre.lureau@xxxxxxxxxx> This series adds support for running virtio GPUs in seperate processes, thanks to vhost-user backend. The QEMU support landed for 4.1. There are several benefits of running the GPU/virgl in an external process, since Mesa is rather heavy on the qemu main loop, and may block for a while, or crash. The external GPU process is started with one end of a socket pair, the other end is given to a QEMU chardev attached to a device. The external process is also added to the cgroup to limit resources usage. vhost-user is a generic mechanism that allows implementing virtio device dataplane handling in a separate userspace process. vhost-user-gpu here notably moves the virgl 3d handling out of the main qemu process. The external process will be /usr/libexec/vhost-user-gpu, which comes from qemu.git contrib/vhost-user-gpu code, first released in qemu-4.1. Part of this series deals with discovering the location on disk of the vhost-user-gpu binary, and what capabilities it provides. This uses a similar mechanism to firmware.json, described in qemu docs/interop/vhost-user.json https://github.com/qemu/qemu/blob/master/docs/interop/vhost-user.json qemu 4.1 ships a 50-qemu-gpu.json to match. I believe virtio-fs will use a similar mechanism when it lands in upstream qemu, as virtiofsd is a separate process that communicates with qemu over vhost-user. For a bit more background on vhost-user-gpu process handling and the json interop motivation, here's some of the qemu discussion: https://lists.nongnu.org/archive/html/qemu-devel/2018-08/msg02610.html https://lists.nongnu.org/archive/html/qemu-devel/2018-09/msg00807.html For this series, the XML to enable this is: <video model='virtio'> <driver name='vhostuser'/> <acceleration accel3d='yes' rendernode='/path/to/rendernode'/> </video> rendernode is optional qemu_vhost_user.c handles vhost-user.json qemu_vhost_user_gpu.c handles the process management for vhost-user-gpu v5: addressing v4 reviews - replaced "util: ignore EACCESS in virDirOpenIfExists" with more specific "qemu-interop: ignore non-readable directories" - use virFileSanitizePath & virBufferEscapeString on rendernode path - fix ->accel NULL crash, replace qemuSetupVideoAccelCgroup with qemuSetupVideoCgroup - fix src/vz virDomainDefAddImplicitVideo usage - few indent changes, rebased, added r-b tags v4: - rebased - simplify vhost-user-gpu pidfile checking - fix check/syntax-check v3: - rename qemu_configs -> qemu_interop_config - replace <model .. vhostuser='yes'/> with <driver name='vhostuser'/> - move vhost_user_binary to virDomainVideoDriverDef - some VIR_AUTO usage improvements - introduce qemuDomainVideoPrivate to store vhost-user fd - improved selection of -device model - use DO_TEST_CAPS_LATEST - allocate a rendernode with virHostGetDRMRenderNode() - but no clear idea how to have a test - add a patch to ignore EPERM in virDirOpenIfExists() - better domain checks/validate - added Ján r-b - no s-o-b from Cole, per request and commits taken from his git branch - rebase, indentation/style fixes, update doc, version.. v2: - rebase to master - if test file build by dropping LDADDS usage - syntax-check issues: * use #pragma once * if () bracket issues * jump label indent issues * error message %s usage * size_t for loops Marc-André Lureau (20): qemu: generalize qemuFetchConfigs qemu-interop: ignore non-readable directories conf: format/parse/rng/docs for video <driver name='qemu|vhostuser'/> domain: add rendernode attribute on <accel> qemu-cgroup: allow accel rendernode access qemu: add vhost-user-gpu capabilities checks qemu: check that qemu is vhost-user-vga capable qemu: validate virtio-gpu with vhost-user qemu: restrict 'virgl=' option to non-vhostuser video type qemu: add vhost-user helpers qemu: add qemuSecurityStartVhostUserGPU helper conf: add privateData to virDomainVideoDef qemu: add qemuDomainVideoPrivate qemu: add vhost-user-gpu helper unit tests: mock execv/execve tests: wrap vhost-user paths in qemuxml2argvtest qemu: prepare domain for vhost-user GPU qemu: start/stop the vhost-user-gpu external device qemu: build vhost-user GPU devices tests: add vhost-user-gpu xml2argv tests docs/formatdomain.html.in | 18 +- docs/schemas/domaincommon.rng | 13 + src/conf/domain_conf.c | 85 +++- src/conf/domain_conf.h | 22 +- src/qemu/Makefile.inc.am | 6 + src/qemu/qemu_capabilities.c | 6 + src/qemu/qemu_capabilities.h | 4 + src/qemu/qemu_cgroup.c | 28 ++ src/qemu/qemu_command.c | 65 ++- src/qemu/qemu_domain.c | 52 ++- src/qemu/qemu_domain.h | 12 + src/qemu/qemu_extdevice.c | 74 ++- src/qemu/qemu_extdevice.h | 5 + src/qemu/qemu_firmware.c | 144 +----- src/qemu/qemu_interop_config.c | 189 ++++++++ src/qemu/qemu_interop_config.h | 25 ++ src/qemu/qemu_process.c | 61 ++- src/qemu/qemu_security.c | 40 ++ src/qemu/qemu_security.h | 6 + src/qemu/qemu_vhost_user.c | 422 ++++++++++++++++++ src/qemu/qemu_vhost_user.h | 48 ++ src/qemu/qemu_vhost_user_gpu.c | 275 ++++++++++++ src/qemu/qemu_vhost_user_gpu.h | 49 ++ src/vz/vz_sdk.c | 2 +- tests/Makefile.am | 9 + .../caps_4.1.0.x86_64.xml | 2 + .../etc/qemu/vhost-user/40-gpu.json | 1 + .../etc/qemu/vhost-user/50-gpu.json | 0 .../qemu/vhost-user/test-vhost-user-gpu | 11 + .../usr/share/qemu/vhost-user/30-gpu.json | 1 + .../usr/share/qemu/vhost-user/50-gpu.json | 8 + .../usr/share/qemu/vhost-user/60-gpu.json | 1 + tests/qemuvhostusertest.c | 132 ++++++ ...host-user-gpu-secondary.x86_64-latest.args | 43 ++ .../vhost-user-gpu-secondary.xml | 46 ++ .../vhost-user-vga.x86_64-latest.args | 40 ++ tests/qemuxml2argvdata/vhost-user-vga.xml | 42 ++ tests/qemuxml2argvdata/virtio-options.args | 5 +- tests/qemuxml2argvdata/virtio-options.xml | 4 +- tests/qemuxml2argvtest.c | 16 +- tests/virfilewrapper.c | 22 + 41 files changed, 1828 insertions(+), 206 deletions(-) create mode 100644 src/qemu/qemu_interop_config.c create mode 100644 src/qemu/qemu_interop_config.h create mode 100644 src/qemu/qemu_vhost_user.c create mode 100644 src/qemu/qemu_vhost_user.h create mode 100644 src/qemu/qemu_vhost_user_gpu.c create mode 100644 src/qemu/qemu_vhost_user_gpu.h create mode 120000 tests/qemuvhostuserdata/etc/qemu/vhost-user/40-gpu.json create mode 100644 tests/qemuvhostuserdata/etc/qemu/vhost-user/50-gpu.json create mode 100755 tests/qemuvhostuserdata/usr/libexec/qemu/vhost-user/test-vhost-user-gpu create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/30-gpu.json create mode 100644 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/50-gpu.json create mode 120000 tests/qemuvhostuserdata/usr/share/qemu/vhost-user/60-gpu.json create mode 100644 tests/qemuvhostusertest.c create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-gpu-secondary.xml create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-vga.xml -- 2.23.0 -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
