On Fri, May 27, 2022 at 10:47:58 +0100, Daniel P. Berrangé wrote:
> Libvirt provides QMP passthrough APIs for the QEMU driver and these are
> exposed in virsh. It is not especially pleasant, however, using the raw
> QMP JSON syntax. QEMU has a tool 'qmp-shell' which can speak QMP and
> exposes a human friendly interactive shell. It is not possible to use
> this with libvirt managed guest, however, since only one client can
> attach to he QMP socket at any point in time.
>
> The virt-qmp-proxy tool aims to solve this problem. It opens a UNIX
> socket and listens for incoming client connections, speaking QMP on
> the connected socket. It will forward any QMP commands received onto
> the running libvirt QEMU guest, and forward any replies back to the
> QMP client.
>
> $ virsh start demo
> $ virt-qmp-proxy demo demo.qmp &
> $ qmp-shell demo.qmp
> Welcome to the QMP low-level shell!
> Connected to QEMU 6.2.0
>
> (QEMU) query-kvm
> {
> "return": {
> "enabled": true,
> "present": true
> }
> }
>
> Note this tool of course has the same risks as the raw libvirt
> QMP passthrough. It is safe to run query commands to fetch information
> but commands which change the QEMU state risk disrupting libvirt's
> management of QEMU, potentially resulting in data loss/corruption in
> the worst case.
>
> Signed-off-by: Daniel P. Berrangé <berrange@xxxxxxxxxx>
> ---
>
> CC'ing QEMU since this is likely of interest to maintainers and users
> who work with QEMU and libvirt
>
> Note this impl is fairly crude in that it assumes it is receiving
> the QMP commands linewise one at a time. None the less it is good
> enough to work with qmp-shell already, so I figured it was worth
> exposing to the world. It also lacks support for forwarding events
> back to the QMP client.
>
> docs/manpages/meson.build | 1 +
> docs/manpages/virt-qmp-proxy.rst | 123 ++++++++++++++++++++++++++++
> tools/meson.build | 5 ++
> tools/virt-qmp-proxy | 133 +++++++++++++++++++++++++++++++
> 4 files changed, 262 insertions(+)
> create mode 100644 docs/manpages/virt-qmp-proxy.rst
> create mode 100755 tools/virt-qmp-proxy
[...]
> diff --git a/docs/manpages/virt-qmp-proxy.rst b/docs/manpages/virt-qmp-proxy.rst
> new file mode 100644
> index 0000000000..94679406ab
> --- /dev/null
> +++ b/docs/manpages/virt-qmp-proxy.rst
> @@ -0,0 +1,123 @@
> +==============
> +virt-qmp-proxy
> +==============
> +
> +--------------------------------------------------
> +Expose a QMP proxy server for a libvirt QEMU guest
> +--------------------------------------------------
> +
> +:Manual section: 1
> +:Manual group: Virtualization Support
> +
> +.. contents::
> +
> +
> +SYNOPSIS
> +========
> +
> +``virt-qmp-proxy`` [*OPTION*]... *DOMAIN* *QMP-SOCKET-PATH*
> +
> +
> +DESCRIPTION
> +===========
> +
> +This tool provides a way to expose a QMP proxy server that communicates
> +with a QEMU guest managed by libvirt. This enables standard QMP client
> +tools to interact with libvirt managed guests.
> +
> +**NOTE: use of this tool will result in the running QEMU guest being
> +marked as tainted.** It is strongly recommended that this tool *only be
> +used to send commands which query information* about the running guest.
> +If this tool is used to make changes to the state of the guest, this
> +may have negative interactions with the QEMU driver, resulting in an
> +inability to manage the guest operation thereafter, and in the worst
> +case **potentially lead to data loss or corruption**.
> +
> +The ``virt-qmp-proxy`` program will listen on a UNIX socket for incoming
> +client connections, and run the QMP protocol over the connection. Any
> +commands received will be sent to the running libvirt guest, and replies
> +sent back.
> +
> +The ``virt-qemu-proxy`` program may be interrupted (eg Ctrl-C) when it
> +is no longer required. The libvirt QEMU guest will continue running.
> +
> +
> +OPTIONS
> +=======
> +
> +*DOMAIN*
> +
> +The ID or UUID or Name of the libvirt QEMU guest.
> +
> +*QMP-SOCKET-PATH*
> +
> +The filesystem path at which to run the QMP server, listening for
> +incoming connections.
> +
> +``-c`` *CONNECTION-URI*
> +``--connect``\ =\ *CONNECTION-URI*
> +
> +The URI for the connection to the libvirt QEMU driver. If omitted,
> +a URI will be auto-detected.
> +
> +``-v``, ``--verbose``
> +
> +Run in verbose mode, printing all QMP commands and replies that
> +are handled.
> +
> +``-h``, ``--help``
> +
> +Display the command line help.
> +
> +
> +EXIT STATUS
> +===========
> +
> +Upon successful shutdown, an exit status of 0 will be set. Upon
> +failure a non-zero status will be set.
> +
> +
> +AUTHOR
> +======
> +
> +Daniel P. Berrangé
> +
> +
> +BUGS
> +====
> +
> +Please report all bugs you discover. This should be done via either:
> +
> +#. the mailing list
> +
> + `https://libvirt.org/contact.html <https://libvirt.org/contact.html>`_
> +
> +#. the bug tracker
> +
> + `https://libvirt.org/bugs.html <https://libvirt.org/bugs.html>`_
> +
> +Alternatively, you may report bugs to your software distributor / vendor.
> +
> +NOTE: at this time there is no support for forwarding QMP events back
> +to the clients
Also add caveat about FD passing support.
[...]
> diff --git a/tools/virt-qmp-proxy b/tools/virt-qmp-proxy
> new file mode 100755
> index 0000000000..57f9759fab
> --- /dev/null
> +++ b/tools/virt-qmp-proxy
> @@ -0,0 +1,133 @@
> +#!/usr/bin/env python3
> +
> +import argparse
> +import libvirt
> +import libvirt_qemu
> +import os
> +import re
> +import socket
> +import sys
> +import json
> +
> +
> +def get_domain(uri, domstr):
> + conn = libvirt.open(uri)
> +
> + dom = None
> + if re.match(r'^\d+$', domstr):
> + dom = conn.lookupByID(int(domstr))
> + elif re.match(r'^[+a-f0-9]+$', domstr):
This works very poorly if you have a VM named for example 'cd' or any
combination of just letters abcdef.
> + dom = conn.lookupByUUIDString(domstr)
> + else:
> + dom = conn.lookupByName(domstr)
> +
> + if not dom.isActive():
> + raise Exception(
> + "Domain must be running to validate measurement")
This should mention the current usage or a generic error ;)
> +
> + return conn, dom
> +
> +
> +def qmp_server(conn, dom, client, verbose):
> + ver = conn.getVersion()
So this gets the version of the "default" emulator version, but if your
VM is using a custom one this will report it wrong.
E.g in my case I have a git qemu for a VM:
517 2022-05-27 14:01:09.604+0000: 373562: debug : qemuMonitorJSONIOProcessLine:199 : Line [{"QMP": {"version": {"qemu": {"micro": 50, "minor": 0, "major": 7}, "package": "v7.0.0-1253-g2417cbd591"}, "capabilities": ["oob"]}}]
> + major = int(ver / 1000000) % 1000
> + minor = int(ver / 1000) % 1000
> + micro = ver % 1000
> +
> + greetingobj = {
> + "QMP": {
> + "version": {
> + "qemu": {
> + "major": major,
> + "minor": minor,
> + "micro": micro,
> + },
> + "package": f"qemu-{major}.{minor}.{micro}",
> + },
> + "capabilities": [
> + "oob"
> + ],
> + }
> + }
But when I conect I get:
{"QMP": {"version": {"qemu": {"major": 7, "minor": 0, "micro": 0}, "package": "qemu-7.0.0"}, "capabilities": ["oob"]}}
At the very least this should be documented.
> + greeting = json.dumps(greetingobj) + "\r\n"
> + if verbose:
> + print(greeting, end='')
> + client.send(greeting.encode("utf-8"))
> +
> + while True:
> + # XXX shouldn't blindly assume this one read
> + # will fully capture one-and-only-one cmd
> + cmd = client.recv(1024).decode('utf8')
IIUC this limits the buffer to 1k max. Libvirt's RPC supports up to 4M.
1k could be limiting with some commands such as blockdev-add.
> + if verbose:
> + print(cmd)
> +
> + if cmd == "":
> + break
> +
> + if "qmp_capabilities" in cmd:
> + capabilitiesobj = {
> + "return": {},
> + }
> + capabilities = json.dumps(capabilitiesobj) + "\r\n"
> + if verbose:
> + print(capabilities, end='')
> + client.send(capabilities.encode("utf-8"))
> + continue
> +
> + id = None
> + if "id" in cmd:
> + id = cmd[id]
> +
> + res = libvirt_qemu.qemuMonitorCommand(dom, cmd, 0)
If 'cmd' is not JSON this breaks horribly:
$ tools/virt-qmp-proxy 2 /tmp/asdf
libvirt: error : internal error: cannot parse json test
: lexical error: invalid string in json text.
test
(right here) ------^
tools/virt-qmp-proxy: internal error: cannot parse json test
: lexical error: invalid string in json text.
test
(right here) ------^
and stops working, while real qemu behaves differently:
$ qemu-system-x86_64 -qmp stdio
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 7}, "package": "qemu-7.0.0-2.fc35"}, "capabilities": ["oob"]}}
help
{"error": {"class": "GenericError", "desc": "JSON parse error, invalid keyword 'help'"}}
Also since it's just a simple loop without event handling from qemu.
E.g. if I destroy the VM while it's running it simply waits. When I
issue another command, then the proxy exits:
$ tools/virt-qmp-proxy 2 /tmp/asdf
libvirt: Domain Config error : Requested operation is not valid: domain is not running
tools/virt-qmp-proxy: Requested operation is not valid: domain is not running
but the client itself just sees a closed socket.
Given the use case it's not a big problem but it should be at least
mentioned in the docs.
> +
> + resobj = json.loads(res)
> + del resobj["id"]
> + if id is not None:
> + resobj["id"] = id
> + res = json.dumps(resobj) + "\r\n"
> + if verbose:
> + print(res, end='')
> +
> + client.send(res.encode('utf8'))
> +
> +
> +def parse_commandline():
> + parser = argparse.ArgumentParser(description="Libvirt QMP proxy")
> + parser.add_argument("--connect", "-c",
> + help="Libvirt QEMU driver connection URI")
> + parser.add_argument("--verbose", "-v", action='store_true',
> + help="Display QMP traffic")
> + parser.add_argument("domain", metavar="DOMAIN",
> + help="Libvirt guest domain ID/UUID/Name")
> + parser.add_argument("sockpath", metavar="QMP-SOCK-PATH",
> + help="UNIX socket path for QMP server")
> +
> + return parser.parse_args()
> +
> +
> +def main():
> + args = parse_commandline()
> +
> + conn, dom = get_domain(args.connect, args.domain)
> +
> + if conn.getType() != "QEMU":
> + raise Exception("QMP proxy requires a QEMU driver connection not %s" %
> + conn.getType())
> +
> + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
> + if os.path.exists(args.sockpath):
> + os.unlink(args.sockpath)
> + sock.bind(args.sockpath)
> + sock.listen(1)
> +
> + while True:
> + client, peeraddr = sock.accept()
> + qmp_server(conn, dom, client, args.verbose)
> +
> +
> +try:
> + main()
> + sys.exit(0)
> +except Exception as e:
> + print("%s: %s" % (sys.argv[0], str(e)))
> + sys.exit(1)
> --
> 2.36.1
>
