Re: [libvirt PATCH] tools: add virt-qmp-proxy for proxying QMP clients to libvirt QEMU guests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, May 27, 2022 at 10:47:58 +0100, Daniel P. Berrangé wrote:
> Libvirt provides QMP passthrough APIs for the QEMU driver and these are
> exposed in virsh. It is not especially pleasant, however, using the raw
> QMP JSON syntax. QEMU has a tool 'qmp-shell' which can speak QMP and
> exposes a human friendly interactive shell. It is not possible to use
> this with libvirt managed guest, however, since only one client can
> attach to he QMP socket at any point in time.
> 
> The virt-qmp-proxy tool aims to solve this problem. It opens a UNIX
> socket and listens for incoming client connections, speaking QMP on
> the connected socket. It will forward any QMP commands received onto
> the running libvirt QEMU guest, and forward any replies back to the
> QMP client.
> 
>   $ virsh start demo
>   $ virt-qmp-proxy demo demo.qmp &
>   $ qmp-shell demo.qmp
>   Welcome to the QMP low-level shell!
>   Connected to QEMU 6.2.0
> 
>   (QEMU) query-kvm
>   {
>       "return": {
>           "enabled": true,
>           "present": true
>       }
>   }
> 
> Note this tool of course has the same risks as the raw libvirt
> QMP passthrough. It is safe to run query commands to fetch information
> but commands which change the QEMU state risk disrupting libvirt's
> management of QEMU, potentially resulting in data loss/corruption in
> the worst case.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@xxxxxxxxxx>
> ---
> 
> CC'ing QEMU since this is likely of interest to maintainers and users
> who work with QEMU and libvirt
> 
> Note this impl is fairly crude in that it assumes it is receiving
> the QMP commands linewise one at a time. None the less it is good
> enough to work with qmp-shell already, so I figured it was worth
> exposing to the world. It also lacks support for forwarding events
> back to the QMP client.
> 
>  docs/manpages/meson.build        |   1 +
>  docs/manpages/virt-qmp-proxy.rst | 123 ++++++++++++++++++++++++++++
>  tools/meson.build                |   5 ++
>  tools/virt-qmp-proxy             | 133 +++++++++++++++++++++++++++++++
>  4 files changed, 262 insertions(+)
>  create mode 100644 docs/manpages/virt-qmp-proxy.rst
>  create mode 100755 tools/virt-qmp-proxy

[...]

> diff --git a/docs/manpages/virt-qmp-proxy.rst b/docs/manpages/virt-qmp-proxy.rst
> new file mode 100644
> index 0000000000..94679406ab
> --- /dev/null
> +++ b/docs/manpages/virt-qmp-proxy.rst
> @@ -0,0 +1,123 @@
> +==============
> +virt-qmp-proxy
> +==============
> +
> +--------------------------------------------------
> +Expose a QMP proxy server for a libvirt QEMU guest
> +--------------------------------------------------
> +
> +:Manual section: 1
> +:Manual group: Virtualization Support
> +
> +.. contents::
> +
> +
> +SYNOPSIS
> +========
> +
> +``virt-qmp-proxy`` [*OPTION*]... *DOMAIN* *QMP-SOCKET-PATH*
> +
> +
> +DESCRIPTION
> +===========
> +
> +This tool provides a way to expose a QMP proxy server that communicates
> +with a QEMU guest managed by libvirt. This enables standard QMP client
> +tools to interact with libvirt managed guests.
> +
> +**NOTE: use of this tool will result in the running QEMU guest being
> +marked as tainted.** It is strongly recommended that this tool *only be
> +used to send commands which query information* about the running guest.
> +If this tool is used to make changes to the state of the guest, this
> +may have negative interactions with the QEMU driver, resulting in an
> +inability to manage the guest operation thereafter, and in the worst
> +case **potentially lead to data loss or corruption**.
> +
> +The ``virt-qmp-proxy`` program will listen on a UNIX socket for incoming
> +client connections, and run the QMP protocol over the connection. Any
> +commands received will be sent to the running libvirt guest, and replies
> +sent back.
> +
> +The ``virt-qemu-proxy`` program may be interrupted (eg Ctrl-C) when it
> +is no longer required. The libvirt QEMU guest will continue running.
> +
> +
> +OPTIONS
> +=======
> +
> +*DOMAIN*
> +
> +The ID or UUID or Name of the libvirt QEMU guest.
> +
> +*QMP-SOCKET-PATH*
> +
> +The filesystem path at which to run the QMP server, listening for
> +incoming connections.
> +
> +``-c`` *CONNECTION-URI*
> +``--connect``\ =\ *CONNECTION-URI*
> +
> +The URI for the connection to the libvirt QEMU driver. If omitted,
> +a URI will be auto-detected.
> +
> +``-v``, ``--verbose``
> +
> +Run in verbose mode, printing all QMP commands and replies that
> +are handled.
> +
> +``-h``, ``--help``
> +
> +Display the command line help.
> +
> +
> +EXIT STATUS
> +===========
> +
> +Upon successful shutdown, an exit status of 0 will be set. Upon
> +failure a non-zero status will be set.
> +
> +
> +AUTHOR
> +======
> +
> +Daniel P. Berrangé
> +
> +
> +BUGS
> +====
> +
> +Please report all bugs you discover.  This should be done via either:
> +
> +#. the mailing list
> +
> +   `https://libvirt.org/contact.html <https://libvirt.org/contact.html>`_
> +
> +#. the bug tracker
> +
> +   `https://libvirt.org/bugs.html <https://libvirt.org/bugs.html>`_
> +
> +Alternatively, you may report bugs to your software distributor / vendor.
> +
> +NOTE: at this time there is no support for forwarding QMP events back
> +to the clients

Also add caveat about FD passing support.

[...]

> diff --git a/tools/virt-qmp-proxy b/tools/virt-qmp-proxy
> new file mode 100755
> index 0000000000..57f9759fab
> --- /dev/null
> +++ b/tools/virt-qmp-proxy
> @@ -0,0 +1,133 @@
> +#!/usr/bin/env python3
> +
> +import argparse
> +import libvirt
> +import libvirt_qemu
> +import os
> +import re
> +import socket
> +import sys
> +import json
> +
> +
> +def get_domain(uri, domstr):
> +    conn = libvirt.open(uri)
> +
> +    dom = None
> +    if re.match(r'^\d+$', domstr):
> +        dom = conn.lookupByID(int(domstr))
> +    elif re.match(r'^[+a-f0-9]+$', domstr):

This works very poorly if you have a VM named for example 'cd' or any
combination of just letters abcdef.

> +        dom = conn.lookupByUUIDString(domstr)
> +    else:
> +        dom = conn.lookupByName(domstr)
> +
> +    if not dom.isActive():
> +        raise Exception(
> +            "Domain must be running to validate measurement")

This should mention the current usage or a generic error ;)

> +
> +    return conn, dom
> +
> +
> +def qmp_server(conn, dom, client, verbose):
> +    ver = conn.getVersion()

So this gets the version of the "default" emulator version, but if your
VM is using a custom one this will report it wrong.

E.g in my case I have a git qemu for a VM:

 517 2022-05-27 14:01:09.604+0000: 373562: debug : qemuMonitorJSONIOProcessLine:199 : Line [{"QMP": {"version": {"qemu": {"micro": 50, "minor": 0, "major": 7}, "package": "v7.0.0-1253-g2417cbd591"}, "capabilities": ["oob"]}}]



> +    major = int(ver / 1000000) % 1000
> +    minor = int(ver / 1000) % 1000
> +    micro = ver % 1000
> +
> +    greetingobj = {
> +        "QMP": {
> +            "version": {
> +                "qemu": {
> +                    "major": major,
> +                    "minor": minor,
> +                    "micro": micro,
> +                },
> +                "package": f"qemu-{major}.{minor}.{micro}",
> +            },
> +            "capabilities": [
> +                "oob"
> +            ],
> +        }
> +    }

But when I conect I get:

{"QMP": {"version": {"qemu": {"major": 7, "minor": 0, "micro": 0}, "package": "qemu-7.0.0"}, "capabilities": ["oob"]}}


At the very least this should be documented.

> +    greeting = json.dumps(greetingobj) + "\r\n"
> +    if verbose:
> +        print(greeting, end='')
> +    client.send(greeting.encode("utf-8"))
> +
> +    while True:
> +        # XXX shouldn't blindly assume this one read
> +        # will fully capture one-and-only-one cmd
> +        cmd = client.recv(1024).decode('utf8')

IIUC this limits the buffer to 1k max. Libvirt's RPC supports up to 4M.
1k could be limiting with some commands such as blockdev-add.

> +        if verbose:
> +            print(cmd)
> +
> +        if cmd == "":
> +            break
> +
> +        if "qmp_capabilities" in cmd:
> +            capabilitiesobj = {
> +                "return": {},
> +            }
> +            capabilities = json.dumps(capabilitiesobj) + "\r\n"
> +            if verbose:
> +                print(capabilities, end='')
> +            client.send(capabilities.encode("utf-8"))
> +            continue
> +
> +        id = None
> +        if "id" in cmd:
> +            id = cmd[id]
> +
> +        res = libvirt_qemu.qemuMonitorCommand(dom, cmd, 0)

If 'cmd' is not JSON this breaks horribly:

$ tools/virt-qmp-proxy 2 /tmp/asdf
libvirt:  error : internal error: cannot parse json test
: lexical error: invalid string in json text.
                                       test
                     (right here) ------^
tools/virt-qmp-proxy: internal error: cannot parse json test
: lexical error: invalid string in json text.
                                       test
                     (right here) ------^

and stops working, while real qemu behaves differently:

$ qemu-system-x86_64 -qmp stdio
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 7}, "package": "qemu-7.0.0-2.fc35"}, "capabilities": ["oob"]}}


help
{"error": {"class": "GenericError", "desc": "JSON parse error, invalid keyword 'help'"}}


Also since it's just a simple loop without event handling from qemu.
E.g. if I destroy the VM while it's running it simply waits. When I
issue another command, then the proxy exits:

$ tools/virt-qmp-proxy 2 /tmp/asdf
libvirt: Domain Config error : Requested operation is not valid: domain is not running
tools/virt-qmp-proxy: Requested operation is not valid: domain is not running


but the client itself just sees a closed socket.

Given the use case it's not a big problem but it should be at least
mentioned in the docs.

> +
> +        resobj = json.loads(res)
> +        del resobj["id"]
> +        if id is not None:
> +            resobj["id"] = id
> +        res = json.dumps(resobj) + "\r\n"
> +        if verbose:
> +            print(res, end='')
> +
> +        client.send(res.encode('utf8'))
> +
> +
> +def parse_commandline():
> +    parser = argparse.ArgumentParser(description="Libvirt QMP proxy")
> +    parser.add_argument("--connect", "-c",
> +                        help="Libvirt QEMU driver connection URI")
> +    parser.add_argument("--verbose", "-v", action='store_true',
> +                        help="Display QMP traffic")
> +    parser.add_argument("domain", metavar="DOMAIN",
> +                        help="Libvirt guest domain ID/UUID/Name")
> +    parser.add_argument("sockpath", metavar="QMP-SOCK-PATH",
> +                        help="UNIX socket path for QMP server")
> +
> +    return parser.parse_args()
> +
> +
> +def main():
> +    args = parse_commandline()
> +
> +    conn, dom = get_domain(args.connect, args.domain)
> +
> +    if conn.getType() != "QEMU":
> +        raise Exception("QMP proxy requires a QEMU driver connection not %s" %
> +                        conn.getType())
> +
> +    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
> +    if os.path.exists(args.sockpath):
> +        os.unlink(args.sockpath)
> +    sock.bind(args.sockpath)
> +    sock.listen(1)
> +
> +    while True:
> +        client, peeraddr = sock.accept()
> +        qmp_server(conn, dom, client, args.verbose)
> +
> +
> +try:
> +    main()
> +    sys.exit(0)
> +except Exception as e:
> +    print("%s: %s" % (sys.argv[0], str(e)))
> +    sys.exit(1)
> -- 
> 2.36.1
> 




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux