On Mon, 19 Jun 2017 10:39:28 +0100, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote:
> On Fri, Jun 16, 2017 at 05:34:48PM +0200, Jan Kiszka wrote:
> > On 2017-06-16 17:18, Mihai Donțu wrote:
> > > On Fri, 2017-06-16 at 16:45 +0200, Jan Kiszka wrote:
> > >> On 2017-06-16 15:43, Adalbert Lazar wrote:
> > >>> This patch series proposes an interface that will allow a guest
> > >>> introspection tool to monitor and control other guests, in order to
> > >>> protect them against different forms of exploits. This type of interface
> > >>> is already present in the XEN hypervisor.
> > >>>
> > >>> With the current implementation, the introspection tool connects to
> > >>> the KVMi (the introspection subsystem from KVM) using a vsock socket,
> > >>> establishes a main communication channel, used for a few messages
> > >>> (KVMI_EVENT_GUEST_ON, KVMI_EVENT_GUEST_OFF, KVMI_GET_GUESTS and
> > >>> KVMI_GET_VERSION).
> > >>>
> > >>> Every KVMI_EVENT_GUEST_ON notification, makes the introspection tool
> > >>> establish a new connection, used to monitor and control that guest.
> > >
> > >> What prevented building this on top of the already existing guest debug
> > >> interfaces of KVM, maybe extending it where needed? Could be win-win.
> > >
> > > I might be mistaking, but this would require the application using the
> > > introspection capabilities to run on the host. If so, what we are
> > > trying to do is to isolate the application into its own VM. This is why
> > > we use vSock to communicate with the host.
> >
> > The hypervisor process could terminate your link, providing that other
> > VM the introspection access. Or you even have a gdb-speaking process
> > running on the host, just reusing the existing gdbstub of QEMU. Just
> > wild ideas, I didn't look into details, and you may further elaborate on
> > your requirements.
>
> QEMU userspace can provide the interface on an AF_VSOCK listen socket.
> Here is a rough idea:
>
> qemu --chardev socket,id=chardev0,type=vsock,port=1234,server,nowait \
> --guest-introspection chardev=chardev0,allowed-cids=10
>
> Since it uses a chardev this means the guest introspection API is also
> available via UNIX domain sockets to local applications, etc.
Exposing the introspections commands and events as IOCTLs can be done easily.
However, dropping the vsock access from the kernel, or having
the IOCTL <-> vsock adapter only in userspace will change more things.
With the proposed API, the guest introspection looks like:
------------------ -----------------------------
| |<-- /dev/kvm -->| qemu VM1 |
| | |------- |
| | | Linux | |
| KVM | -----------------------------
| |<-- /dev/kvm -->| qemu VM2 |
| | |--------- |
| | | Windows | |
| | -----------------------------
| |<-- /dev/kvm -->| qemu VM3 |
| --------------| |--------------------------- |
| | kvmi <- vhost-vsock -----------> guest introspection tool | |
------------------ -----------------------------
The introspection tool connects directly to the introspection subsystem (kvmi).
Moving the vsock to userland will change this:
-----------------------------
/----- /dev/kvm -->| new_tool (guest on/off/list)|<-- vsock -->\
| ----------------------------- |
| |
----------------v- ----------------------------- |
| |<-- /dev/kvm -->| qemu VM1 |<-- vsock -->|
| | |------- | |
| | | Linux | | |
| KVM | ----------------------------- |
| |<-- /dev/kvm -->| qemu VM2 |<-- vsock -->|
| | |--------- | |
| | | Windows | | |
| | ----------------------------- |
| |<-- /dev/kvm -->| qemu VM3 /----->|<-- vsock -->/
| -------| |---------------------v---- |
| | kvmi | | guest introspection tool | |
------------------ -----------------------------
There will be a need for a new tool (and/or libvirt modified) to get
the guest events (on/off/list) and change the VM1, VM2 invocations (to
make them connect with the introspection tool). This might also be a
problem with products having the host locked down (eg. RHEV).
Moving more code to QEMU will make the introspection harder with other
hw emulators (from kvmtool to Google's implementation) because they will
need more than just:
- migration notification: the introspection tool needs to unhook
from the introspected guests [minor change]
- KVM_SET_VM_UUID support [minor change]
- vsock [medium change], but only if this particular emulator is used
to start the guest introspection tool
Also, the path between kvmi and the introspection tool (when running
isolated in a guest) will be bigger, adding some overhead, which is
a big problem with live introspectors. EXTERIOR VMI paper shows[1]
between 5-20 times slower execution for small programs (ps, uptime).
So, a smaller path will help in keeping the overhead lower.
My colleague will follow up with some stats collected during an
introspection session. Hopefully they will shed more light on the
performance required from the tool-KVM communication channel.
[1]: https://www.utdallas.edu/~zhiqiang.lin/file/VEE13-Slides.pdf
