On 2017-06-20 16:58, alazar@xxxxxxxxxxxxxxx wrote: > On Mon, 19 Jun 2017 10:39:28 +0100, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote: >> On Fri, Jun 16, 2017 at 05:34:48PM +0200, Jan Kiszka wrote: >>> On 2017-06-16 17:18, Mihai Donțu wrote: >>>> On Fri, 2017-06-16 at 16:45 +0200, Jan Kiszka wrote: >>>>> On 2017-06-16 15:43, Adalbert Lazar wrote: >>>>>> This patch series proposes an interface that will allow a guest >>>>>> introspection tool to monitor and control other guests, in order to >>>>>> protect them against different forms of exploits. This type of interface >>>>>> is already present in the XEN hypervisor. >>>>>> >>>>>> With the current implementation, the introspection tool connects to >>>>>> the KVMi (the introspection subsystem from KVM) using a vsock socket, >>>>>> establishes a main communication channel, used for a few messages >>>>>> (KVMI_EVENT_GUEST_ON, KVMI_EVENT_GUEST_OFF, KVMI_GET_GUESTS and >>>>>> KVMI_GET_VERSION). >>>>>> >>>>>> Every KVMI_EVENT_GUEST_ON notification, makes the introspection tool >>>>>> establish a new connection, used to monitor and control that guest. >>>> >>>>> What prevented building this on top of the already existing guest debug >>>>> interfaces of KVM, maybe extending it where needed? Could be win-win. >>>> >>>> I might be mistaking, but this would require the application using the >>>> introspection capabilities to run on the host. If so, what we are >>>> trying to do is to isolate the application into its own VM. This is why >>>> we use vSock to communicate with the host. >>> >>> The hypervisor process could terminate your link, providing that other >>> VM the introspection access. Or you even have a gdb-speaking process >>> running on the host, just reusing the existing gdbstub of QEMU. Just >>> wild ideas, I didn't look into details, and you may further elaborate on >>> your requirements. >> >> QEMU userspace can provide the interface on an AF_VSOCK listen socket. >> Here is a rough idea: >> >> qemu --chardev socket,id=chardev0,type=vsock,port=1234,server,nowait \ >> --guest-introspection chardev=chardev0,allowed-cids=10 >> >> Since it uses a chardev this means the guest introspection API is also >> available via UNIX domain sockets to local applications, etc. > > Exposing the introspections commands and events as IOCTLs can be done easily. > > However, dropping the vsock access from the kernel, or having > the IOCTL <-> vsock adapter only in userspace will change more things. > With the proposed API, the guest introspection looks like: > > ------------------ ----------------------------- > | |<-- /dev/kvm -->| qemu VM1 | > | | |------- | > | | | Linux | | > | KVM | ----------------------------- > | |<-- /dev/kvm -->| qemu VM2 | > | | |--------- | > | | | Windows | | > | | ----------------------------- > | |<-- /dev/kvm -->| qemu VM3 | > | --------------| |--------------------------- | > | | kvmi <- vhost-vsock -----------> guest introspection tool | | > ------------------ ----------------------------- > > The introspection tool connects directly to the introspection subsystem (kvmi). > > Moving the vsock to userland will change this: > > ----------------------------- > /----- /dev/kvm -->| new_tool (guest on/off/list)|<-- vsock -->\ > | ----------------------------- | > | | > ----------------v- ----------------------------- | > | |<-- /dev/kvm -->| qemu VM1 |<-- vsock -->| > | | |------- | | > | | | Linux | | | > | KVM | ----------------------------- | > | |<-- /dev/kvm -->| qemu VM2 |<-- vsock -->| > | | |--------- | | > | | | Windows | | | > | | ----------------------------- | > | |<-- /dev/kvm -->| qemu VM3 /----->|<-- vsock -->/ > | -------| |---------------------v---- | > | | kvmi | | guest introspection tool | | > ------------------ ----------------------------- > > There will be a need for a new tool (and/or libvirt modified) to get > the guest events (on/off/list) and change the VM1, VM2 invocations (to > make them connect with the introspection tool). This might also be a > problem with products having the host locked down (eg. RHEV). > > Moving more code to QEMU will make the introspection harder with other > hw emulators (from kvmtool to Google's implementation) because they will I would bet that (not only) Google folks will look rather skeptical at any proposal to widen the existing kvm user interface anyway, because of security implications. Reducing that should be a primary design goal IMHO. Jan > need more than just: > - migration notification: the introspection tool needs to unhook > from the introspected guests [minor change] > - KVM_SET_VM_UUID support [minor change] > - vsock [medium change], but only if this particular emulator is used > to start the guest introspection tool > > Also, the path between kvmi and the introspection tool (when running > isolated in a guest) will be bigger, adding some overhead, which is > a big problem with live introspectors. EXTERIOR VMI paper shows[1] > between 5-20 times slower execution for small programs (ps, uptime). > So, a smaller path will help in keeping the overhead lower. > > My colleague will follow up with some stats collected during an > introspection session. Hopefully they will shed more light on the > performance required from the tool-KVM communication channel. > > [1]: https://www.utdallas.edu/~zhiqiang.lin/file/VEE13-Slides.pdf > -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux
