On Fri, Nov 20, 2020 at 07:50:38PM +0100, Eugenio Pérez wrote: > This series enable vDPA software assisted live migration for vhost-net > devices. This is a new method of vhost devices migration: Instead of > relay on vDPA device's dirty logging capability, SW assisted LM > intercepts dataplane, forwarding the descriptors between VM and device. Pros: + vhost/vDPA devices don't need to implement dirty memory logging + Obsoletes ioctl(VHOST_SET_LOG_BASE) and friends Cons: - Not generic, relies on vhost-net-specific ioctls - Doesn't support VIRTIO Shared Memory Regions https://github.com/oasis-tcs/virtio-spec/blob/master/shared-mem.tex - Performance (see below) I think performance will be significantly lower when the shadow vq is enabled. Imagine a vDPA device with hardware vq doorbell registers mapped into the guest so the guest driver can directly kick the device. When the shadow vq is enabled a vmexit is needed to write to the shadow vq ioeventfd, then the host kernel scheduler switches to a QEMU thread to read the ioeventfd, the descriptors are translated, QEMU writes to the vhost hdev kick fd, the host kernel scheduler switches to the vhost worker thread, vhost/vDPA notifies the virtqueue, and finally the vDPA driver writes to the hardware vq doorbell register. That is a lot of overhead compared to writing to an exitless MMIO register! If the shadow vq was implemented in drivers/vhost/ and QEMU used the existing ioctl(VHOST_SET_LOG_BASE) approach, then the overhead would be reduced to just one set of ioeventfd/irqfd. In other words, the QEMU dirty memory logging happens asynchronously and isn't in the dataplane. In addition, hardware that supports dirty memory logging as well as software vDPA devices could completely eliminate the shadow vq for even better performance. But performance is a question of "is it good enough?". Maybe this approach is okay and users don't expect good performance while dirty memory logging is enabled. I just wanted to share the idea of moving the shadow vq into the kernel in case you like that approach better. Stefan
Attachment:
signature.asc
Description: PGP signature
