On Tue, Jan 12, 2021 at 4:12 AM Jason Wang <jasowang@xxxxxxxxxx> wrote: > > > On 2021/1/11 下午8:26, Eli Cohen wrote: > > On Wed, Dec 16, 2020 at 02:48:18PM +0800, Jason Wang wrote: > >> This patch introduces the control virtqueue support for vDPA > >> simulator. This is a requirement for supporting advanced features like > >> multiqueue. > >> > >> A requirement for control virtqueue is to isolate its memory access > >> from the rx/tx virtqueues. This is because when using vDPA device > >> for VM, the control virqueue is not directly assigned to VM. Userspace > >> (Qemu) will present a shadow control virtqueue to control for > >> recording the device states. > >> > >> The isolation is done via the virtqueue groups and ASID support in > >> vDPA through vhost-vdpa. The simulator is extended to have: > >> > >> 1) three virtqueues: RXVQ, TXVQ and CVQ (control virtqueue) > >> 2) two virtqueue groups: group 0 contains RXVQ and TXVQ; group 1 > >> contains CVQ > >> 3) two address spaces and the simulator simply implements the address > >> spaces by mapping it 1:1 to IOTLB. > >> > >> For the VM use cases, userspace(Qemu) may set AS 0 to group 0 and AS 1 > >> to group 1. So we have: > >> > >> 1) The IOTLB for virtqueue group 0 contains the mappings of guest, so > >> RX and TX can be assigned to guest directly. > >> 2) The IOTLB for virtqueue group 1 contains the mappings of CVQ which > >> is the buffers that allocated and managed by VMM only. So CVQ of > >> vhost-vdpa is visible to VMM only. And Guest can not access the CVQ > >> of vhost-vdpa. > >> > >> For the other use cases, since AS 0 is associated to all virtqueue > >> groups by default. All virtqueues share the same mapping by default. > >> > >> To demonstrate the function, VIRITO_NET_F_CTRL_MACADDR is > >> implemented in the simulator for the driver to set mac address. > >> > > Hi Jason, > > > > is there any version of qemu/libvirt available that I can see the > > control virtqueue working in action? > > > Not yet, the qemu part depends on the shadow virtqueue work of Eugenio. > But it will work as: > > 1) qemu will use a separated address space for the control virtqueue > (shadow) exposed through vhost-vDPA > 2) the commands sent through control virtqueue by guest driver will > intercept by qemu > 3) Qemu will send those commands to the shadow control virtqueue > > Eugenio, any ETA for the new version of shadow virtqueue support in Qemu? > Hi Jason. Sorry for the late response. For the notification part I have addressed all the issues of the RFC [1], except the potential race conditions Stefan pointed, and tested with vdpa devices. You can find at https://github.com/eugpermar/qemu/tree/vdpa_sw_live_migration.d/notifications.rfc . Since the shadow path is activated only through QMP and does not interfere with regular operation, I could post to the qemu list if you prefer. The series will be smaller if merged in steps. Adding the buffer forwarding on top should not take long. [1] https://lkml.org/lkml/2020/9/23/1243 Thanks!
