On 2020/9/24 下午6:17, Stefan Hajnoczi wrote:
On Thu, Sep 24, 2020 at 11:21:01AM +0800, Jason Wang wrote:
This series tries to add the support for control virtqueue in vDPA.
Please include documentation for both driver authors and vhost-vdpa
ioctl users. vhost-vdpa ioctls are only documented with a single
sentence. Please add full information on arguments, return values, and a
high-level explanation of the feature (like this cover letter) to
introduce the API.
Right, this is in the TODO list. (And we probably need to start with
documenting vDPA bus operations first).
What is the policy for using virtqueue groups? My guess is:
1. virtio_vdpa simply enables all virtqueue groups.
2. vhost_vdpa relies on userspace policy on how to use virtqueue groups.
Are the semantics of virtqueue groups documented somewhere so
userspace knows what to do? If a vDPA driver author decides to create
N virtqueue groups, N/2 virtqueue groups, or just 1 virtqueue group,
how will userspace know what to do?
So the mapping from virtqueue to virtqueue group is mandated by the vDPA
device(driver). vDPA bus driver (like vhost-vDPA), can only change the
association between virtqueue groups and ASID.
By default, it is required all virtqueue groups to be associated to
address space 0. This make sure virtio_vdpa can work without any special
groups/asid configuration.
I admit we need document all those semantics/polices.
Maybe a document is needed to describe the recommended device-specific
virtqueue groups that vDPA drivers should implement (e.g. "put the net
control vq into its own virtqueue group")?
Yes, note that this depends on the hardware capability actually. It can
only put control vq in other virtqueue group if:
1) hardware support to isolate control vq DMA from the rest virtqueues
(PASID or simply using PA (translated address) for control vq)
or
2) the control vq is emulated by vDPA device driver (like vdpa_sim did).
This could become messy with guidelines. For example, drivers might be
shipped that aren't usable for certain use cases just because the author
didn't know that a certain virtqueue grouping is advantageous.
Right.
BTW I like how general this feature is. It seems to allow vDPA devices
to be split into sub-devices for further passthrough. Who will write the
first vDPA-on-vDPA driver? :)
Yes, that's an interesting question. For now, I can imagine we can
emulated a SRIOV based virtio-net VFs via this.
If we want to expose the ASID setting to guest as well, it probably
needs more thought.
Thanks
Stefan
