On Wed, Jan 20, 2021 at 1:54 AM Randy Dunlap <rdunlap@xxxxxxxxxxxxx> wrote: > > Hi, > > Documentation comments only: > Will fix it. Thanks, Yongji > On 1/18/21 9:07 PM, Xie Yongji wrote: > > > > Signed-off-by: Xie Yongji <xieyongji@xxxxxxxxxxxxx> > > --- > > Documentation/driver-api/vduse.rst | 85 ++ > > > > diff --git a/Documentation/driver-api/vduse.rst b/Documentation/driver-api/vduse.rst > > new file mode 100644 > > index 000000000000..9418a7f6646b > > --- /dev/null > > +++ b/Documentation/driver-api/vduse.rst > > @@ -0,0 +1,85 @@ > > +================================== > > +VDUSE - "vDPA Device in Userspace" > > +================================== > > + > > +vDPA (virtio data path acceleration) device is a device that uses a > > +datapath which complies with the virtio specifications with vendor > > +specific control path. vDPA devices can be both physically located on > > +the hardware or emulated by software. VDUSE is a framework that makes it > > +possible to implement software-emulated vDPA devices in userspace. > > + > > +How VDUSE works > > +------------ > > +Each userspace vDPA device is created by the VDUSE_CREATE_DEV ioctl on > > +the VDUSE character device (/dev/vduse). Then a file descriptor pointing > > +to the new resources will be returned, which can be used to implement the > > +userspace vDPA device's control path and data path. > > + > > +To implement control path, the read/write operations to the file descriptor > > +will be used to receive/reply the control messages from/to VDUSE driver. > > +Those control messages are mostly based on the vdpa_config_ops which defines > > +a unified interface to control different types of vDPA device. > > + > > +The following types of messages are provided by the VDUSE framework now: > > + > > +- VDUSE_SET_VQ_ADDR: Set the addresses of the different aspects of virtqueue. > > + > > +- VDUSE_SET_VQ_NUM: Set the size of virtqueue > > + > > +- VDUSE_SET_VQ_READY: Set ready status of virtqueue > > + > > +- VDUSE_GET_VQ_READY: Get ready status of virtqueue > > + > > +- VDUSE_SET_VQ_STATE: Set the state (last_avail_idx) for virtqueue > > + > > +- VDUSE_GET_VQ_STATE: Get the state (last_avail_idx) for virtqueue > > + > > +- VDUSE_SET_FEATURES: Set virtio features supported by the driver > > + > > +- VDUSE_GET_FEATURES: Get virtio features supported by the device > > + > > +- VDUSE_SET_STATUS: Set the device status > > + > > +- VDUSE_GET_STATUS: Get the device status > > + > > +- VDUSE_SET_CONFIG: Write to device specific configuration space > > + > > +- VDUSE_GET_CONFIG: Read from device specific configuration space > > + > > +- VDUSE_UPDATE_IOTLB: Notify userspace to update the memory mapping in device IOTLB > > + > > +Please see include/linux/vdpa.h for details. > > + > > +In the data path, vDPA device's iova regions will be mapped into userspace with > > +the help of VDUSE_IOTLB_GET_FD ioctl on the userspace vDPA device fd: > > + > > +- VDUSE_IOTLB_GET_FD: get the file descriptor to iova region. Userspace can > > + access this iova region by passing the fd to mmap(2). > > + > > +Besides, the eventfd mechanism is used to trigger interrupt callbacks and > > +receive virtqueue kicks in userspace. The following ioctls on the userspace > > +vDPA device fd are provided to support that: > > + > > +- VDUSE_VQ_SETUP_KICKFD: set the kickfd for virtqueue, this eventfd is used > > + by VDUSE driver to notify userspace to consume the vring. > > + > > +- VDUSE_VQ_SETUP_IRQFD: set the irqfd for virtqueue, this eventfd is used > > + by userspace to notify VDUSE driver to trigger interrupt callbacks. > > + > > +MMU-based IOMMU Driver > > +---------------------- > > +In virtio-vdpa case, VDUSE framework implements a MMU-based on-chip IOMMU > > an MMU-based > > > +driver to support mapping the kernel dma buffer into the userspace iova > > DMA > > > +region dynamically. > > + > > +The basic idea behind this driver is treating MMU (VA->PA) as IOMMU (IOVA->PA). > > +The driver will set up MMU mapping instead of IOMMU mapping for the DMA transfer > > +so that the userspace process is able to use its virtual address to access > > +the dma buffer in kernel. > > DMA > > > + > > +And to avoid security issue, a bounce-buffering mechanism is introduced to > > +prevent userspace accessing the original buffer directly which may contain other > > +kernel data. During the mapping, unmapping, the driver will copy the data from > > +the original buffer to the bounce buffer and back, depending on the direction of > > +the transfer. And the bounce-buffer addresses will be mapped into the user address > > +space instead of the original one. > > > thanks. > -- > ~Randy >
