Re: [PATCH vhost v11 05/10] virtio_ring: introduce virtqueue_dma_dev()

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, 8 Aug 2023 11:59:04 +0800, Jason Wang <jasowang@xxxxxxxxxx> wrote:
> On Tue, Aug 8, 2023 at 11:57 AM Xuan Zhuo <xuanzhuo@xxxxxxxxxxxxxxxxx> wrote:
> >
> > On Tue, 8 Aug 2023 11:49:08 +0800, Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > On Tue, Aug 8, 2023 at 11:12 AM Xuan Zhuo <xuanzhuo@xxxxxxxxxxxxxxxxx> wrote:
> > > >
> > > > On Tue, 8 Aug 2023 11:08:09 +0800, Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > > > On Tue, Aug 8, 2023 at 10:52 AM Xuan Zhuo <xuanzhuo@xxxxxxxxxxxxxxxxx> wrote:
> > > > > >
> > > > > > On Tue, 8 Aug 2023 10:26:04 +0800, Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > > > > > On Mon, Aug 7, 2023 at 2:15 PM Xuan Zhuo <xuanzhuo@xxxxxxxxxxxxxxxxx> wrote:
> > > > > > > >
> > > > > > > > On Wed, 2 Aug 2023 09:49:31 +0800, Xuan Zhuo <xuanzhuo@xxxxxxxxxxxxxxxxx> wrote:
> > > > > > > > > On Tue, 1 Aug 2023 12:17:47 -0400, "Michael S. Tsirkin" <mst@xxxxxxxxxx> wrote:
> > > > > > > > > > On Fri, Jul 28, 2023 at 02:02:33PM +0800, Xuan Zhuo wrote:
> > > > > > > > > > > On Tue, 25 Jul 2023 19:07:23 +0800, Xuan Zhuo <xuanzhuo@xxxxxxxxxxxxxxxxx> wrote:
> > > > > > > > > > > > On Tue, 25 Jul 2023 03:34:34 -0400, "Michael S. Tsirkin" <mst@xxxxxxxxxx> wrote:
> > > > > > > > > > > > > On Tue, Jul 25, 2023 at 10:13:48AM +0800, Xuan Zhuo wrote:
> > > > > > > > > > > > > > On Mon, 24 Jul 2023 09:43:42 -0700, Christoph Hellwig <hch@xxxxxxxxxxxxx> wrote:
> > > > > > > > > > > > > > > On Thu, Jul 20, 2023 at 01:21:07PM -0400, Michael S. Tsirkin wrote:
> > > > > > > > > > > > > > > > Well I think we can add wrappers like virtio_dma_sync and so on.
> > > > > > > > > > > > > > > > There are NOP for non-dma so passing the dma device is harmless.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Yes, please.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I am not sure I got this fully.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Are you mean this:
> > > > > > > > > > > > > > https://lore.kernel.org/all/20230214072704.126660-8-xuanzhuo@xxxxxxxxxxxxxxxxx/
> > > > > > > > > > > > > > https://lore.kernel.org/all/20230214072704.126660-9-xuanzhuo@xxxxxxxxxxxxxxxxx/
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Then the driver must do dma operation(map and sync) by these virtio_dma_* APIs.
> > > > > > > > > > > > > > No care the device is non-dma device or dma device.
> > > > > > > > > > > > >
> > > > > > > > > > > > > yes
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Then the AF_XDP must use these virtio_dma_* APIs for virtio device.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We'll worry about AF_XDP when the patch is posted.
> > > > > > > > > > > >
> > > > > > > > > > > > YES.
> > > > > > > > > > > >
> > > > > > > > > > > > We discussed it. They voted 'no'.
> > > > > > > > > > > >
> > > > > > > > > > > > http://lore.kernel.org/all/20230424082856.15c1e593@xxxxxxxxxx
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Hi guys, this topic is stuck again. How should I proceed with this work?
> > > > > > > > > > >
> > > > > > > > > > > Let me briefly summarize:
> > > > > > > > > > > 1. The problem with adding virtio_dma_{map, sync} api is that, for AF_XDP and
> > > > > > > > > > > the driver layer, we need to support these APIs. The current conclusion of
> > > > > > > > > > > AF_XDP is no.
> > > > > > > > > > >
> > > > > > > > > > > 2. Set dma_set_mask_and_coherent, then we can use DMA API uniformly inside
> > > > > > > > > > > driver. This idea seems to be inconsistent with the framework design of DMA. The
> > > > > > > > > > > conclusion is no.
> > > > > > > > > > >
> > > > > > > > > > > 3. We noticed that if the virtio device supports VIRTIO_F_ACCESS_PLATFORM, it
> > > > > > > > > > > uses DMA API. And this type of device is the future direction, so we only
> > > > > > > > > > > support DMA premapped for this type of virtio device. The problem with this
> > > > > > > > > > > solution is that virtqueue_dma_dev() only returns dev in some cases, because
> > > > > > > > > > > VIRTIO_F_ACCESS_PLATFORM is supported in such cases.
> > > > > > >
> > > > > > > Could you explain the issue a little bit more?
> > > > > > >
> > > > > > > E.g if we limit AF_XDP to ACESS_PLATFROM only, why does
> > > > > > > virtqueue_dma_dev() only return dev in some cases?
> > > > > >
> > > > > > The behavior of virtqueue_dma_dev() is not related to AF_XDP.
> > > > > >
> > > > > > The return value of virtqueue_dma_dev() is used for the DMA APIs. So it can
> > > > > > return dma dev when the virtio is with ACCESS_PLATFORM. If virtio is without
> > > > > > ACCESS_PLATFORM then it MUST return NULL.
> > > > > >
> > > > > > In the virtio-net driver, if the virtqueue_dma_dev() returns dma dev,
> > > > > > we can enable AF_XDP. If not, we return error to AF_XDP.
> > > > >
> > > > > Yes, as discussed, just having wrappers in the virtio_ring and doing
> > > > > the switch there. Then can virtio-net use them without worrying about
> > > > > DMA details?
> > > >
> > > >
> > > > Yes. In the virtio drivers, we can use the wrappers. That is ok.
> > > >
> > > > But we also need to support virtqueue_dma_dev() for AF_XDP, because that the
> > > > AF_XDP will not use the wrappers.
> > >
> > > You mean AF_XDP core or other? Could you give me an example?
> >
> >
> > Yes. The AF_XDP core.
> >
> > Now the AF_XDP core will do the dma operation.  Because that the memory is
> > allocated by the user from the user space.  So before putting the memory to the
> > driver, the AF_XDP will do the dma mapping.
> >
> >
> > int xp_dma_map(struct xsk_buff_pool *pool, struct device *dev,
> >                unsigned long attrs, struct page **pages, u32 nr_pages)
> > {
>
> I think it's the driver who passes the device pointer here. Anything I missed?

YES.

When the AF_XPD is bonding to the device queue, the driver should call this to
do the dma inside the AF_XDP core, and the dev is recorded  by the AF_XDP core.

The AF_XDP will call dma_sync APIs when the driver reads the buffers from the rx
queue and the driver xmit the buffers to the device.

The AF_XDP will do some DMA operation automatically.
The DMA APIs is used by the AF_XDP core.

Thanks.





>
> Thanks
>
_______________________________________________
Virtualization mailing list
Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/virtualization




[Index of Archives]     [KVM Development]     [Libvirt Development]     [Libvirt Users]     [CentOS Virtualization]     [Netdev]     [Ethernet Bridging]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Bugtraq]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux Admin]     [Samba]

  Powered by Linux