Re: [EXT] [RFC PATCH 0/4] PCI: endpoint: Introduce a virtio-net EP function

[Shunsuke Mie <mie@xxxxxxxxxx>]

On 2023/02/08 1:02, Frank Li wrote:
> > We project extending this module to support RDMA. The plan is based on
> > virtio-rdma[1].
> > It extends the virtio-net and we are plan to implement the proposed
> > spec based on this patch.
> > [1] virtio-rdma
> > - proposal:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > ernel.org%2Fall%2F20220511095900.343-1-
> > xieyongji%40bytedance.com%2FT%2F&data=05%7C01%7Cfrank.li%40nxp.co
> > m%7C0ef2bd62eda945c413be08db08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5
> > c301635%7C0%7C0%7C638113625610341574%7CUnknown%7CTWFpbGZsb3d
> > 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> > 3D%7C3000%7C%7C%7C&sdata=HyhpRTG8MNx%2BtfmWn6x3srmdBjHcZAo
> > 2qbxL9USph9o%3D&reserved=0
> > - presentation on kvm forum:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyout
> > u.be%2FQrhv6hC_YK4&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62
> > eda945c413be08db08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638113625610341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=ucOsGR1letTjxf0gKN6uls5y951CXaIspZtLGnASEC8%3D&res
> > erved=0
> Sorry for our outlook client always change link.  This previous discussion.
> https://lore.kernel.org/imx/d098a631-9930-26d3-48f3-8f95386c8e50@xxxxxx/T/#t
>
> Look like Endpoint maintainer Kishon like endpoint side work as vhost.
> Previous  Haotian Wang submit similar patches, which just not use eDMA, just use memcpy.
> But overall idea is the same.
>
> I think your and haotian's method is more reasonable for PCI-RC EP connection.
>
> Kishon is not active recently.   Maybe need Lorenzo Pieralisi and Bjorn helgass's comments
> for overall directions.
I think so too. Thank you for your summarization. I've commented on the 
e-mail.
> Frank Li
>
> > Please feel free to comment and suggest.
> > > Frank Li
> > >
> > > > To realize the function, this patchset has few changes and introduces a
> > > > new APIs to PCI EP framework related to virtio. Furthermore, it device
> > > > depends on the some patchtes that is discussing. Those depended
> > patchset
> > > > are following:
> > > > - [PATCH 1/2] dmaengine: dw-edma: Fix to change for continuous
> > transfer
> > > > link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
> > 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
> > 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
> > VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
> > > > ernel.org%2Fdmaengine%2F20221223022608.550697-1-
> > mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > > > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=tIn0MHzEvrdxaC4KKTvTRvYXBzQ6MyrFa2GXpa3ePv0%3D&
> > > > reserved=0
> > > > - [RFC PATCH 0/3] Deal with alignment restriction on EP side
> > > > link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
> > 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
> > 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
> > VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
> > > > ernel.org%2Flinux-pci%2F20230113090350.1103494-1-
> > mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > > > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=RLpnDiLwfqQd5QMXdiQyPVCkfOj8q2AyVeZOwWHvlsM%3
> > > > D&reserved=0
> > > > - [RFC PATCH v2 0/7] Introduce a vringh accessor for IO memory
> > > > link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> > %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C0ef2bd62eda945c413be08db
> > 08f62ba3%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6381136256
> > 10341574%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d
> > VZMaheX3eR1xA2wQtecmT857h2%2BFtUbhDSHXwgvsEY%3D&reserved=0
> > > > ernel.org%2Fvirtualization%2F20230202090934.549556-1-
> > mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cac57a62d4
> > > > 10b458a5ba408db05ce0a4e%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%
> > 7C0%7C638110154722945380%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> > wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> > C%7C%7C&sdata=6jgY76BMSbvamb%2Fl3Urjt4Gcizeqon%2BZE5nPssc2kDA%
> > > > 3D&reserved=0
> > > >
> > > > About this patchset has 4 patches. The first of two patch is little changes
> > > > to virtio. The third patch add APIs to easily access virtio data structure
> > > > on PCIe Host side memory. The last one introduce a virtio-net EP device
> > > > function. Details are in commit respectively.
> > > >
> > > > Currently those network devices are testd using ping only. I'll add a
> > > > result of performance evaluation using iperf and etc to the future version
> > > > of this patchset.
> > > >
> > > > Shunsuke Mie (4):
> > > >    virtio_pci: add a definition of queue flag in ISR
> > > >    virtio_ring: remove const from vring getter
> > > >    PCI: endpoint: Introduce virtio library for EP functions
> > > >    PCI: endpoint: function: Add EP function driver to provide virtio net
> > > >      device
> > > >
> > > >   drivers/pci/endpoint/Kconfig                  |   7 +
> > > >   drivers/pci/endpoint/Makefile                 |   1 +
> > > >   drivers/pci/endpoint/functions/Kconfig        |  12 +
> > > >   drivers/pci/endpoint/functions/Makefile       |   1 +
> > > >   .../pci/endpoint/functions/pci-epf-vnet-ep.c  | 343 ++++++++++
> > > >   .../pci/endpoint/functions/pci-epf-vnet-rc.c  | 635
> > ++++++++++++++++++
> > > >   drivers/pci/endpoint/functions/pci-epf-vnet.c | 387 +++++++++++
> > > >   drivers/pci/endpoint/functions/pci-epf-vnet.h |  62 ++
> > > >   drivers/pci/endpoint/pci-epf-virtio.c         | 113 ++++
> > > >   drivers/virtio/virtio_ring.c                  |   2 +-
> > > >   include/linux/pci-epf-virtio.h                |  25 +
> > > >   include/linux/virtio.h                        |   2 +-
> > > >   include/uapi/linux/virtio_pci.h               |   2 +
> > > >   13 files changed, 1590 insertions(+), 2 deletions(-)
> > > >   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-ep.c
> > > >   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet-rc.c
> > > >   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.c
> > > >   create mode 100644 drivers/pci/endpoint/functions/pci-epf-vnet.h
> > > >   create mode 100644 drivers/pci/endpoint/pci-epf-virtio.c
> > > >   create mode 100644 include/linux/pci-epf-virtio.h
> > > >
> > > > --
> > > > 2.25.1
> > Best,
> > Shunsuke

Best,

Shunsuke.