> -----Original Message-----
> From: Shunsuke Mie <mie@xxxxxxxxxx>
> Sent: Tuesday, April 18, 2023 5:31 AM
> To: Frank Li <frank.li@xxxxxxx>; Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
> Cc: Krzysztof Wilczyński <kw@xxxxxxxxx>; Manivannan Sadhasivam
> <mani@xxxxxxxxxx>; Kishon Vijay Abraham I <kishon@xxxxxxxxxx>; Bjorn
> Helgaas <bhelgaas@xxxxxxxxxx>; Michael S. Tsirkin <mst@xxxxxxxxxx>;
> Jason Wang <jasowang@xxxxxxxxxx>; Jon Mason <jdmason@xxxxxxxx>;
> Randy Dunlap <rdunlap@xxxxxxxxxxxxx>; Ren Zhijie
> <renzhijie2@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; linux-
> pci@xxxxxxxxxxxxxxx; virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
> Subject: Re: [EXT] [RFC PATCH 0/3] Introduce a PCIe endpoint virtio console
>
> Caution: EXT Email
>
> On 2023/04/18 0:19, Frank Li wrote:
> >
> >> -----Original Message-----
> >> From: Shunsuke Mie <mie@xxxxxxxxxx>
> >> Sent: Sunday, April 16, 2023 9:12 PM
> >> To: Frank Li <frank.li@xxxxxxx>; Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
> >> Cc: Krzysztof Wilczyński <kw@xxxxxxxxx>; Manivannan Sadhasivam
> >> <mani@xxxxxxxxxx>; Kishon Vijay Abraham I <kishon@xxxxxxxxxx>; Bjorn
> >> Helgaas <bhelgaas@xxxxxxxxxx>; Michael S. Tsirkin <mst@xxxxxxxxxx>;
> >> Jason Wang <jasowang@xxxxxxxxxx>; Jon Mason <jdmason@xxxxxxxx>;
> >> Randy Dunlap <rdunlap@xxxxxxxxxxxxx>; Ren Zhijie
> >> <renzhijie2@xxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx; linux-
> >> pci@xxxxxxxxxxxxxxx; virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
> >> Subject: Re: [EXT] [RFC PATCH 0/3] Introduce a PCIe endpoint virtio
> console
> >>
> >> Caution: EXT Email
> >>
> >> On 2023/04/14 23:39, Frank Li wrote:
> >>>> -----Original Message-----
> >>>> From: Shunsuke Mie <mie@xxxxxxxxxx>
> >>>> Sent: Friday, April 14, 2023 7:39 AM
> >>>> To: Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
> >>>> Cc: Krzysztof Wilczyński <kw@xxxxxxxxx>; Manivannan Sadhasivam
> >>>> <mani@xxxxxxxxxx>; Kishon Vijay Abraham I <kishon@xxxxxxxxxx>;
> Bjorn
> >>>> Helgaas <bhelgaas@xxxxxxxxxx>; Michael S. Tsirkin
> <mst@xxxxxxxxxx>;
> >>>> Jason Wang <jasowang@xxxxxxxxxx>; Shunsuke Mie
> <mie@xxxxxxxxxx>;
> >>>> Frank Li <frank.li@xxxxxxx>; Jon Mason <jdmason@xxxxxxxx>; Randy
> >>>> Dunlap <rdunlap@xxxxxxxxxxxxx>; Ren Zhijie
> <renzhijie2@xxxxxxxxxx>;
> >>>> linux-kernel@xxxxxxxxxxxxxxx; linux-pci@xxxxxxxxxxxxxxx;
> >>>> virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
> >>>> Subject: [EXT] [RFC PATCH 0/3] Introduce a PCIe endpoint virtio console
> >>>>
> >>>> Caution: EXT Email
> >>>>
> >>>> PCIe endpoint framework provides APIs to implement PCIe endpoint
> >>>> function.
> >>>> This framework allows defining various PCIe endpoint function
> behaviors
> >> in
> >>>> software. This patch extend the framework for virtio pci device. The
> >>>> virtio is defined to communicate guest on virtual machine and host side.
> >>>> Advantage of the virtio is the efficiency of data transfer and the
> >> conciseness
> >>>> of implementation device using software. It also be applied to PCIe
> >>>> endpoint function.
> >>>>
> >>>> We designed and implemented a PCIe EP virtio console function driver
> >> using
> >>>> the extended PCIe endpoint framework for virtio. It can be
> communicate
> >>>> host and endpoint over virtio as console.
> >>>>
> >>>> An architecture of the function driver is following:
> >>>>
> >>>> ┌────────────┐ ┌───────────
> ─
> >> ──
> >>>> ────────┬────────────┐
> >>>> │virtio │ │ │virtio │
> >>>> │console drv │ ├───────────────┐ │
> >> console
> >>>> drv │
> >>>> ├────────────┤ │(virtio console│ ├──
> ─
> >> ──
> >>>> ───────┤
> >>>> │ virtio bus │ │ device) │◄────►│ virtio bus │
> >>>> ├────────────┤ ├---------------┤ └───
> ─
> >> ──
> >>>> ──────┤
> >>>> │ │ │ pci ep virtio │ │
> >>>> │ pci bus │ │ console drv │ │
> >>>> │ │ pcie ├───────────────┤
> │
> >>>> │ │ ◄─────► │ pci ep Bus │ │
> >>>> └────────────┘ └───────────
> ─
> >> ──
> >>>> ─┴───────────────────┘
> >>>> PCIe Root PCIe Endpoint
> >>>>
> >>> [Frank Li] Some basic question,
> >>> I see you call register_virtio_device at epf_vcon_setup_vdev,
> >>> Why call it as virtio console? I suppose it should be virtiobus directly?
> >> I'm sorry I didn't understand your question. What do you mean the
> >> virtiobus directly?
> > I go through your code again. I think I understand why you need pci-epf-
> vcon.c.
> > Actually, my means is like virtio_mmio_probe.
> >
> > vm_dev->vdev.id.device = readl(vm_dev->base +
> VIRTIO_MMIO_DEVICE_ID);
> > vm_dev->vdev.id.vendor = readl(vm_dev->base +
> VIRTIO_MMIO_VENDOR_ID);
> >
> > I am not sure that if VIRTIO_MMIO_VENDOR_ID and
> VIRTIO_MMIO_DEVICE_ID
> > reuse PCI's vendor ID and Device ID. If yes, you can directly get such
> information
> > from epf. If no, a customer field can been added at epf driver.
> >
> > So you needn't write pci-epf-vcon and pci-epf-vnet .....
> >
> > Of cause it will be wonderful if directly use virtio_mmio_probe by dynmatic
> create platform
> > Devices. It may have some difficult because pci memory map requirement.
> I think that some fields are shared between the vdev and epf device.
> However, we need to implement device emulation because each virtio
> device
> has its specific set of tasks. For example, the virtio-net device has a
> control queue, and the driver can request MAC filters, VLANs, and other
> settings via this queue. These requests have to be processed by the virtio
> device that we are implementing in pci-epf-vnet.
>
> The simplest virtio-console device doesn’t have these tasks, but the other
> virtio devices requireprocessing them.
>
> That's why the current pci-epf-virtio design requires a specific
> implementation for each virtio device.
>
> Is this what you meant
I see. Thank you for explain.
Thing may become complex. How to implement composite at EPF driver?
Such as console + net
>
> >
> >>> Previous you use virtio-net, why change to virtio-console here? Does it
> >> matter?
> >>
> >> No, it doesn't. Just I'd like to break down the changes into smaller
> >> steps to make it easier to review and merge the changes.
> >>
> >> As a first step, I propose adding a simplest virtio function driver with
> >> the extension defined in pci-epf-virtio.{h,c}.
> >>
> >>> All virtio-XXX should work?
> >> Yes, the extension is designed to use any type of virtio device.
> >>> You removed EDMA support this version?
> >> I planed the support will be added with epf virtio-net patches.
> >>>> Introduced driver is `pci ep virtio console drv` in the figure. It works
> >>>> as ep function for PCIe root and virtual virtio console device for PCIe
> >>>> endpoint. Each side of virtio console driver has virtqueue, and
> >>>> introduced driver transfers data on the virtqueue to each other. A data
> >>>> on root tx queue is transfered to endpoint rx queue and vice versa.
> >>>>
> >>>> This patchset is depend follwing patches which are under discussion.
> >>>>
> >>>> - [RFC PATCH 0/3] Deal with alignment restriction on EP side
> >>>> link:
> >>>>
> >>
> https://lore.k/
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C80a237184ff4420a96de08db
> 3ff80d78%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63817410683
> 2489955%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LSP
> LeFzTYpUVTZpymoM00Es9kOpxlN90%2BDM6ceDJMFE%3D&reserved=0
> >> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7Cff59a16f88c643913e3908
> db3
> >>
> ee91ca8%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63817294315
> >>
> 3821831%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> >>
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hTB
> >> 5UDaJoJfta9ohMG%2BrxCVJY34ANn10iSLP9iCHX3M%3D&reserved=0
> >>>> ernel.org%2Flinux-pci%2F20230113090350.1103494-1-
> >>>>
> >>
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cea6513dbf
> >>
> 4084b80ced208db3ce54133%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7
> >>
> C0%7C638170727558800720%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> >>
> LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> >>>> %7C%7C&sdata=jYgy%2Bxk84ZXZRVfqm0GCXoRnCTLMrX4zTfV%2Bs5
> Mm
> >> svo
> >>>> %3D&reserved=0
> >>>> - [RFC PATCH v2 0/7] Introduce a vringh accessor for IO memory
> >>>> link:
> >>>>
> >>
> https://lore.k/
> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7C80a237184ff4420a96de08db
> 3ff80d78%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63817410683
> 2489955%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LSP
> LeFzTYpUVTZpymoM00Es9kOpxlN90%2BDM6ceDJMFE%3D&reserved=0
> >> %2F&data=05%7C01%7Cfrank.li%40nxp.com%7Cff59a16f88c643913e3908
> db3
> >>
> ee91ca8%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C63817294315
> >>
> 3821831%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> >>
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hTB
> >> 5UDaJoJfta9ohMG%2BrxCVJY34ANn10iSLP9iCHX3M%3D&reserved=0
> >>>> ernel.org%2Fvirtualization%2F20230202090934.549556-1-
> >>>>
> >>
> mie%40igel.co.jp%2F&data=05%7C01%7CFrank.Li%40nxp.com%7Cea6513dbf
> >>
> 4084b80ced208db3ce54133%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7
> >>
> C0%7C638170727558800720%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
> >>
> LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> >>>> %7C%7C&sdata=K4El76GSAGtsWkNBXJK5%2Fn7flCN20eEMZpZYTX2WI
> Z0
> >> %3
> >>>> D&reserved=0
> >>>>
> >>>> First of this patchset is introduce a helper function to realize pci
> >>>> virtio function using PCIe endpoint framework. The second one is
> adding
> >>>> a missing definition for virtio pci header. The last one is for PCIe
> >>>> endpoint virtio console driver.
> >>>>
> >>>> This is tested on linux-20230406 and RCar S4 board as PCIe endpoint.
> >>>>
> >>>> Shunsuke Mie (3):
> >>>> PCI: endpoint: introduce a helper to implement pci ep virtio function
> >>>> virtio_pci: add a definition of queue flag in ISR
> >>>> PCI: endpoint: Add EP function driver to provide virtio-console
> >>>> functionality
> >>>>
> >>>> drivers/pci/endpoint/functions/Kconfig | 19 +
> >>>> drivers/pci/endpoint/functions/Makefile | 2 +
> >>>> drivers/pci/endpoint/functions/pci-epf-vcon.c | 554
> >> ++++++++++++++++++
> >>>> .../pci/endpoint/functions/pci-epf-virtio.c | 469 +++++++++++++++
> >>>> .../pci/endpoint/functions/pci-epf-virtio.h | 123 ++++
> >>>> include/uapi/linux/virtio_pci.h | 3 +
> >>>> 6 files changed, 1170 insertions(+)
> >>>> create mode 100644 drivers/pci/endpoint/functions/pci-epf-vcon.c
> >>>> create mode 100644 drivers/pci/endpoint/functions/pci-epf-virtio.c
> >>>> create mode 100644 drivers/pci/endpoint/functions/pci-epf-virtio.h
> >>>>
> >>>> --
> >>>> 2.25.1
