hi shunsuke, all, what about vfio-user + qemu? qemu already has libvfio-user as submodule. there is ongoing work to add qemu vfio-user client functionality. adding people involved to loop, not sure if i forgot somebody. regards jiri On Tue, Oct 03, 2023 at 01:56:03PM +0900, Shunsuke Mie wrote: > Hi Vaishnav, > > On 2023/09/26 21:40, Vaishnav Achath wrote: > > Hi Kishon, all, > > > > On 26/09/23 15:17, Shunsuke Mie wrote: > > > On 2023/09/21 18:11, Kishon Vijay Abraham I wrote: > > > > +Vaishnav > > > > > > > > Hi Shunsuke, > > > > > > > > On 8/18/2023 7:16 PM, Shunsuke Mie wrote: > > > > > Hi all, > > > > > > > > > > We are proposing to add a new test syste to Linux for PCIe Endpoint. That > > > > > can be run on QEMU without real hardware. At present, partially we have > > > > > confirmed that pci-epf-test is working, but it is not yet complete. > > > > > However, we would appreciate your comments on the architecture design. > > > > > > > > > > # Background > > > > > The background is as follows. > > > > > > > > > > PCI Endpoint function driver is implemented using the PCIe Endpoint > > > > > framework, but it requires physical boards for testing, and it is difficult > > > > > to test sufficiently. In order to find bugs and hardware-dependent > > > > > implementations early, continuous testing is required. Since it is > > > > > difficult to automate tests that require hardware, this RFC proposes a > > > > > virtual environment for testing PCI endpoint function drivers. > > > > This would be quite useful and thank you for attempting it! I would like to > > > > compare other mechanisms available in-addition to QEMU before going with the > > > > QEMU approach. > > > I got it. I'll make a table to compare some methods that includes greybus to > > > realize this emulation environment. > > > > > > > > > Best, > > > > > > Shunsuke > > > > > > > Though I don't understand this fully, Looking at > > > > https://osseu2023.sched.com/event/1OGk8/emulating-devices-in-linux-using-greybus-subsystem-vaishnav-mohandas-achath-texas-instruments, Vaishnav seems to solve the same problem using greybus for multiple type s of devices. > > > > > > > > Vaishnav, we'd wait for your OSS presentation but do you have any initial > > > > thoughts on how greybus could be used to test PCIe endpoint drivers? > > > > > > Apologies for the delay, I don't think greybus can be used for PCIe testing as > > there is no greybus equivalent for PCIe[1], it can only be used for relatively > > simpler devices today, I guess roadtest(UML based)[2] could be an alternative in > > this case. > > Thank you for your comment. > > To my understanding, the roadtest uses UML and it interact with hardware > model written in python to do testing. This would be grate for automated > testing to test drivers and subsystems. > > For this PCIe endpoint, I think we need to hosts, one that works as a PCIe > endpoint and one that is a PCIe Root Complex to it. Is it possible to > realize the system? > like: > UML + PCIe endpoint function driver <-> python HW model (PCI Endpoint > controller) <-> UML + pci driver for the function > > > As another option, I'm considering the feasibility of dummy PCIe EPC driver. > It works as a PCIe EPC device in kernel and show pci device according to the > PCIe endpoint function driver to the same host. so It could be realize a > single host and test the function driver. > > > Best, > > Shunsuke > > > 1 - > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/staging/greybus > > 2 - https://lore.kernel.org/lkml/YjN1ksNGujV611Ka@xxxxxxxxxxxxx/ > > > > Thanks and Regards, > > Vaishnav > > > > > > Thanks, > > > > Kishon > > > > > > > > > # Architecture > > > > > The overview of the architecture is as follows. > > > > > > > > > > Guest 1 Guest 2 > > > > > +-------------------------+ +----------------------------+ > > > > > | Linux kernel | | Linux kernel | > > > > > | | | | > > > > > | PCI EP function driver | | | > > > > > | (e.g. pci-epf-test) | | | > > > > > |-------------------------| | PCI Device Driver | > > > > > | (2) QEMU EPC Driver | | (e.g. pci_endpoint_test) | > > > > > +-------------------------+ +----------------------------+ > > > > > +-------------------------+ +----------------------------+ > > > > > | QEMU | | QEMU | > > > > > |-------------------------| |----------------------------| > > > > > | (1) QEMU PCI EPC Device *----* (3) QEMU EPF Bridge Device | > > > > > +-------------------------+ +----------------------------+ > > > > > > > > > > At present, it is designed to work guests only on the same host, and > > > > > communication is done through Unix domain sockets. > > > > > > > > > > The three parts shown in the figure were introduced this time. > > > > > > > > > > (1) QEMU PCI Endpoint Controller(EPC) Device > > > > > PCI Endpoint Controller implemented as QEMU PCI device. > > > > > (2) QEMU PCI Endpoint Controller(EPC) Driver > > > > > Linux kernel driver that drives the device (1). It registers a epc device > > > > > to linux kernel and handling each operations for the epc device. > > > > > (3) QEMU PCI Endpoint function(EPF) Bridge Device > > > > > QEMU PCI device that cooperates with (1) and performs accesses to pci > > > > > configuration space, BAR and memory space to communicate each guests, and > > > > > generates interruptions to the guest 1. > > > > > > > > > > Each projects are: > > > > > (1), (3) https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1 > > > > > <https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1> > > > > > files: hw/misc/{qemu-epc.{c,h}, epf-bridge.c} > > > > > (2) https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc > > > > > <https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc> > > > > > files: drivers/pci/controller/pcie-qemu-ep.c > > > > > > > > > > # Protocol > > > > > > > > > > PCI, PCIe has a layer structure that includes Physical, Data Lane and > > > > > Transaction. The communicates between the bridge(3) and controller (1) > > > > > mimic the Transaction. Specifically, a protocol is implemented for > > > > > exchanging fd for communication protocol version check and communication, > > > > > in addition to the interaction equivalent to PCIe Transaction Layer Packet > > > > > (Read and Write of I/O, Memory, Configuration space and Message). In my > > > > > mind, we need to discuss the communication mor. > > > > > > > > > > We also are planning to post the patch set after the code is organized and > > > > > the protocol discussion is matured. > > > > > > > > > > Best regards, > > > > > Shunsuke
