+Vaishnav
Hi Shunsuke,
On 8/18/2023 7:16 PM, Shunsuke Mie wrote:
Hi all,
We are proposing to add a new test syste to Linux for PCIe Endpoint. That
can be run on QEMU without real hardware. At present, partially we have
confirmed that pci-epf-test is working, but it is not yet complete.
However, we would appreciate your comments on the architecture design.
# Background
The background is as follows.
PCI Endpoint function driver is implemented using the PCIe Endpoint
framework, but it requires physical boards for testing, and it is difficult
to test sufficiently. In order to find bugs and hardware-dependent
implementations early, continuous testing is required. Since it is
difficult to automate tests that require hardware, this RFC proposes a
virtual environment for testing PCI endpoint function drivers.
This would be quite useful and thank you for attempting it! I would like
to compare other mechanisms available in-addition to QEMU before going
with the QEMU approach.
Though I don't understand this fully, Looking at
https://osseu2023.sched.com/event/1OGk8/emulating-devices-in-linux-using-greybus-subsystem-vaishnav-mohandas-achath-texas-instruments,
Vaishnav seems to solve the same problem using greybus for multiple type
s of devices.
Vaishnav, we'd wait for your OSS presentation but do you have any
initial thoughts on how greybus could be used to test PCIe endpoint drivers?
Thanks,
Kishon
# Architecture
The overview of the architecture is as follows.
Guest 1 Guest 2
+-------------------------+ +----------------------------+
| Linux kernel | | Linux kernel |
| | | |
| PCI EP function driver | | |
| (e.g. pci-epf-test) | | |
|-------------------------| | PCI Device Driver |
| (2) QEMU EPC Driver | | (e.g. pci_endpoint_test) |
+-------------------------+ +----------------------------+
+-------------------------+ +----------------------------+
| QEMU | | QEMU |
|-------------------------| |----------------------------|
| (1) QEMU PCI EPC Device *----* (3) QEMU EPF Bridge Device |
+-------------------------+ +----------------------------+
At present, it is designed to work guests only on the same host, and
communication is done through Unix domain sockets.
The three parts shown in the figure were introduced this time.
(1) QEMU PCI Endpoint Controller(EPC) Device
PCI Endpoint Controller implemented as QEMU PCI device.
(2) QEMU PCI Endpoint Controller(EPC) Driver
Linux kernel driver that drives the device (1). It registers a epc device
to linux kernel and handling each operations for the epc device.
(3) QEMU PCI Endpoint function(EPF) Bridge Device
QEMU PCI device that cooperates with (1) and performs accesses to pci
configuration space, BAR and memory space to communicate each guests, and
generates interruptions to the guest 1.
Each projects are:
(1), (3) https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1
<https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1>
files: hw/misc/{qemu-epc.{c,h}, epf-bridge.c}
(2) https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc
<https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc>
files: drivers/pci/controller/pcie-qemu-ep.c
# Protocol
PCI, PCIe has a layer structure that includes Physical, Data Lane and
Transaction. The communicates between the bridge(3) and controller (1)
mimic the Transaction. Specifically, a protocol is implemented for
exchanging fd for communication protocol version check and communication,
in addition to the interaction equivalent to PCIe Transaction Layer Packet
(Read and Write of I/O, Memory, Configuration space and Message). In my
mind, we need to discuss the communication mor.
We also are planning to post the patch set after the code is organized and
the protocol discussion is matured.
Best regards,
Shunsuke
