On Tue, Apr 5, 2022 at 5:34 AM Kishon Vijay Abraham I <kishon@xxxxxx> wrote: > > Hi Frank Li, > > On 22/02/22 9:53 pm, Frank Li wrote: > > This implement NTB function for PCIe EP to RC connections. > > The existed ntb epf need two PCI EPs and two PCI Host. > > As I had earlier mentioned in [1], IMHO ideal solution would be build on virtio > layer instead of trying to build on NTB layer (which is specific to RC<->RC > communication). > > Are there any specific reasons for not taking that path? 1. EP side work as vHOST mode. vHost suppose access all memory of virtual io. But there are only map windows on the EP side to access RC side memory. You have to move map windows for each access. It is quite low efficiency. 2. So far as I know, virtio is still not DMA yet. CPU access PCI can't generate longer PCI TLP, So the speed is quite slow. NTB already has DMA support. If you use system level DMA, no change is needed at NTB level. If we want to use a PCI controller embedded DMA, some small changes need if based on my other Designware PCI eDMA patches, which are under review. 3. All the major data transfer of NTB is using write. Because TLP write needn't wait for complete, write performance is better than reading. On our platform, write performance is about 10% better than read. Frank > > Thanks, > Kishon > > [1] -> https://lore.kernel.org/r/459745d1-9fe7-e792-3532-33ee9552bc4d@xxxxxx > > > > This just need EP to RC connections. > > > > ┌────────────┐ ┌─────────────────────────────────────┐ > > │ │ │ │ > > ├────────────┤ │ ┌──────────────┤ > > │ NTB │ │ │ NTB │ > > │ NetDev │ │ │ NetDev │ > > ├────────────┤ │ ├──────────────┤ > > │ NTB │ │ │ NTB │ > > │ Transfer │ │ │ Transfer │ > > ├────────────┤ │ ├──────────────┤ > > │ │ │ │ │ > > │ PCI NTB │ │ │ │ > > │ EPF │ │ │ │ > > │ Driver │ │ │ PCI Virtual │ > > │ │ ├───────────────┐ │ NTB Driver │ > > │ │ │ PCI EP NTB │◄────►│ │ > > │ │ │ FN Driver │ │ │ > > ├────────────┤ ├───────────────┤ ├──────────────┤ > > │ │ │ │ │ │ > > │ PCI BUS │ ◄─────► │ PCI EP BUS │ │ Virtual PCI │ > > │ │ PCI │ │ │ BUS │ > > └────────────┘ └───────────────┴──────┴──────────────┘ > > PCI RC PCI EP > > > > > > > > Frank Li (4): > > PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address > > NTB: epf: Allow more flexibility in the memory BAR map method > > PCI: endpoint: Support NTB transfer between RC and EP > > Documentation: PCI: Add specification for the PCI vNTB function device > > > > Documentation/PCI/endpoint/index.rst | 2 + > > .../PCI/endpoint/pci-vntb-function.rst | 126 ++ > > Documentation/PCI/endpoint/pci-vntb-howto.rst | 167 ++ > > drivers/ntb/hw/epf/ntb_hw_epf.c | 48 +- > > .../pci/controller/dwc/pcie-designware-ep.c | 10 +- > > drivers/pci/endpoint/functions/Kconfig | 11 + > > drivers/pci/endpoint/functions/Makefile | 1 + > > drivers/pci/endpoint/functions/pci-epf-vntb.c | 1424 +++++++++++++++++ > > 8 files changed, 1775 insertions(+), 14 deletions(-) > > create mode 100644 Documentation/PCI/endpoint/pci-vntb-function.rst > > create mode 100644 Documentation/PCI/endpoint/pci-vntb-howto.rst > > create mode 100644 drivers/pci/endpoint/functions/pci-epf-vntb.c > >
