On Tue, Apr 5, 2022 at 10:35 AM Zhi Li <lznuaa@xxxxxxxxx> wrote: > > On Tue, Apr 5, 2022 at 5:34 AM Kishon Vijay Abraham I <kishon@xxxxxx> wrote: > > > > Hi Frank Li, > > > > On 22/02/22 9:53 pm, Frank Li wrote: > > > This implement NTB function for PCIe EP to RC connections. > > > The existed ntb epf need two PCI EPs and two PCI Host. > > > > As I had earlier mentioned in [1], IMHO ideal solution would be build on virtio > > layer instead of trying to build on NTB layer (which is specific to RC<->RC > > communication). > > > > Are there any specific reasons for not taking that path? > > 1. EP side work as vHOST mode. vHost suppose access all memory of virtual io. > But there are only map windows on the EP side to access RC side > memory. You have to move > map windows for each access. It is quite low efficiency. > > 2. So far as I know, virtio is still not DMA yet. CPU access PCI > can't generate longer PCI TLP, > So the speed is quite slow. NTB already has DMA support. If you use > system level DMA, > no change is needed at NTB level. If we want to use a PCI controller > embedded DMA, some small > changes need if based on my other Designware PCI eDMA patches, which > are under review. > > 3. All the major data transfer of NTB is using write. Because TLP > write needn't wait for complete, write > performance is better than reading. On our platform, write > performance is about 10% better than read. > > Frank Any Comments or rejection? @Kishon Vijay Abraham I best regards Frank Li > > > > > Thanks, > > Kishon > > > > [1] -> https://lore.kernel.org/r/459745d1-9fe7-e792-3532-33ee9552bc4d@xxxxxx > > > > > > This just need EP to RC connections. > > > > > > ┌────────────┐ ┌─────────────────────────────────────┐ > > > │ │ │ │ > > > ├────────────┤ │ ┌──────────────┤ > > > │ NTB │ │ │ NTB │ > > > │ NetDev │ │ │ NetDev │ > > > ├────────────┤ │ ├──────────────┤ > > > │ NTB │ │ │ NTB │ > > > │ Transfer │ │ │ Transfer │ > > > ├────────────┤ │ ├──────────────┤ > > > │ │ │ │ │ > > > │ PCI NTB │ │ │ │ > > > │ EPF │ │ │ │ > > > │ Driver │ │ │ PCI Virtual │ > > > │ │ ├───────────────┐ │ NTB Driver │ > > > │ │ │ PCI EP NTB │◄────►│ │ > > > │ │ │ FN Driver │ │ │ > > > ├────────────┤ ├───────────────┤ ├──────────────┤ > > > │ │ │ │ │ │ > > > │ PCI BUS │ ◄─────► │ PCI EP BUS │ │ Virtual PCI │ > > > │ │ PCI │ │ │ BUS │ > > > └────────────┘ └───────────────┴──────┴──────────────┘ > > > PCI RC PCI EP > > > > > > > > > > > > Frank Li (4): > > > PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address > > > NTB: epf: Allow more flexibility in the memory BAR map method > > > PCI: endpoint: Support NTB transfer between RC and EP > > > Documentation: PCI: Add specification for the PCI vNTB function device > > > > > > Documentation/PCI/endpoint/index.rst | 2 + > > > .../PCI/endpoint/pci-vntb-function.rst | 126 ++ > > > Documentation/PCI/endpoint/pci-vntb-howto.rst | 167 ++ > > > drivers/ntb/hw/epf/ntb_hw_epf.c | 48 +- > > > .../pci/controller/dwc/pcie-designware-ep.c | 10 +- > > > drivers/pci/endpoint/functions/Kconfig | 11 + > > > drivers/pci/endpoint/functions/Makefile | 1 + > > > drivers/pci/endpoint/functions/pci-epf-vntb.c | 1424 +++++++++++++++++ > > > 8 files changed, 1775 insertions(+), 14 deletions(-) > > > create mode 100644 Documentation/PCI/endpoint/pci-vntb-function.rst > > > create mode 100644 Documentation/PCI/endpoint/pci-vntb-howto.rst > > > create mode 100644 drivers/pci/endpoint/functions/pci-epf-vntb.c > > >
