Hi Kishon, > Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express Non- > Transparent Bridge > > Hi Sherry, > > On 11/11/20 8:19 am, Sherry Sun wrote: > > Hi Kishon, > > > >> Subject: Re: [PATCH v7 15/18] NTB: Add support for EPF PCI-Express > >> Non- Transparent Bridge > >> > >> Hi Sherry, Arnd, > >> > >> On 10/11/20 8:29 pm, Arnd Bergmann wrote: > >>> On Tue, Nov 10, 2020 at 3:20 PM Kishon Vijay Abraham I > >>> <kishon@xxxxxx> > >> wrote: > >>>> On 10/11/20 7:55 am, Sherry Sun wrote: > >>> > >>>>> But for VOP, only two boards are needed(one board as host and one > >>>>> board as card) to realize the communication between the two > >>>>> systems, > >> so my question is what are the advantages of using NTB? > >>>> > >>>> NTB is a bridge that facilitates communication between two > >>>> different systems. So it by itself will not be source or sink of > >>>> any data unlike a normal EP to RP system (or the VOP) which will be > >>>> source or sink > >> of data. > >>>> > >>>>> Because I think the architecture of NTB seems more complicated. > >>>>> Many > >> thanks! > >>>> > >>>> yeah, I think it enables a different use case all together. > >>>> Consider you have two x86 HOST PCs (having RP) and they have to be > >>>> communicate using PCIe. NTB can be used in such cases for the two > >>>> x86 PCs to communicate with each other over PCIe, which wouldn't be > >>>> possible > >> without NTB. > >>> > >>> I think for VOP, we should have an abstraction that can work on > >>> either NTB or directly on the endpoint framework but provide an > >>> interface that then lets you create logical devices the same way. > >>> > >>> Doing VOP based on NTB plus the new NTB_EPF driver would also work > >>> and just move the abstraction somewhere else, but I guess it would > >>> complicate setting it up for those users that only care about the > >>> simpler endpoint case. > >> > >> I'm not sure if you've got a chance to look at [1], where I added > >> support for RP<->EP system both running Linux, with EP configured > >> using Linux EP framework (as well as HOST ports connected to NTB > >> switch, patches 20 and 21, that uses the Linux NTB framework) to > >> communicate using virtio over PCIe. > >> > > > > I saw your patches at [1], here you take a rpmsg as an example to > > communicate between two SoCs using PCIe RC<->EP and HOST1-NTB- > HOST2 for different usercases. > > The VOP code works under the PCIe RC<->EP framework, which means that > > we can also make VOP works under the Linux NTB framework, just like the > rpmsg way you did here, right? > > Does VOP really work with EP framework? At-least whatever is in upstream > doesn't seem to indicate so. > We did write a pci_epf driver to support VOP, looks like pci-epf-test.c, and it works well. So certainly VOP can work with EP framework. But it's a pity that the VOP related codes has been deleted before we send the pci_epf_vop driver patches to upstream. > The NTB framework lets one host with RP port to communicate with another > host with RP port. > > The EP Framework lets one device with EP port to communicate with a host > with RP port. > > Rest of the trick should be how you tie them together. > > PCIe framework creates "pci_device" for each of the devices it enumerates. > NTB framework works on this pci_device to communicate with the remote > host using PCIe bridge. The remote host will use NTB framework as well. > > So depends on what interfaces VOP device provides you can use either NTB > framework or EP framework. If it's going to connect two different devices in > turn creating pci_device on each of the systems, then you can use NTB > framework. > Thanks for your detailed explanation! It is clear. I think maybe VOP is suitable for the basic PCIe framework instead of NTB. Best regards Sherry > Regards > Kishon
