On 08/01/18 11:34 AM, Christoph Hellwig wrote:
But P2P is _not_ a factor of the dma_ops implementation at all,
it is something that happens behind the dma_map implementation.
Think about what the dma mapping routines do:
(a) translate from host address to bus addresses
and
(b) flush caches (in non-coherent architectures)
Both are obviously not needed for P2P transfers, as they never reach
the host.
Isn't pci_p2pdma_map_sg doing (a)? It's just translating from a host
address to a PCI bus address.
Very long term the IOMMUs under the ops will need to care about this,
so the wrapper is not an optimal place to put it - but I wouldn't
object if it gets it out of RDMA :)
Unless you have an IOMMU on your PCIe switch and not before/inside
the root complex that is not correct.
Per the ACS discussion, in the future we might want to implement "ACS
Direct Translated P2P" as Alex described. I expect it would be the IOMMU
that needs to set that up. So, I think, we also have the dma_map
implementations also doing something like:
(c) setup/manage any security permissions on mappings
Which P2P may at some point be concerned with.
Logan
