On Thu, Sep 24, 2020 at 07:13:25AM +0100, Christoph Hellwig wrote:
> On Thu, Sep 24, 2020 at 06:53:45AM +0100, Christoph Hellwig wrote:
> > Because it isn't DMA, and not we need crappy workarounds like the one
> > in the PCIe P2P code. It also enforces the same size for dma_addr_t
> > and a pointer, which is not true in various cases. More importantly
> > it forces a very strange design in the IB APIs (actually it seems the
> > other way around - the weird IB Verbs APIs forced this decision, but
> > it cements it in). For example most modern Mellanox cards can include
> > immediate data in the command capsule (sorry NVMe terms, I'm pretty
> > sure you guys use something else) that is just copied into the BAR
> > using MMIO. But the IB API design still forces the ULP to dma map
> > it, which is idiotic.
>
> FYI, here is the quick patch to kill of dma_virt_ops on top of the
> patch under discussion here. I think the diffstat alone speaks for
> itself, never mind the fact that we avoid indirect calls for the
> drivers that don't want the DMA mapping.
Yes looks reasonable, I can't recall why things went in the other
direction.
> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> index bf66b0622b49c8..f2621eaecb7211 100644
> +++ b/drivers/pci/p2pdma.c
> @@ -556,15 +556,6 @@ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients,
> return -1;
>
> for (i = 0; i < num_clients; i++) {
> -#ifdef CONFIG_DMA_VIRT_OPS
> - if (clients[i]->dma_ops == &dma_virt_ops) {
> - if (verbose)
> - dev_warn(clients[i],
> - "cannot be used for peer-to-peer DMA because the driver makes use of dma_virt_ops\n");
> - return -1;
> - }
> -#endif
We still need to block p2p for any of the non DMA devices. We can't
pass a p2p sgl into SW devices, they will try to memcpy from the MMIO
__iommem which is not allowed.
Now the check changes to !ib_device->dma_device in the callers
Jason
