On Wed, Sep 23, 2020 at 03:35:56PM -0300, Jason Gunthorpe wrote: > On Wed, Sep 23, 2020 at 06:39:55AM +0100, Christoph Hellwig wrote: > > > I wonder if dma_virt_ops ignores the masking.. Still seems best to set > > > it consistently when using dma_virt_ops. > > > > dma_virt_ops doesn't look at the mask. But in retrospective > > dma_virt_ops has been a really bad idea. I'd much rather move the > > handling of non-physical devices back into the RDMA core in the long > > run. > > Doesn't that mean we need to put all the ugly IB DMA ops wrappers back > though? All the wrappers still exists, and as far a I can tell are used by the core and ULPs properly. > Why was dma_virt_ops such a bad idea? Because it isn't DMA, and not we need crappy workarounds like the one in the PCIe P2P code. It also enforces the same size for dma_addr_t and a pointer, which is not true in various cases. More importantly it forces a very strange design in the IB APIs (actually it seems the other way around - the weird IB Verbs APIs forced this decision, but it cements it in). For example most modern Mellanox cards can include immediate data in the command capsule (sorry NVMe terms, I'm pretty sure you guys use something else) that is just copied into the BAR using MMIO. But the IB API design still forces the ULP to dma map it, which is idiotic.
