On 2019-06-24 7:46 a.m., Jason Gunthorpe wrote: > On Mon, Jun 24, 2019 at 09:31:26AM +0200, Christoph Hellwig wrote: >> On Thu, Jun 20, 2019 at 04:33:53PM -0300, Jason Gunthorpe wrote: >>>> My primary concern with this is that ascribes a level of generality >>>> that just isn't there for peer-to-peer dma operations. "Peer" >>>> addresses are not "DMA" addresses, and the rules about what can and >>>> can't do peer-DMA are not generically known to the block layer. >>> >>> ?? The P2P infrastructure produces a DMA bus address for the >>> initiating device that is is absolutely a DMA address. There is some >>> intermediate CPU centric representation, but after mapping it is the >>> same as any other DMA bus address. >>> >>> The map function can tell if the device pair combination can do p2p or >>> not. >> >> At the PCIe level there is no such thing as a DMA address, it all >> is bus address with MMIO and DMA in the same address space (without >> that P2P would have not chance of actually working obviously). But >> that bus address space is different per "bus" (which would be an >> root port in PCIe), and we need to be careful about that. > > Sure, that is how dma_addr_t is supposed to work - it is always a > device specific value that can be used only by the device that it was > created for, and different devices could have different dma_addr_t > values for the same memory. > > So when Logan goes and puts dma_addr_t into the block stack he must > also invert things so that the DMA map happens at the start of the > process to create the right dma_addr_t early. Yes, that's correct. The intent was to invert it so the dma_map could happen at the start of the process so that P2PDMA code could be called with all the information it needs to make it's decision on how to map; without having to hook into the mapping process of every driver that wants to participate. Logan
