On Wed, Jan 15, 2025 at 09:34:19AM -0400, Jason Gunthorpe wrote: > > Or do you mean some that don't have pages associated with them, and > > thus have pfn_valid fail on them? They still have a PFN, just not > > one that is valid to use in most of the Linux MM. > > He is talking about private interconnect hidden inside clusters of > devices. > > Ie the system may have many GPUs and those GPUs have their own private > interconnect between them. It is not PCI, and packets don't transit > through the CPU SOC at all, so the IOMMU is not involved. > > DMA can happen on that private interconnect, but from a Linux > perspective it is not DMA API DMA, and the addresses used to describe > it are not part of the CPU address space. The initiating device will > have a way to choose which path the DMA goes through when setting up > the DMA. So how is this in any way relevant to dma_buf which operates on a dma_addr_t right now and thus by definition can't be used for these?
