On Wed, Oct 16, 2024 at 09:41:03AM -0700, Christoph Hellwig wrote: > On Wed, Oct 16, 2024 at 12:44:28PM -0300, Jason Gunthorpe wrote: > > > We are talking about P2P memory here. How do you manage to get a page > > > that dma_map_page can be used on? All P2P memory needs to use the P2P > > > aware dma_map_sg as the pages for P2P memory are just fake zone device > > > pages. > > > > FWIW, I've been expecting this series to be rebased on top of Leon's > > new DMA API series so it doesn't have this issue.. > > That's not going to make a difference at this level. I'm not sure what you are asking then. Patch 2 does pci_p2pdma_add_resource() and so a valid struct page with a P2P ZONE_DEVICE type exists, and that gets returned back to the hmm/odp code. Today odp calls dma_map_page() which only works by chance in limited cases. With Leon's revision it will call hmm_dma_map_pfn() -> dma_iova_link() which does call pci_p2pdma_map_type() and should do the right thing. > > I'm guessing they got their testing done so far on a system without an > > iommu translation? > > IOMMU or not doens't matter much for P2P. The important difference is > through the host bridge or through a switch. dma_map_page will work > for P2P through the host brige (assuming the host bridge even support > it as it also lacks the error handling for when not), but it lacks the > handling for P2P through a switch. On most x86 systems the BAR/bus address of the P2P memory is the same as the CPU address, so without an IOMMU translation dma_map_page() will return the CPU/host physical address which is the same as the BAR/bus address and that will take the P2P switch path for testing. Jason
