On Tue, Jul 30, 2019 at 10:35:31AM -0600, Logan Gunthorpe wrote: > Here's v2 of the patchset. It doesn't sound like there's anything > terribly controversial here so this version is mostly just some > cleanup changes for clarity. > > Changes in v2: > * Rebase on v5.3-rc2 (No changes) > * Re-introduce the private pagemap structure and move the p2p-specific > elements out of the commond dev_pagemap (per Christoph) > * Use flags instead of bool in the whitelist (per Jason) > * Only store the mapping type in the xarray (instead of the distance > with flags) such that a function can return the mapping method > with a switch statement to decide how to map. (per Christoph) > * Drop find_parent_pci_dev() on the fast path and rely on the fact > that the struct device passed to the mapping functions *must* be > a PCI device and convert it directly. (per suggestions from > Christoph and Jason) > * Collected Christian's Reviewed-by's > -- > > As discussed on the list previously, in order to fully support the > whitelist Christian added with the IOMMU, we must ensure that we > map any buffer going through the IOMMU with an aprropriate dma_map > call. This patchset accomplishes this by cleaning up the output of > upstream_bridge_distance() to better indicate the mapping requirements, > caching these requirements in an xarray, then looking them up at map > time and applying the appropriate mapping method. > > After this patchset, it's possible to use the NVMe-of P2P support to > transfer between devices without a switch on the whitelisted root > complexes. A couple Intel device I have tested this on have also > been added to the white list. > > Most of the changes are contained within the p2pdma.c, but there are > a few minor touches to other subsystems, mostly to add support > to call an unmap function. > > The final patch in this series demonstrates a possible > pci_p2pdma_map_resource() function that I expect Christian will need > but does not have any users at this time so I don't intend for it to be > considered for merging. I don't see pci_p2pdma_map_resource() in any of these patches. I tentatively applied these to pci/p2pdma with minor typographical updates (below), but I'll update the branch if necessary. > This patchset is based on 5.3-rc2 and a git branch is available here: > > https://github.com/sbates130272/linux-p2pmem/ p2pdma_rc_map_v2 > > -- > > Logan Gunthorpe (14): > PCI/P2PDMA: Introduce private pagemap structure > PCI/P2PDMA: Add the provider's pci_dev to the pci_p2pdma_pagemap > struct > PCI/P2PDMA: Add constants for not-supported result > upstream_bridge_distance() > PCI/P2PDMA: Factor out __upstream_bridge_distance() > PCI/P2PDMA: Apply host bridge white list for ACS > PCI/P2PDMA: Factor out host_bridge_whitelist() > PCI/P2PDMA: Add whitelist support for Intel Host Bridges > PCI/P2PDMA: Add attrs argument to pci_p2pdma_map_sg() > PCI/P2PDMA: Introduce pci_p2pdma_unmap_sg() > PCI/P2PDMA: Factor out __pci_p2pdma_map_sg() > PCI/P2PDMA: Store mapping method in an xarray > PCI/P2PDMA: dma_map P2PDMA map requests that traverse the host bridge > PCI/P2PDMA: No longer require no-mmu for host bridge whitelist > PCI/P2PDMA: Update documentation for pci_p2pdma_distance_many() > > drivers/infiniband/core/rw.c | 6 +- > drivers/nvme/host/pci.c | 10 +- > drivers/pci/p2pdma.c | 361 +++++++++++++++++++++++++---------- > include/linux/memremap.h | 1 - > include/linux/pci-p2pdma.h | 28 ++- > 5 files changed, 296 insertions(+), 110 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index ac6b599a10ef..afa42512e604 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -442,17 +442,17 @@ static int map_types_idx(struct pci_dev *client) * port of the switch, to the common upstream port, back up to the second * downstream port and then to Device B. * - * Any two devices that cannot communicate using p2pdma will return the distance - * with the flag P2PDMA_NOT_SUPPORTED. + * Any two devices that cannot communicate using p2pdma will return the + * distance with the flag P2PDMA_NOT_SUPPORTED. * * Any two devices that have a data path that goes through the host bridge * will consult a whitelist. If the host bridges are on the whitelist, - * then the distance will be returned with the flag P2PDMA_THRU_HOST_BRIDGE set. + * the distance will be returned with the flag P2PDMA_THRU_HOST_BRIDGE set. * If either bridge is not on the whitelist, the flag P2PDMA_NOT_SUPPORTED will * be set. * * If a bridge which has any ACS redirection bits set is in the path - * then this functions will flag the result with P2PDMA_ACS_FORCES_UPSTREAM. + * this function will flag the result with P2PDMA_ACS_FORCES_UPSTREAM. * In this case, a list of all infringing bridge addresses will be * populated in acs_list (assuming it's non-null) for printk purposes. */ @@ -529,8 +529,8 @@ static int upstream_bridge_distance_warn(struct pci_dev *provider, * choice). * * "compatible" means the provider and the clients are either all behind - * the same PCI root port or the host bridge connected to each of the devices - * are is listed in the 'pci_p2pdma_whitelist'. + * the same PCI root port or the host bridges connected to each of the devices + * are listed in the 'pci_p2pdma_whitelist'. */ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, int num_clients, bool verbose) @@ -850,7 +850,7 @@ static int __pci_p2pdma_map_sg(struct pci_p2pdma_pagemap *p2p_pgmap, * @sg: scatter list to map * @nents: elements in the scatterlist * @dir: DMA direction - * @attrs: dma attributes passed to dma_map_sg() (if called) + * @attrs: DMA attributes passed to dma_map_sg() (if called) * * Scatterlists mapped with this function should be unmapped using * pci_p2pdma_unmap_sg_attrs(). @@ -888,7 +888,7 @@ EXPORT_SYMBOL_GPL(pci_p2pdma_map_sg_attrs); * @sg: scatter list to map * @nents: number of elements returned by pci_p2pdma_map_sg() * @dir: DMA direction - * @attrs: dma attributes passed to dma_unmap_sg() (if called) + * @attrs: DMA attributes passed to dma_unmap_sg() (if called) */ void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs)
