Am 19.04.19 um 21:19 schrieb Bjorn Helgaas:
On Thu, Apr 18, 2019 at 01:58:59PM +0200, Christian König wrote:
A lot of root complexes can still do P2P even when PCI devices
don't share a common upstream bridge.
Start adding a whitelist and allow P2P if both participants are
attached to known good root complex.
Signed-off-by: Christian König <christian.koenig@xxxxxxx>
---
drivers/pci/p2pdma.c | 38 +++++++++++++++++++++++++++++++++++---
1 file changed, 35 insertions(+), 3 deletions(-)
diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
index c52298d76e64..212baaa7f93b 100644
--- a/drivers/pci/p2pdma.c
+++ b/drivers/pci/p2pdma.c
@@ -274,6 +274,31 @@ static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *pdev)
seq_buf_printf(buf, "%s;", pci_name(pdev));
}
+/*
+ * If we can't find a common upstream bridge take a look at the root complex and
+ * compare it to a whitelist of known good hardware.
+ */
+static bool root_complex_whitelist(struct pci_dev *dev)
+{
+ struct pci_host_bridge *host = pci_find_host_bridge(dev->bus);
+ struct pci_dev *root = pci_get_slot(host->bus, PCI_DEVFN(0, 0));
The fact that 00.0 is a host bridge and that its vendor/device ID
tells us whether peer-to-peer is supported between two *other* devices
(provider and client) is all totally vendor-specific. There's nothing
in the PCI specs that says the host bridge even has to exist as a PCI
device.
Crap, I hoped that at least the notion that the root complex is always
00.0 is standardized somewhere.
Is there any chance you could identify it by the Root Port
vendor/device instead of groping around for the 00:0 device? That
would have been simpler, so I suppose that doesn't work for some
reason.
I can give that a try. The main issue is that you got more different
root ports than root complexes, so our list is going to get longer this way.
I'm surprised that you don't need both the provider and the client to
figure this out. Wouldn't it would be possible to have two Root Ports
under host bridge A that could do peer-to-peer to each other, but not
to a Root Port under host bridge B? E.g., I'd think you would need
something like this:
At least for our hardware it's actually more likely that when devices
are connected to different host bridges that they can talk directly to
each other.
In other words when you have devices connected to the same host bridge
you need to have P2P support inside that host bridge.
But when you have different host bridges P2P accesses look to the host
bridge just like accesses coming from one of the CPU cores.
Regards,
Christian.
if (pci_find_host_bridge(a->bus) == pci_find_host_bridge(b->bus) &&
a->vendor == PCI_VENDOR_ID_AMD &&
b->vendor == PCI_VENDOR_ID_AMD &&
a->device == X &&
b->device == X)
return true;
return false;
But I guess that still implicitly relies on the notion that everything
under one PNP0A03 device is potentially connected via P2P, and I don't
think there's anything the PCI or ACPI specs that would support that.
Maybe we should add a pci_dev.p2p_group and make it so "a.p2p_group ==
b.p2p_group" means that P2P is possible between them. You'd have to
have a quirk for the root ports to initialize p2p_group using whatever
vendor-specific information you need, then everything under them could
inherit from the parent.
Well exactly that's not going to work.
E.g. it can be that devices A can talk to devices B and C, but on the
other hand B and C can't talk directly to each other.
Regards,
Christian.
+ unsigned short vendor, device;
+
+ if (!root)
+ return false;
+
+ vendor = root->vendor;
+ device = root->device;
+ pci_dev_put(root);
+
+ /* AMD ZEN host bridges can do peer to peer */
+ if (vendor == PCI_VENDOR_ID_AMD && device == 0x1450)
+ return true;
+
+ /* TODO: Extend that to a proper whitelist */
+ return false;
+}
+
/*
* Find the distance through the nearest common upstream bridge between
* two PCI devices.
@@ -317,13 +342,13 @@ static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *pdev)
* In this case, a list of all infringing bridge addresses will be
* populated in acs_list (assuming it's non-null) for printk purposes.
*/
-static int upstream_bridge_distance(struct pci_dev *a,
- struct pci_dev *b,
+static int upstream_bridge_distance(struct pci_dev *provider,
+ struct pci_dev *client,
struct seq_buf *acs_list)
{
+ struct pci_dev *a = provider, *b = client, *bb;
int dist_a = 0;
int dist_b = 0;
- struct pci_dev *bb = NULL;
int acs_cnt = 0;
/*
@@ -354,6 +379,13 @@ static int upstream_bridge_distance(struct pci_dev *a,
dist_a++;
}
+ /* Allow the connection if both devices are on a whitelisted root
+ * complex, but add an arbitary large value to the distance.
+ */
+ if (root_complex_whitelist(provider) &&
+ root_complex_whitelist(client))
+ return 0x1000 + dist_a + dist_b;
+
return -1;
check_b_path_acs:
--
2.17.1
