On Thu, Mar 22, 2018 at 10:57:32PM +0000, Stephen Bates wrote: > > I've seen the response that peers directly below a Root Port could not > > DMA to each other through the Root Port because of the "route to self" > > issue, and I'm not disputing that. > > Bjorn > > You asked me for a reference to RTS in the PCIe specification. As > luck would have it I ended up in an Irish bar with Peter Onufryk > this week at OCP Summit. We discussed the topic. It is not > explicitly referred to as "Route to Self" and it's certainly not > explicit (or obvious) but r6.2.8.1 of the PCIe 4.0 specification > discusses error conditions for virtual PCI bridges. One of these > conditions (given in the very first bullet in that section) applies > to a request that is destined for the same port it came in on. When > this occurs the request must be terminated as a UR. Thanks for that reference! I suspect figure 10-3 in sec 10.1.1 might also be relevant, although it's buried in the ATS section. It shows internal routing between functions of a multifunction device. That suggests that the functions should be able to DMA to each other without those transactions ever appearing on the link. Popping way up the stack, my original point was that I'm trying to remove restrictions on what devices can participate in peer-to-peer DMA. I think it's fairly clear that in conventional PCI, any devices in the same PCI hierarchy, i.e., below the same host-to-PCI bridge, should be able to DMA to each other. The routing behavior of PCIe is supposed to be compatible with conventional PCI, and I would argue that this effectively requires multi-function PCIe devices to have the internal routing required to avoid the route-to-self issue. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
