Hi Bjorn, JC, On 04/13/2017 08:19 PM, Bjorn Helgaas wrote: > I tentatively applied both patches to pci/host-thunder for v4.12. Thanks for that :) > However, I am concerned about the topology here: Various feedback has been provided on this one over the past $time. In addition, I have requested that this serve as an example of why we need a more complete PCIe integration guide for ARMv8. It's on the list of things for my intended opus magnum on the topic ;) > On Thu, Apr 13, 2017 at 08:30:45PM +0000, Jayachandran C wrote: >> On Cavium ThunderX2 arm64 SoCs (called Broadcom Vulcan earlier), the >> PCI topology is slightly unusual. For a multi-node system, it looks >> like: >> >> 00:00.0 [PCI] bridge to [bus 01-1e] >> 01:0a.0 [PCI-PCIe bridge, type 8] bridge to [bus 02-04] >> 02:00.0 [PCIe root port, type 4] bridge to [bus 03-04] (XLATE_ROOT) >> 03:00.0 PCIe Endpoint > > A root port normally has a single PCIe link leading downstream. In integration terms, there's always a bus on the other side of the RC. It's just usually a processor local bus of some kind on a proprietary interconnect, but there's always something there. The issue is when you can see it as PCIe and it's not through a transparent glue bridge. I had originally hoped that the ECAM could be hacked up so that we would first walk the topology at the 02:00.0 as a root and not see what's above it BUT there are other device attachments up there for the on-SoC devices and I think we really intend to use those. Bottom line in my opinion is document this case, use it as a learning example, and move forward. This has been useful in justifying why we need better integration documentation from the server community. And in particular from the OS vendors, of which I guess we can allude to their being more than Linux interested in ARM server these days ;) Jon. -- Computer Architect | Sent from my Fedora powered Ryzen
