On Fri, Apr 21, 2017 at 05:05:41PM +0000, Jayachandran C wrote: > On Fri, Apr 21, 2017 at 10:48:15AM -0500, Bjorn Helgaas wrote: > > On Mon, Apr 17, 2017 at 12:47 PM, Jayachandran C > > <jnair@xxxxxxxxxxxxxxxxxx> wrote: > > > On Fri, Apr 14, 2017 at 09:00:06PM -0500, Bjorn Helgaas wrote: > > >> Could you collect "lspci -vv" output from this system? I'd like to > > >> archive that as background for this IOMMU issue and the ASPM tweaks I > > >> suspect we'll have to do. I *wish* we had more information about that > > >> VIA thing, because I suspect we could get rid of it if we had more > > >> details. > > > > > > The full logs are slightly large, so I have kept them at: > > > https://github.com/jchandra-cavm/thunderx2/blob/master/logs/ > > > The lspci -vv output is lspci-vv.txt and lspci -tvn output is lspci-tvn.txt > > > > > > The output is from 2 socket system, the cards are not on the first slot > > > like the example above, so the bus and device numbers are different. > > > > Can somebody with this system collect the "lspci -xxxx" output as well? > > > > I'm making some lspci changes to handle the PCI-to-PCIe bridge > > correctly, and I can use the "lspci -xxxx" output to create an lspci > > test case. > > [Sorry was AFK for a few days] > > I have updated the above directory with the log. Also tested your next branch > and it works fine on ThunderX2. Thanks! With regard to my lspci changes, they add "Slot-" here: 01:0a.0 PCI bridge: Broadcom Corporation Device 9039 ... - Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge, MSI 00 + Capabilities: [40] Express (v2) PCI/PCI-X to PCI-Express Bridge (Slot-), MSI 00 for all your PCI-to-PCIe bridges. I assume the "Slot-" is correct, i.e., the link is not connected to a slot, right? This comes from the "Slot Implemented" bit in the PCIe Capabilities Register. I did notice that all the Root Port devices claim to *not* be connected to slots, which doesn't seem right. For example, 12:00.0 PCI bridge: Broadcom Corporation Device 9084 Bus: primary=12, secondary=13, subordinate=14, sec-latency=0 Capabilities: [ac] Express (v2) Root Port (Slot-), MSI 00 13:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection It seems strange because the 12:00.0 Root Port looks like it probably *does* lead to a slot where the NIC is plugged in. Or is that NIC really soldered down? But I assume there are *some* PCIe slots, so at some of those Root Ports should advertise "Slot+" (which by itself does not imply hotplug support, if that's the concern). Bjorn
