On Tue, Jun 14, 2011 at 12:54 AM, Rolf Eike Beer <eike-kernel@xxxxxxxxx> wrote: > Jon Mason wrote: >> There is a sizable performance boost for having the largest possible >> maximum payload size on each PCI-E device. However, the maximum payload >> size must be uniform on a given PCI-E fabric, and each device, bridge, >> and root port can have a different max size. To find and configure the >> optimal MPS settings, one must walk the fabric and determine the largest >> MPS available on all the devices on the given fabric. > > So you are basically telling me that if I have one device that only can do 128 > byte no other device can do more? Even if I don't do device2device transfers? > That's odd. Not only have I worked with such configurations on many boards for > years, but it basically renders that totally useless as all motherboards I got > my fingers on have on-chip devices that only support 128 bytes. > > Or did you mean this must be uniform on one bus (since every physical slot has > usually it's own bus)? It's per fabric, which is everything under a root port. From the x86 motherboards I have access to, all hotplug slots are directly connected to a root port. So this is a moot point for those. For fabrics with multiple devices attached (i.e., the integrated devices on an x86 motherboard), the MPS for the root port, all bridges/switches, and devices must be uniform. Otherwise, a device with a smaller MPS could get a payload too large and cause an unrecoverable bus error (since there is no buffering or fragmentation in the bridges/switches). For that, any device with a large MPS will be hindered by the group. Hotplug to these fabrics only exacerbates the issue, but I do not know if such a thing exists in the "real world". Unfortunately, this means that one must be aware of the bus limitations when adding devices. If it is helpful, I could add a dmesg entry specifying the size of limitation. > >> + if (dev->is_hotplug_bridge) >> + smpss = 0; >> + >> + if (*smpss > dev->pcie_mpss) >> + *smpss = dev->pcie_mpss; >> + } > > The first one is wrong. It must be either "*smpss = 0". How it currently is you > only crash the machine with a NULL deref. Good catch. As I said, the hotplug is completely untested (as I don't have the right hardware to test it). Thanks, Jon > > Eike -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
