On Wed, May 08, 2024 at 01:55:24PM -0400, Jim Quinlan wrote: > On Mon, May 6, 2024 at 7:20 PM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote: > ... > > As a user, how do I determine which setting to use? > > Using the "safe" mode will always work. In fact I considered making > this the default mode. > As I said, I cannot enumerate all of the reasons why one mode works > and one does not for a particular device+board+connector combo. The > HW folks have not really been forthcoming on the reasons as well. > > > Trial and error? If so, how do I identify the errors? > > Either PCIe link-up is not happening, or it is happening but the > device driver is non-functional and boot typically hangs. What I'm hearing is that it's trial and error. If we can't tell users how to figure out which mode to use, I think we have to explicitly say "try the modes in this order until you find one that works." That sucks, but if it's all we can do, I guess we don't have much choice, and we should just own up to it. There's no point in telling users "if your card drives CLKREQ# use X, but if not and it can tolerate out-of-spec T_CLRon timing, use Y" because nobody knows how to figure that out. And we can say which features are enabled in each mode so they aren't surprised, e.g., something like this: "default" -- The Root Port supports ASPM L0s, L1, L1 Substates, and Clock Power Management. This provides the best power savings but some devices may not work correctly because the Root Port doesn't comply with T_CLRon timing required for PCIe Mini Cards [1]. "no-l1ss" -- The Root Port supports ASPM L0s, L1 (but not L1 Substates), and Clock Power Management. [I assume there's some other Root Port defect that causes issues with some devices in this mode; I dunno. If we don't know exactly what it is, I guess we can't really say anything.] "safe" -- The Root Port supports ASPM L0, L1, L1 Substates, but not Clock Power Management. All devices should work in this mode. [1] PCIe Mini CEM r2.1, sec 3.2.5.2.2 (I'm not sure which features are *actually* enabled in each mode, I just guessed.) Bjorn