[+cc Mika, Sathy, Rafael, David, Ilpo, Ricky, Mario, linux-pci] On Sat, Nov 04, 2023 at 10:13:24AM -0700, Kenneth R. Crudup wrote: > > I have a Dell XPS-9320 with an Alderlake chipset, and the NVMe behind a > VMD device: > > ---- > [ 0.127342] smpboot: CPU0: 12th Gen Intel(R) Core(TM) i7-1280P (family: 0x6, model: 0x9a, stepping: 0x3) > ---- > 0000:00:0e.0 0104: 8086:467f > Subsystem: 1028:0af3 > Flags: bus master, fast devsel, latency 0, IOMMU group 9 > Memory at 603c000000 (64-bit, non-prefetchable) [size=32M] > Memory at 72000000 (32-bit, non-prefetchable) [size=32M] > a7152be79b6 Memory at 6040100000 (64-bit, non-prefetchable) [size=1M] > Capabilities: <access denied> > Kernel driver in use: vmd > ---- > > The only release kernel that was able to get this laptop to fully get into > low-power (unfortunately only s0ix) was the Ubuntu-6.2.0- ... series from > Ubuntu > (remote git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/lunar). > > I'd bisected it to the following commits (in this order): > > 4ff116d0d5fd PCI/ASPM: Save L1 PM Substates Capability for suspend/resume > 5e85eba6f50d PCI/ASPM: Refactor L1 PM Substates Control Register programming > 1a0102a08f20 UBUNTU: SAUCE: PCI/ASPM: Enable ASPM for links under VMD domain > 47c7bfd31514 UBUNTU: SAUCE: PCI/ASPM: Enable LTR for endpoints behind VMD > 154d48da2c57 UBUNTU: SAUCE: vmd: fixup bridge ASPM by driver name instead Thanks for these. You don't happen to have URLs for those Ubuntu commits, do you? E.g., https://git.kernel.org/linus/4ff116d0d5fd (which was reverted by a7152be79b62 ("Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume"")). > Without the patches I never see Pkg%PC8 or higher(? lower?), nor i915 states > DC5/6, all necssary for SYS%LPI/CPU%LPI. I've attached a little script I use > alongside turbostat for verifying low-power operation (and also for seeing > what chipset subsystem may be preventing it). > > The first two are in Linus' trees, but were reverted (4ff116d0d5fd in > a7152be79b6, 5e85eba6f50d in ff209ecc376a). The last three come from Ubuntu's > Linux trees (see remote spec above). The first two remain reverted in the > Ubuntu trees, but if I put them back, I get increased power savings during > suspend/resume cycles. > > Considering the power draw is really significant without these patches (10s > of %s per hour) and I'd think Dell would have sold some decent number of > these laptops, I'd been patiently waiting for these patches, or some variant > to show up in the stable trees, but so far I'm up to the 6.6 stable kernel > and still having to manually cherry-pick these, so I thought maybe I could > bring this to the PM maintainers' attention so at least start a discussion > about this issue. Thank you very much for raising this again. We really need to make some progress, and Mika recently posted a patch to add the 4ff116d0d5fd functionality again: https://lore.kernel.org/r/20231002070044.2299644-1-mika.westerberg@xxxxxxxxxxxxxxx The big problem is that it works on *most* systems, but it still seems to break a few. So Mika's current patch relies on a denylist of systems where we *don't* restore the substates. It's possible we'll have to give in and use a denylist, but it's obviously not ideal because (a) we don't know *why* it doesn't work on those systems, and (b) it means substates work before suspend but not after resume, which is a poor user experience. Bjorn > #!/bin/bash -e > date > egrep -Hr . /sys/class/drm/card0/power/rc6_residency_ms \ > /sys/devices/system/cpu/cpuidle/low_power_idle_system_residency_us \ > /sys/kernel/debug/pmc_core/package_cstate_show \ > /sys/kernel/debug/pmc_core/slp_s0_residency_usec \ > /sys/kernel/debug/dri/0/i915_edp_psr_status \ > /sys/kernel/debug/dri/0/i915_dmc_info | tee -a ~/Dropbox/XPS-7390/sleep-params > egrep '\(ns\): [^0]' /sys/kernel/debug/pmc_core/ltr_show | cut -d' ' -f1,3,4 | sed -e 's;[ ][ ]*; ;' | tee -a ~/Dropbox/XPS-7390/sleep-params > egrep -Hr ": On" /sys/kernel/debug/pmc_core/pch_ip_power_gating_status | tee -a /dev/tty | tee -a ~/Dropbox/XPS-7390/sleep-params | wc -l > egrep No /sys/kernel/debug/pmc_core/slp_s0_debug_status 2>/dev/null | tee -a ~/Dropbox/XPS-7390/sleep-params
