On 3/1/2023 4:04 PM, Tasos Sahanidis wrote: > On 2023-02-28 20:46, Alex Williamson wrote: >> Can you do the same for the root port to the GPU, ex. use lspci -t to >> find the parent root port. Since the device doesn't seem to be >> achieving D3cold (expected on a desktop system), the other significant >> change of the identified commit is that the root port will also enter a >> low power state. Prior to that commit the device would enter D3hot, but >> we never touched the root port. Perhaps confirm the root port now >> enters D3hot and compare lspci for the root port when using >> disable_idle_d3 to that found when trying to use the device without >> disable_idle_d3. Thanks, >> >> Alex >> > > I seem to have trouble understanding the lspci tree. > Can you please try following way to confirm the path ls -l /sys/bus/pci/devices It generally displays the full path like 0000:03:00.0 -> ../../../devices/pci0000:00/0000:00:1c.5/0000:02:00.0/0000:03:00.0 Also, Can you please print the runtime PM control entry as well /sys/bus/pci/devices/<root_port B:D:F>/power/control Thanks, Abhishek > The tree is as follows: > > -[0000:00]-+-00.0 Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex > [...] | > +-01.2-[02-0d]----00.0-[03-0d]--+-01.0-[04-05]----00.0-[05]--+-00.0 Creative Labs EMU10k2/CA0100/CA0102/CA10200 [Sound Blaster Audigy Series] > | | +-00.1 Creative Labs SB Audigy Game Port > | | +-01.0 Brooktree Corporation Bt878 Video Capture > | | \-01.1 Brooktree Corporation Bt878 Audio Capture > | +-02.0-[06]--+-00.0 Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XT [Radeon HD 7790/8770 / R7 360 / R9 260/360 OEM] > | | \-00.1 Advanced Micro Devices, Inc. [AMD/ATI] Tobago HDMI Audio [Radeon R7 360 / R9 360 OEM] > | +-03.0-[07-08]----00.0-[08]--+-00.0 Philips Semiconductors SAA7131/SAA7133/SAA7135 Video Broadcast Decoder > | | \-01.0 Yamaha Corporation YMF-744B [DS-1S Audio Controller] > | +-05.0-[09]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller > | +-06.0-[0a]--+-00.0 MosChip Semiconductor Technology Ltd. PCIe 9912 Multi-I/O Controller > | | +-00.1 MosChip Semiconductor Technology Ltd. PCIe 9912 Multi-I/O Controller > | | \-00.2 MosChip Semiconductor Technology Ltd. PCIe 9912 Multi-I/O Controller > | +-08.0-[0b]--+-00.0 Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP > | | +-00.1 Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller > | | \-00.3 Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller > | +-09.0-[0c]----00.0 Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] > | \-0a.0-[0d]----00.0 Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] > [...] | > > The parent root port is either 0000:00:01.2 or 0000:00:02.0, correct? > 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge > 02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream > > If so, I tested in 5.18, both before and while running the VM, with 6.2 > both with and without disable_idle_d3, and in all cases they stayed at D0. > > Only difference was the card itself would be at D0 instead of D3hot with > disable_idle_d3. In the working 5.18, without disable_idle_d3, it would > still enter D3hot. > > ==> 5_18_before_vm <== > # cat /sys/bus/pci/devices/0000:02:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:02:00.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:01.2/power_state > D0 > # cat /sys/bus/pci/devices/0000:00:01.2/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:02.0/power_state > unknown > # cat /sys/bus/pci/devices/0000:00:02.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:06:00.0/power_state > D3hot > # cat /sys/bus/pci/devices/0000:06:00.0/power/runtime_status > active > > ==> 5_18_running_vm <== > # cat /sys/bus/pci/devices/0000:02:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:02:00.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:01.2/power_state > D0 > # cat /sys/bus/pci/devices/0000:00:01.2/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:02.0/power_state > unknown > # cat /sys/bus/pci/devices/0000:00:02.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:06:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:06:00.0/power/runtime_status > active > > ==> 6_2_before_vm <== > # cat /sys/bus/pci/devices/0000:02:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:02:00.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:01.2/power_state > D0 > # cat /sys/bus/pci/devices/0000:00:01.2/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:02.0/power_state > unknown > # cat /sys/bus/pci/devices/0000:00:02.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:06:00.0/power_state > D3hot > # cat /sys/bus/pci/devices/0000:06:00.0/power/runtime_status > suspended > > ==> 6_2_running_vm <== > # cat /sys/bus/pci/devices/0000:02:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:02:00.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:01.2/power_state > D0 > # cat /sys/bus/pci/devices/0000:00:01.2/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:02.0/power_state > unknown > # cat /sys/bus/pci/devices/0000:00:02.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:06:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:06:00.0/power/runtime_status > active > > ==> 6_2_before_vm_disable_idle_d3 <== > # cat /sys/bus/pci/devices/0000:02:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:02:00.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:01.2/power_state > D0 > # cat /sys/bus/pci/devices/0000:00:01.2/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:02.0/power_state > unknown > # cat /sys/bus/pci/devices/0000:00:02.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:06:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:06:00.0/power/runtime_status > active > > ==> 6_2_running_vm_disable_idle_d3 <== > # cat /sys/bus/pci/devices/0000:02:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:02:00.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:01.2/power_state > D0 > # cat /sys/bus/pci/devices/0000:00:01.2/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:00:02.0/power_state > unknown > # cat /sys/bus/pci/devices/0000:00:02.0/power/runtime_status > active > # cat /sys/bus/pci/devices/0000:06:00.0/power_state > D0 > # cat /sys/bus/pci/devices/0000:06:00.0/power/runtime_status > active > > 0000:00:02.0 is Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge > and can presumably be ignored. > > -- > Tasos
