Re: [PATCH 5/6] PCI/ASPM: Actually configure the L1 substate settings to the device

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi!

On 03/01/17 06:34, Rajat Jain wrote:
> Add code to actually configure the L1 substate settigns on the
> upstream and downstream device, while taking care of the rules
> dictated by the PCIe spec.

While testing hibernate on an arm64 juno with v4.11-rc1, I get a NULL pointer
dereference from pcie_config_aspm_link():


> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index a70afdf..6735f38 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -597,11 +683,23 @@ static void pcie_config_aspm_dev(struct pci_dev *pdev, u32 val)
>  static void pcie_config_aspm_link(struct pcie_link_state *link, u32 state)
>  {
>  	u32 upstream = 0, dwstream = 0;
> -	struct pci_dev *child, *parent = link->pdev;
> +	struct pci_dev *child = link->downstream, *parent = link->pdev;


Here link->downstream is NULL,


>  	struct pci_bus *linkbus = parent->subordinate;
>  
> -	/* Nothing to do if the link is already in the requested state */
> +	/* Enable only the states that were not explicitly disabled */
>  	state &= (link->aspm_capable & ~link->aspm_disable);
> +
> +	/* Can't enable any substates if L1 is not enabled */
> +	if (!(state & ASPM_STATE_L1))
> +		state &= ~ASPM_STATE_L1SS;
> +
> +	/* Spec says both ports must be in D0 before enabling PCI PM substates*/
> +	if (parent->current_state != PCI_D0 || child->current_state != PCI_D0) {

Which causes a crash[0] here.


> +		state &= ~ASPM_STATE_L1_SS_PCIPM;
> +		state |= (link->aspm_enabled & ASPM_STATE_L1_SS_PCIPM);
> +	}
> +
> +	/* Nothing to do if the link is already in the requested state */
>  	if (link->aspm_enabled == state)
>  		return;
>  	/* Convert ASPM state to upstream/downstream ASPM register state */


(For trees based on v4.10-rc1 I need to cherry-pick b4b8664d291a to make arm64
build)

Testing either side of aeda9adebab8 gives this patch as the first to expose the
problem.


At boot I get the message:
> pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it
with 'pcie_aspm=force'

for the pcie device being resumed here, should this code be called at all for
this device?

The 'pci' section of the boot log for v4.11-rc1 is below[1], and lspci[2].


Thanks,

James


[0] git checkout aeda9adebab8; git cherry-pick b4b8664d291a
[  170.379816] PM: noirq thaw of devices complete after 0.702 msecs
[  170.387234] PM: early thaw of devices complete after 1.390 msecs
[  170.421447] Unable to handle kernel NULL pointer dereference at virtual addre
ss 00000080
[  170.429497] pgd = ffff000008f10000
[  170.432878] [00000080] *pgd=00000009ffffe003
[  170.432882] , *pud=00000009ffffd003
[  170.437118] , *pmd=0000000000000000
[  170.440579]

[  170.445513] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[  170.451027] Modules linked in:
[  170.454055] CPU: 4 PID: 2509 Comm: kworker/u12:9 Not tainted 4.10.0-rc1-00006
-g51e9dca242d2 #7202
[  170.462837] Hardware name: ARM Juno development board (r1) (DT)
[  170.468707] Workqueue: events_unbound async_run_entry_fn
[  170.473965] task: ffff8009764c2580 task.stack: ffff80097374c000
[  170.479831] PC is at pcie_config_aspm_link+0x4c/0x300
[  170.484833] LR is at pcie_config_aspm_path+0x40/0x88
[  170.489747] pc : [<ffff000008411ebc>] lr : [<ffff0000084121b0>] pstate: 40000
045
[  170.497066] sp : ffff80097374fb40
[  170.500343] x29: ffff80097374fb40 x28: ffff800976c0c2a8
[  170.505603] x27: ffff800976c0c078 x26: ffff800976c0c020
[  170.510863] x25: 0000000000000001 x24: ffff8009750f9120
[  170.516122] x23: ffff8009764bd100 x22: ffff000008045000
[  170.521381] x21: ffff800976c24000 x20: ffff000008ef3320
[  170.526640] x19: 0000000000000000 x18: 0000000000000000
[  170.531899] x17: 0000000000000000 x16: 0000000000000000
[  170.537158] x15: 0000000000000000 x14: 00000000000000dc
[  170.542417] x13: ffff000008e17628 x12: ffff800976fa8028
[  170.547676] x11: ffff80097641f000 x10: ffff000008e17608
[  170.552936] x9 : ffff800976fa8028 x8 : 0000000000000000
[  170.558194] x7 : 000000000000007b x6 : 0000000000000000
[  170.563453] x5 : 0000000000000fa0 x4 : ffff80097641fd60
[  170.568712] x3 : 0000000000000000 x2 : 0000000000000000
[  170.573971] x1 : 0000000000000000 x0 : ffff80097641fd00
[  170.579229]
[  170.580701] Process kworker/u12:9 (pid: 2509, stack limit = 0xffff80097374c00
0)
[  170.587936] Stack: (0xffff80097374fb40 to 0xffff800973750000)
[  170.888248] Call trace:
[  170.974579] [<ffff000008411ebc>] pcie_config_aspm_link+0x4c/0x300
[  170.980614] [<ffff0000084121b0>] pcie_config_aspm_path+0x40/0x88
[  170.986564] [<ffff0000084130f4>] pcie_aspm_pm_state_change+0x5c/0x80
[  170.992856] [<ffff0000083ff0a4>] pci_raw_set_power_state+0x15c/0x230
[  170.999148] [<ffff0000084018ec>] pci_set_power_state+0x64/0x150
[  171.005013] [<ffff00000859ae20>] ata_pci_device_do_resume+0x18/0x70
[  171.011220] [<ffff0000085baa3c>] sil24_pci_device_resume+0x24/0x78
[  171.017341] [<ffff000008404fd8>] pci_legacy_resume+0x38/0x58
[  171.022945] [<ffff000008405f00>] pci_pm_thaw+0xa0/0xd0
[  171.028034] [<ffff000008546ce4>] dpm_run_callback.isra.4+0x1c/0x70
[  171.034154] [<ffff0000085470a0>] device_resume+0x88/0x150
[  171.039500] [<ffff00000854718c>] async_resume+0x24/0x58
[  171.044674] [<ffff0000080de55c>] async_run_entry_fn+0x3c/0xf8
[  171.050365] [<ffff0000080d4de8>] process_one_work+0x1d0/0x390
[  171.056055] [<ffff0000080d4ff0>] worker_thread+0x48/0x480
[  171.061400] [<ffff0000080daec0>] kthread+0xf8/0x128
[  171.066230] [<ffff0000080836c0>] ret_from_fork+0x10/0x50
[  171.071490] Code: 12196e61 f27e027f 1a930033 35000063 (b9408101)
[  171.077566] ---[ end trace 93b51adc52c63085 ]---

[1] v4.11-rc1 pci boot messages
[    0.732114] OF: PCI: host bridge /pcie-controller@30000000 ranges:
[    0.732133] OF: PCI:    IO 0x5f800000..0x5fffffff -> 0x5f800000
[    0.732145] OF: PCI:   MEM 0x50000000..0x57ffffff -> 0x50000000
[    0.732152] OF: PCI:   MEM 0x4000000000..0x40ffffffff -> 0x4000000000
[    0.732204] pci-host-generic 40000000.pcie-controller: ECAM at [mem 0x4000000
0-0x4fffffff] for [bus 00-ff]
[    0.732401] pci-host-generic 40000000.pcie-controller: PCI host bridge to bus
 0000:00
[    0.732410] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.732421] pci_bus 0000:00: root bus resource [io  0x0000-0x7fffff] (bus add
ress [0x5f800000-0x5fffffff])
[    0.732428] pci_bus 0000:00: root bus resource [mem 0x50000000-0x57ffffff]
[    0.732435] pci_bus 0000:00: root bus resource [mem 0x4000000000-0x40ffffffff
 pref]
[    0.732462] pci 0000:00:00.0: [1556:1100] type 01 class 0x060400
[    0.732485] pci 0000:00:00.0: reg 0x10: [mem 0x57d00000-0x57d03fff 64bit pref]
[    0.732543] pci 0000:00:00.0: supports D1 D2
[    0.732832] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.733252] pci 0000:01:00.0: [111d:8090] type 01 class 0x060400
[    0.733423] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    0.744804] pci 0000:02:01.0: [111d:8090] type 01 class 0x060400
[    0.744993] pci 0000:02:01.0: PME# supported from D0 D3hot D3cold
[    0.745223] pci 0000:02:02.0: [111d:8090] type 01 class 0x060400
[    0.745414] pci 0000:02:02.0: PME# supported from D0 D3hot D3cold
[    0.745641] pci 0000:02:03.0: [111d:8090] type 01 class 0x060400
[    0.745828] pci 0000:02:03.0: PME# supported from D0 D3hot D3cold
[    0.746076] pci 0000:02:0c.0: [111d:8090] type 01 class 0x060400
[    0.746263] pci 0000:02:0c.0: PME# supported from D0 D3hot D3cold
[    0.746482] pci 0000:02:10.0: [111d:8090] type 01 class 0x060400
[    0.746646] pci 0000:02:10.0: PME# supported from D0 D3hot D3cold
[    0.746886] pci 0000:02:1f.0: [111d:8090] type 01 class 0x060400
[    0.747074] pci 0000:02:1f.0: PME# supported from D0 D3hot D3cold
[    0.747490] pci 0000:03:00.0: [1095:3132] type 00 class 0x018000
[    0.747531] pci 0000:03:00.0: reg 0x10: [mem 0x57f04000-0x57f0407f 64bit]
[    0.747559] pci 0000:03:00.0: reg 0x18: [mem 0x57f00000-0x57f03fff 64bit]
[    0.747577] pci 0000:03:00.0: reg 0x20: initial BAR value 0x00000000 invalid
[    0.747584] pci 0000:03:00.0: reg 0x20: [io  size 0x0080]
[    0.747614] pci 0000:03:00.0: reg 0x30: [mem 0xfff80000-0xffffffff pref]
[    0.747720] pci 0000:03:00.0: supports D1 D2
[    0.747886] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device.  You can
 enable it with 'pcie_aspm=force'
[    0.747910] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
[    0.748044] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[    0.748180] pci_bus 0000:05: busn_res: [bus 05-ff] end is updated to 05
[    0.748311] pci_bus 0000:06: busn_res: [bus 06-ff] end is updated to 06
[    0.748441] pci_bus 0000:07: busn_res: [bus 07-ff] end is updated to 07
[    0.748614] pci 0000:08:00.0: [11ab:4380] type 00 class 0x020000
[    0.748651] pci 0000:08:00.0: reg 0x10: [mem 0x57e00000-0x57e03fff 64bit]
[    0.748668] pci 0000:08:00.0: reg 0x18: initial BAR value 0x00000000 invalid
[    0.748675] pci 0000:08:00.0: reg 0x18: [io  size 0x0100]
[    0.748814] pci 0000:08:00.0: supports D1 D2
[    0.748821] pci 0000:08:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.749085] pci_bus 0000:08: busn_res: [bus 08-ff] end is updated to 08
[    0.749099] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 08
[    0.749112] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 08
[    0.749249] pci 0000:00:00.0: BAR 14: assigned [mem 0x50000000-0x501fffff]
[    0.749260] pci 0000:00:00.0: BAR 0: assigned [mem 0x4000000000-0x4000003fff
64bit pref]
[    0.749272] pci 0000:00:00.0: BAR 13: assigned [io  0x1000-0x2fff]
[    0.749283] pci 0000:01:00.0: BAR 14: assigned [mem 0x50000000-0x501fffff]
[    0.749290] pci 0000:01:00.0: BAR 13: assigned [io  0x1000-0x2fff]
[    0.749303] pci 0000:02:01.0: BAR 14: assigned [mem 0x50000000-0x500fffff]
[    0.749311] pci 0000:02:1f.0: BAR 14: assigned [mem 0x50100000-0x501fffff]
[    0.749318] pci 0000:02:01.0: BAR 13: assigned [io  0x1000-0x1fff]
[    0.749325] pci 0000:02:1f.0: BAR 13: assigned [io  0x2000-0x2fff]
[    0.749337] pci 0000:03:00.0: BAR 6: assigned [mem 0x50000000-0x5007ffff pref]
[    0.749347] pci 0000:03:00.0: BAR 2: assigned [mem 0x50080000-0x50083fff 64bit]
[    0.749372] pci 0000:03:00.0: BAR 0: assigned [mem 0x50084000-0x5008407f 64bit]
[    0.749394] pci 0000:03:00.0: BAR 4: assigned [io  0x1000-0x107f]
[    0.749407] pci 0000:02:01.0: PCI bridge to [bus 03]
[    0.749416] pci 0000:02:01.0:   bridge window [io  0x1000-0x1fff]
[    0.749429] pci 0000:02:01.0:   bridge window [mem 0x50000000-0x500fffff]
[    0.749451] pci 0000:02:02.0: PCI bridge to [bus 04]
[    0.749479] pci 0000:02:03.0: PCI bridge to [bus 05]
[    0.749507] pci 0000:02:0c.0: PCI bridge to [bus 06]
[    0.749535] pci 0000:02:10.0: PCI bridge to [bus 07]
[    0.749569] pci 0000:08:00.0: BAR 0: assigned [mem 0x50100000-0x50103fff 64bit]
[    0.749590] pci 0000:08:00.0: BAR 2: assigned [io  0x2000-0x20ff]
[    0.749600] pci 0000:02:1f.0: PCI bridge to [bus 08]
[    0.749608] pci 0000:02:1f.0:   bridge window [io  0x2000-0x2fff]
[    0.749622] pci 0000:02:1f.0:   bridge window [mem 0x50100000-0x501fffff]
[    0.749643] pci 0000:01:00.0: PCI bridge to [bus 02-08]
[    0.749651] pci 0000:01:00.0:   bridge window [io  0x1000-0x2fff]
[    0.749664] pci 0000:01:00.0:   bridge window [mem 0x50000000-0x501fffff]
[    0.749685] pci 0000:00:00.0: PCI bridge to [bus 01-08]
[    0.749691] pci 0000:00:00.0:   bridge window [io  0x1000-0x2fff]
[    0.749699] pci 0000:00:00.0:   bridge window [mem 0x50000000-0x501fffff]
[    0.750009] pcieport 0000:00:00.0: Signaling PME with IRQ 39
[    0.750155] pcieport 0000:00:00.0: AER enabled with IRQ 39

[2] lspci
root@juno-r1:~# lspci
00:00.0 PCI bridge: PLDA PCI Express Core Reference Design (rev 01)
01:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
02:01.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
02:02.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
02:03.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
02:0c.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
02:10.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
02:1f.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8090 (rev 02
)
03:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II
 Controller (rev 01)
08:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8057 PCI-E Gigabit
 Ethernet Controller



[Index of Archives]     [DMA Engine]     [Linux Coverity]     [Linux USB]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [Greybus]

  Powered by Linux