Re: PCI trouble on mvebu (Turris Omnia)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

On 30/10/2020 15:23, Pali Rohár wrote:

On Friday 30 October 2020 14:02:22 Toke Høiland-Jørgensen wrote:

Pali Rohár <pali@xxxxxxxxxx> writes:

My experience with that WLE900VX card, aardvark driver and aspm code:

Link training in GEN2 mode for this card succeed only once after reset.
Repeated link retraining fails and it fails even when aardvark is
reconfigured to GEN1 mode. Reset via PERST# signal is required to have
working link training.

What I did in aardvark driver: Set mode to GEN2, do link training. If
success read "negotiated link speed" from "Link Control Status Register"
(for WLE900VX it is 0x1 - GEN1) and set it into aardvark. And then
retrain link again (for WLE900VX now it would be at GEN1). After that
card is stable and all future retraining (e.g. from aspm.c) also passes.

If I do not change aardvark mode from GEN2 to GEN1 the second link
training fails. And if I change mode to GEN1 after this failed link
training then nothing happen, link training do not success.

So just speculation now... In current setup initialization of card does
one link training at GEN2. Then aspm.c is called which is doing second
link retraining at GEN2. And if it fails then below patch issue third
link retraining at GEN1. If A38x/pci-mvebu has same problem as aardvark
then second link retraining must be at GEN1 (not GEN2) to workaround
this issue.

Bjorn, Toke: what about trying to hack aspm.c code to never do link
retraining at GEN2 speed? And always force GEN1 speed prior link
training?

Sounds like a plan. I poked around in aspm.c and must confess to being a
bit lost in the soup of registers ;)

So if one of you can cook up a patch, that would be most helpful!

I modified Bjorn's patch, explicitly set tls to 1 and added debug info
about cls (current link speed, that what is used by aardvark). It is
untested, I just tried to compile it.

Can try it?

diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 253c30cc1967..f934c0b52f41 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -206,6 +206,27 @@ static bool pcie_retrain_link(struct pcie_link_state *link)
  	unsigned long end_jiffies;
  	u16 reg16;
+ u32 lnkcap2; 
+	u16 lnksta, lnkctl2, cls, tls;
+
+	pcie_capability_read_dword(parent, PCI_EXP_LNKCAP2, &lnkcap2);
+	pcie_capability_read_word(parent, PCI_EXP_LNKSTA, &lnksta);
+	pcie_capability_read_word(parent, PCI_EXP_LNKCTL2, &lnkctl2);
+	cls = lnksta & PCI_EXP_LNKSTA_CLS;
+	tls = lnkctl2 & PCI_EXP_LNKCTL2_TLS;
+
+	pci_info(parent, "lnkcap2 %#010x sls %#04x lnksta %#06x cls %#03x lnkctl2 %#06x tls %#03x\n",
+		lnkcap2, (lnkcap2 & 0x3F) >> 1,
+		lnksta, cls,
+		lnkctl2, tls);
+
+	tls = 1;
+	pcie_capability_clear_and_set_word(parent, PCI_EXP_LNKCTL2,
+					PCI_EXP_LNKCTL2_TLS, tls);
+	pcie_capability_read_word(parent, PCI_EXP_LNKCTL2, &lnkctl2);
+	pci_info(parent, "lnkctl2 %#010x new tls %#03x\n",
+		lnkctl2, tls);
+
  	pcie_capability_read_word(parent, PCI_EXP_LNKCTL, &reg16);
  	reg16 |= PCI_EXP_LNKCTL_RL;
  	pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16);
@@ -227,6 +248,8 @@ static bool pcie_retrain_link(struct pcie_link_state *link)
  			break;
  		msleep(1);
  	} while (time_before(jiffies, end_jiffies));
+	pci_info(parent, "lnksta %#06x new cls %#03x\n",
+		lnksta, (cls & PCI_EXP_LNKSTA_CLS));
  	return !(reg16 & PCI_EXP_LNKSTA_LT);
  }


Still exhibiting the BAR update error, run tested with next--20201030


0.396182] mvebu-pcie soc:pcie: host bridge /soc/pcie ranges:
0.396205] mvebu-pcie soc:pcie: Parsing ranges property...
0.396222] mvebu-pcie soc:pcie:      MEM 0x00f1080000..0x00f1081fff -> 0x0000080000 0.396251] mvebu-pcie soc:pcie:      MEM 0x00f1040000..0x00f1041fff -> 0x0000040000 0.396278] mvebu-pcie soc:pcie:      MEM 0x00f1044000..0x00f1045fff -> 0x0000044000 0.396303] mvebu-pcie soc:pcie:      MEM 0x00f1048000..0x00f1049fff -> 0x0000048000 0.396329] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0100000000 0.396340] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0100000000 0.396351] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0200000000 0.396361] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0200000000 0.396372] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0300000000 0.396382] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0300000000 0.396393] mvebu-pcie soc:pcie:      MEM 0xffffffffffffffff..0x00fffffffe -> 0x0400000000 0.396400] mvebu-pcie soc:pcie:       IO 0xffffffffffffffff..0x00fffffffe -> 0x0400000000 
0.397280] mvebu-pcie soc:pcie: PCI host bridge to bus 0000:00
0.397299] pci_bus 0000:00: root bus resource [bus 00-ff]
0.397314] pci_bus 0000:00: root bus resource [mem 0xf1080000-0xf1081fff] (bus address [0x00080000-0x00081fff]) 0.397327] pci_bus 0000:00: root bus resource [mem 0xf1040000-0xf1041fff] (bus address [0x00040000-0x00041fff]) 0.397348] pci_bus 0000:00: root bus resource [mem 0xf1044000-0xf1045fff] (bus address [0x00044000-0x00045fff]) 0.397360] pci_bus 0000:00: root bus resource [mem 0xf1048000-0xf1049fff] (bus address [0x00048000-0x00049fff]) 
0.397371] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xe7ffffff]
0.397383] pci_bus 0000:00: root bus resource [io  0x1000-0xeffff]
0.397388] pci_bus 0000:00: scanning bus
0.397495] pci 0000:00:01.0: [11ab:6820] type 01 class 0x060400
0.397509] pci 0000:00:01.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
0.398052] pci 0000:00:02.0: [11ab:6820] type 01 class 0x060400
0.398064] pci 0000:00:02.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
0.398585] pci 0000:00:03.0: [11ab:6820] type 01 class 0x060400
0.398597] pci 0000:00:03.0: reg 0x38: [mem 0x00000000-0x000007ff pref]
0.399755] pci_bus 0000:00: fixups for bus
0.399773] pci 0000:00:01.0: scanning [bus 00-00] behind bridge, pass 0
0.399777] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring 
0.399784] pci 0000:00:02.0: scanning [bus 00-00] behind bridge, pass 0
0.399787] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring 
0.399794] pci 0000:00:03.0: scanning [bus 00-00] behind bridge, pass 0
0.399797] pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring 
0.399803] pci 0000:00:01.0: scanning [bus 00-00] behind bridge, pass 1
0.400032] pci_bus 0000:01: scanning bus
0.400784] pci_bus 0000:01: fixups for bus
0.400794] pci_bus 0000:01: bus scan returning with max=01
0.400800] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
0.400808] pci 0000:00:02.0: scanning [bus 00-00] behind bridge, pass 1
0.401032] pci_bus 0000:02: scanning bus
0.401078] pci 0000:02:00.0: [168c:003c] type 00 class 0x028000
0.401098] pci 0000:02:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
0.401125] pci 0000:02:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
0.401217] pci 0000:02:00.0: supports D1 D2
0.401614] pci 0000:00:02.0: ASPM: current common clock configuration is inconsistent, reconfiguring 0.401626] pci 0000:00:02.0: lnkcap2 0x00000000 sls 0x00 lnksta 0x1011 cls 0x1 lnkctl2 0x0000 tls 0x0 
0.401632] pci 0000:00:02.0: lnkctl2 0x00000000 new tls 0x1
0.428701] pci 0000:00:02.0: lnksta 0x1011 new cls 0x1
0.429486] pci_bus 0000:02: fixups for bus
0.429498] pci_bus 0000:02: bus scan returning with max=02
0.429504] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
0.429514] pci 0000:00:03.0: scanning [bus 00-00] behind bridge, pass 1
0.429778] pci_bus 0000:03: scanning bus
0.429831] pci 0000:03:00.0: [168c:002e] type 00 class 0x028000
0.429854] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
0.429978] pci 0000:03:00.0: supports D1
0.429985] pci 0000:03:00.0: PME# supported from D0 D1 D3hot
0.429992] pci 0000:03:00.0: PME# disabled
0.430403] pci 0000:00:03.0: ASPM: current common clock configuration is inconsistent, reconfiguring 0.430416] pci 0000:00:03.0: lnkcap2 0x00000000 sls 0x00 lnksta 0x1011 cls 0x1 lnkctl2 0x0000 tls 0x0 
0.430421] pci 0000:00:03.0: lnkctl2 0x00000000 new tls 0x1
0.460692] pci 0000:00:03.0: lnksta 0x1011 new cls 0x1
0.461459] pci_bus 0000:03: fixups for bus
0.461470] pci_bus 0000:03: bus scan returning with max=03
0.461476] pci_bus 0000:03: busn_res: [bus 03-ff] end is updated to 03
0.461482] pci_bus 0000:00: bus scan returning with max=03
0.461552] pci 0000:00:02.0: BAR 8: assigned [mem 0xe0000000-0xe02fffff]
0.461561] pci 0000:00:03.0: BAR 8: assigned [mem 0xe0300000-0xe03fffff]
0.461568] pci 0000:00:01.0: BAR 6: assigned [mem 0xe0400000-0xe04007ff pref]
0.461576] pci 0000:00:02.0: BAR 6: assigned [mem 0xe0500000-0xe05007ff pref]
0.461583] pci 0000:00:03.0: BAR 6: assigned [mem 0xe0600000-0xe06007ff pref]
0.461593] pci 0000:00:01.0: PCI bridge to [bus 01]
0.461620] pci 0000:02:00.0: BAR 0: assigned [mem 0xe0000000-0xe01fffff 64bit] 
0.461627] pci 0000:02:00.0: BAR 0: error updating (0xe0000004 != 0xffffffff)
0.461633] pci 0000:02:00.0: BAR 0: error updating (high 0x000000 != 0xffffffff) 
0.461639] pci 0000:02:00.0: BAR 6: assigned [mem 0xe0200000-0xe020ffff pref]
0.461645] pci 0000:00:02.0: PCI bridge to [bus 02]
0.461651] pci 0000:00:02.0:   bridge window [mem 0xe0000000-0xe02fffff]
0.461666] pci 0000:03:00.0: BAR 0: assigned [mem 0xe0300000-0xe030ffff 64bit] 
0.461673] pci 0000:03:00.0: BAR 0: error updating (0xe0300004 != 0xffffffff)
0.461678] pci 0000:03:00.0: BAR 0: error updating (high 0x000000 != 0xffffffff) 
0.461683] pci 0000:00:03.0: PCI bridge to [bus 03]
0.461689] pci 0000:00:03.0:   bridge window [mem 0xe0300000-0xe03fffff]
0.461701] pci 0000:00:01.0: Max Payload Size set to  128/ 128 (was 128), Max Read Rq  128 0.461710] pci 0000:00:02.0: Max Payload Size set to  128/ 128 (was 128), Max Read Rq  128 
0.461715] pci 0000:02:00.0: Failed attempting to set the MPS
0.461721] pci 0000:02:00.0: Max Payload Size set to  128/ 256 (was 128), Max Read Rq  128 0.461729] pci 0000:00:03.0: Max Payload Size set to  128/ 128 (was 128), Max Read Rq  128 
0.461734] pci 0000:03:00.0: Failed attempting to set the MPS
0.461740] pci 0000:03:00.0: Max Payload Size set to  128/ 128 (was 128), Max Read Rq  128 
0.461855] pcieport 0000:00:01.0: assign IRQ: got 0
0.461866] pcieport 0000:00:01.0: enabling bus mastering
0.461959] pcieport 0000:00:02.0: assign IRQ: got 0
0.461966] pcieport 0000:00:02.0: enabling device (0140 -> 0142)
0.461980] pcieport 0000:00:02.0: enabling bus mastering
0.462065] pcieport 0000:00:03.0: assign IRQ: got 0
0.462070] pcieport 0000:00:03.0: enabling device (0140 -> 0142)
0.462080] pcieport 0000:00:03.0: enabling bus mastering
2.467153] pci 0000:00:03.0: enabling bus mastering
2.519024] ath10k_pci 0000:02:00.0: of_irq_parse_pci: failed with rc=134
2.531459] ath10k_pci 0000:02:00.0: assign IRQ: got 0
2.536915] pci 0000:00:02.0: enabling bus mastering
2.540553] ath10k_pci 0000:02:00.0: can't change power state from D3hot to D0 (config space inaccessible) 
2.580450] ath10k_pci 0000:02:00.0: failed to wake up device : -110
2.586973] ath10k_pci 0000:02:00.0: disabling bus mastering
2.587220] ath10k_pci: probe of 0000:02:00.0 failed with error -110
2.605598] ehci-pci: EHCI PCI platform driver

Attachment: OpenPGP_0x729CFF47A416598B.asc
Description: application/pgp-keys

Attachment: OpenPGP_signature
Description: OpenPGP digital signature

[Index of Archives]     [DMA Engine]     [Linux Coverity]     [Linux USB]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [Greybus]

  Powered by Linux