Hello, This series of patches introduces PCIe support for the Marvell Armada 370 and Armada XP. This PATCHv3 follows: * PATCHv2, sent on January, 28th 2013 * RFCv1, sent on December, 7th 2012 Thanks to all the people who discussed on the previous version of the patch set. The discussions have been long and complicated, but certainly very useful. In order to make progress on this, and help reducing the size of the patch set, I would ask if it would be possible to merge patches 1 to 17 from the series for 3.9 (only preparation work), keeping the rest for 3.10. The patches in question are PCI-related, ARM-related, and mvebu/orion-related. Changes between v2 and v3: * Use of_irq_map_pci() instead of of_irq_map_raw(), as suggested by Andrew Murray. In order to do this, we moved the interrupt-map and interrupt-map-mask DT properties from the main PCIe controller node to the DT subnodes representing each PCIe interface. * Remove the usage of the emulated host bridge. * Move the emulated PCI-to-PCI bridge code into the Marvell PCI driver itself, in order to allow a tighter integration. Suggested by Bjorn Helgaas and Jason Gunthorpe. * Make the allocation of address decoding windows dynamic: it's when memory accesses or I/O accesses are enabled at the PCI-to-PCI bridge level that we allocate and setup the corresponding address decoding window. Requested by Bjorn Helgaas. * Fixed the implementation of I/O accesses to use I/O addresses that fall within the normal IO_SPACE_LIMIT. This required using the "remap" functionality of address decoding windows, and therefore some changes in the address decoding window allocator. Follows a long discussion about I/O accesses. * Set up a correct bus number in the configuration of the PCIe interfaces so that we don't have to fake bus numbers anymore. Requested by Jason Gunthorpe. * Fix the of_pci_get_devfn() implementation according to Stephen Warren's comment. * Use CFLAGS_ instead of ccflags to add the mach-mvebu and plat-orion include paths when building the pci-mvebu driver. This ensures that the include paths are only added when building this specific driver. Requested by Stephen Warren. * Fix the ->resource_align() to only apply on bus 0 (the one on which the emulated PCI-to-PCI bridges sit), and to request an alignment on the size of the window (and not only 64 KB for I/O windows and 1 MB for memory windows). * Clarified the commit log of "clk: mvebu: create parent-child relation for PCIe clocks on Armada 370" A quick description of the patches: * Patches 1 to 3 add PCI-related Device Tree parsing functions. Those patches are common with the Nvidia Tegra PCIe patch set from Thierry Redding. They are included in this series so that it can be tested easily. * Patch 4 extends the ARM PCI core to store a per-controller private data pointer. This patch is common with the Nvidia Tegra PCIe patch set from Thierry Redding. It is included in this series so that it can be tested easily. * Patch 5 fixes a problem in lib/devres.c that prevents certain PCI-related functions from being visible on NO_IOPORT platforms. I know this patch isn't acceptable by itself, but the discussion about this has been so huge and went in so many directions that in the end, I don't know what is the correct way of fixing this. If an agreement is found on how to fix this properly, I'm willing to work on it if needed. * Patch 6 extends the ARM PCI core with an additional hook that a PCI controller driver can register and get called to realign PCI ressource addresses. This is needed for the support of Marvell PCIe interfaces because the address decoding windows for I/O ranges have a granularity of 64 KB, while the PCI standard requires only a 4 KB alignement. See the patch itself for details. * Patch 7 fixes a mistake in the interrupt controller node of the Armada 370/XP Device Tree, which was invisible until we started using the of_irq_map_raw() function, needed in our PCIe support. * Patches 8 and 9 fix some issues in the Armada 370/XP clock gating driver, related to PCIe interfaces. * Patches 10 and 11 are cleanup/refactoring of the common plat-orion address decoding code, in preparation for further changes related to PCIe. * Patches 12 to 17 introduce a ORION_ADDR_MAP_NO_REMAP define that is used by existing Marvell SoC code to say "I don't need this window to remap anything". Previously a -1 value was used as the remap address to communicate the fact that no remap is needed, but this prevents any remap address higher than 2 GB. * Patch 18 removes __init from a few address window decoding functions that are now needed after boot. * Patch 19 introduces in the common plat-orion address decoding code functions to allocate/free an address decoding window. Until now, the address decoding windows were configured statically. With Armada XP having up to 10 PCIe interfaces, we don't want to allocate useless address decoding windows statically, so we move to a more dynamic model in which address decoding windows are configured only for the PCIe interfaces that are actually in use. * Patch 20 removes __init from a few PCIe functions that are now needed after boot. * Patch 21 improves the Armada 370/XP specific address decoding code to provide functions that add and remove an address decoding window for a given PCIe interface. It relies on the common functions added in patch 19. * Patch 22 makes the common plat-orion PCIe code available on PLAT_ORION platforms such as ARCH_MVEBU. * Patch 23 creates the drivers/pci/host directory and makes the related minimal changes to Kconfig/Makefile. This patch will trivially conflict with the NVidia Tegra PCIe support posted by Thierry Redding, which also creates the drivers/pci/host directory. * Patch 24 contains the Armada 370/XP PCIe driver itself, that implements the necessary operations required by the ARM PCI core, and configures the address decoding windows as needed. This driver relies on a Device Tree description of the PCIe interfaces. * Patch 25 marks the ARCH_MVEBU platform has having PCI available, which allows the compilation of the PCIe support. * Patches 26 and 27 add the SoC-level Device Tree informations related to PCIe for Armada 370 and Armada XP. * Patch 28 to 31 add the board-level Device Tree informations related to PCIe for the Armada XP DB, Armada 370 DB, PlatHome OpenBlocks AX3-4 and GlobalScale Mirabox boards. * Patch 32 updates mvebu_defconfig with PCI and USB support. This patch set applies on top of v3.8-rc7, and has been pushed at: git://github.com/MISL-EBU-System-SW/mainline-public.git marvell-pcie-v3 Thanks, Thomas --- Output of lspci -vvv: 00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+ Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: fff00000-000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+ PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+ Capabilities: [fc] <chain broken> 00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+ Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: fff00000-000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+ PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+ Capabilities: [fc] <chain broken> 00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+ Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=03, subordinate=03, sec-latency=0 I/O behind bridge: 00010000-00010fff Memory behind bridge: c1000000-c10fffff Prefetchable memory behind bridge: c1100000-c11fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+ PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+ Capabilities: [fc] <chain broken> 00:04.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+ Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=04, subordinate=04, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: fff00000-000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+ PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+ Capabilities: [fc] <chain broken> 00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+ Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=05, subordinate=05, sec-latency=0 I/O behind bridge: 00020000-00020fff Memory behind bridge: c1200000-c12fffff Prefetchable memory behind bridge: c1300000-c13fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+ PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+ Capabilities: [fc] <chain broken> 00:06.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF+ FastB2B+ ParErr+ DEVSEL=?? >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx+ Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=06, subordinate=06, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: fff00000-000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA+ VGA+ MAbort+ >Reset+ FastB2B+ PriDiscTmr+ SecDiscTmr+ DiscTmrStat+ DiscTmrSERREn+ Capabilities: [fc] <chain broken> 03:00.0 SCSI storage controller: Marvell Technology Group Ltd. 88SX7042 PCI-e 4-port SATA-II (rev 02) Subsystem: Marvell Technology Group Ltd. Device 11ab Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 105 Region 0: Memory at c1000000 (64-bit, non-prefetchable) [size=1M] Region 2: I/O ports at 10000 [size=256] [virtual] Expansion ROM at c1100000 [disabled] [size=512K] Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000012345678 Data: 0000 Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 <256ns, L1 unlimited ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Kernel driver in use: sata_mv 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) Subsystem: Intel Corporation PRO/1000 PT Server Adapter Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 106 Region 0: Memory at c1200000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at c1220000 (32-bit, non-prefetchable) [size=128K] Region 2: I/O ports at 20000 [disabled] [size=32] [virtual] Expansion ROM at c1300000 [disabled] [size=128K] Capabilities: [c8] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [e0] Express (v1) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <4us, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-c1-c4-fe Kernel driver in use: e1000e Boot messages from the PCI subsystem: mvebu-pcie pcie-controller.1: PCIe0.0: link down mvebu-pcie pcie-controller.1: PCIe0.1: link down mvebu-pcie pcie-controller.1: PCIe0.2: link up mvebu-pcie pcie-controller.1: PCIe0.3: link down mvebu-pcie pcie-controller.1: PCIe2.0: link up mvebu-pcie pcie-controller.1: PCIe3.0: link down mvebu-pcie pcie-controller.1: PCI host bridge to bus 0000:00 pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] pci_bus 0000:00: root bus resource [mem 0xc1000000-0xc8ffffff] pci_bus 0000:00: root bus resource [bus 00-ff] PCI: bus0: Fast back to back transfers disabled pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:04.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:05.0: bridge configuration invalid ([bus 00-00]), reconfiguring pci 0000:00:06.0: bridge configuration invalid ([bus 00-00]), reconfiguring PCI: bus1: Fast back to back transfers enabled PCI: bus2: Fast back to back transfers enabled PCI: bus3: Fast back to back transfers disabled PCI: bus4: Fast back to back transfers enabled PCI: bus5: Fast back to back transfers disabled PCI: bus6: Fast back to back transfers enabled pci 0000:00:03.0: BAR 8: assigned [mem 0xc1000000-0xc10fffff] pci 0000:00:03.0: BAR 9: assigned [mem 0xc1100000-0xc11fffff pref] pci 0000:00:05.0: BAR 8: assigned [mem 0xc1200000-0xc12fffff] pci 0000:00:05.0: BAR 9: assigned [mem 0xc1300000-0xc13fffff pref] pci 0000:00:03.0: BAR 7: assigned [io 0x10000-0x10fff] pci 0000:00:05.0: BAR 7: assigned [io 0x20000-0x20fff] pci 0000:00:01.0: PCI bridge to [bus 01] pci 0000:00:02.0: PCI bridge to [bus 02] pci 0000:03:00.0: BAR 0: assigned [mem 0xc1000000-0xc10fffff 64bit] pci 0000:03:00.0: BAR 6: assigned [mem 0xc1100000-0xc117ffff pref] pci 0000:03:00.0: BAR 2: assigned [io 0x10000-0x100ff] pci 0000:00:03.0: PCI bridge to [bus 03] pci 0000:00:03.0: bridge window [io 0x10000-0x10fff] pci 0000:00:03.0: bridge window [mem 0xc1000000-0xc10fffff] pci 0000:00:03.0: bridge window [mem 0xc1100000-0xc11fffff pref] pci 0000:00:04.0: PCI bridge to [bus 04] pci 0000:05:00.0: BAR 0: assigned [mem 0xc1200000-0xc121ffff] pci 0000:05:00.0: BAR 1: assigned [mem 0xc1220000-0xc123ffff] pci 0000:05:00.0: BAR 6: assigned [mem 0xc1300000-0xc131ffff pref] pci 0000:05:00.0: BAR 2: assigned [io 0x20000-0x2001f] pci 0000:00:05.0: PCI bridge to [bus 05] pci 0000:00:05.0: bridge window [io 0x20000-0x20fff] pci 0000:00:05.0: bridge window [mem 0xc1200000-0xc12fffff] pci 0000:00:05.0: bridge window [mem 0xc1300000-0xc13fffff pref] pci 0000:00:06.0: PCI bridge to [bus 06] PCI: enabling device 0000:00:01.0 (0140 -> 0143) PCI: enabling device 0000:00:02.0 (0140 -> 0143) PCI: enabling device 0000:00:03.0 (0140 -> 0143) PCI: enabling device 0000:00:04.0 (0140 -> 0143) PCI: enabling device 0000:00:05.0 (0140 -> 0143) PCI: enabling device 0000:00:06.0 (0140 -> 0143) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
