On Monday 03 February 2014 16:31:37 Jason Gunthorpe wrote: > Specifying 'use EHCI, AHCI, etc' - which are all PCI based standards > without clearly specifying exactly how PCI is suppose to work is > completely bonkers. > > What is needed is a spec that says: > 1) Here is how you generate config TLPs. A MMIO region that > conforms to the already specified x86 ECAM would > be perfect > 2) Here is a dword by dword break down of the entire config space in > a SOC. Here is where a on-board AHCI controller must show up in > config space. Here is how an external PCI-E port must show > up. Etc. Most of this is already specified, but it clearly needs > to be layed out explicitly for ARM SOCs to actually follow it. > 3) Here is how you specify the aperture(s) associated with PCI BAR's > and bridge windows in config space. And yes: The CONFIG SPACE > BARS MUST WORK. > 4) Here is how MSI works, these are the values you put in the > address/data and here is how you collect the interrupt. > 5) Here is how Legacy INTx must be mapped into the GIC. > > This is what x86 does, and they have been doing it well for 10 > years. If you want to play in the server game you have to properly > implement PCI. I'm pretty sure the authors of the SBSA actually thought that was what they wrote, by referring to external specifications (pci-3.0, ehci, ahci, ...). However, it seems they were either foolish enough to believe that hardware designers would follow these specs, or they were intentionally misled and got talked into putting ambiguous terminology in because there were already hardware designs that are not exactly in line with the spirit of the SBSA but can be argued to be conforming to the text for a lax interpretation. I think EHCI is a much better example than PCI here, because the spec has exactly one line to say about it, where it spends a whole chapter on PCI. Here is how a sane person would read SBSA to create a compliant implementation: I have to use EHCI version 1.1 to provide USB-2.0 support. EHCI is a PCI device, so I'll put it behind a PCIe port that complies to the PCIe section of the SBSA. Since EHCI by itself only provides high-speed USB, and USB-2.0 mandates I provide low-speed and full-speed as well, I have to add a USB hub device. It would have been easier to just use OHCI for these, but SBSA says I can't. Now I want to integrate the EHCI into my SoC and not waste one of my precious PCIe root ports, so I have to create another PCI domain with its own ECAM compliant config space to put it into. Fortunately SBSA lets me add an arbitrary number of PCI domains, as long as they are all strictly compliant. To software it will look exactly as if it was on an external card, I just have to ensure the boot loader correctly sets up the clocks and the phy before an SBSA compliant OS gets loaded, all runtime power management will get handled through the EHCI-1.1 energy-efficiency extensions. Here is how a crazy person would read the same sentence in the SBSA: I have an IP block that implements the EHCI register set, that should be good enough. It's not a fast device, so I can put it on a non-coherent bus. Since the SoC will be used for networking, I'll put the registers into big-endian configuration to make it easier for the OS to access. I'm not allowed to have USB-1.1 according to SBSA, so I can get away without a hub or an extra OHCI. I can't support MSI because it's not a PCI device, and the GIC is pretty full, so I'll just connect the IRQ line to the GPIO controller. In order to do better power management, I'll design a fancy PHY that the device driver will manage for implementing autosuspend. I should also give the OS fine-grained control over the clocks, but it will have to share the clock domain with the other devices on the same edge of the chip. The EHCI device is not part of PCI, which measn I don't have to use the standard SMMU. However, my EHCI implementation can only do 32-bit DMA, and I'll have to design my own IOMMU to let it access the entire memory range. USB-OTG is a great feature and we already paid for having this in our EHCI implementation, better make sure it comes up in endpoint mode after waking up from power saving. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
