Hello David, On 08/03/2017 14:54, David Laight wrote: > Mason wrote: > >> 2) On my platform, there are two revisions of the PCIe controller. >> Rev1 muxes config and mem inside a 256 MB window, and doesn't support >> I/O space. >> Rev2 muxes all 3 spaces inside a 256 MB window. > > Don't think config space fits. > With the 'obvious' mapping the 'bus number' is in the top > 8 bits of the address. https://www.kernel.org/doc/Documentation/devicetree/bindings/pci/host-generic-pci.txt cfg_offset(bus, device, function, register) = bus << 20 | device << 15 | function << 12 | register 8 bits for bus, 5 bits for device, 3 bits for function, 12 bits for reg offset 1 MB per bus, 256 buses max => 256 MB max Supporting "only" 64 buses is good enough, I believe. >> 3) What happens if a device requires more than 256 MB of >> mem space? (Is that common? What kind of device? GPUs?) >> Our controller supports a remapping "facility" to add an >> offset to the bus address. Is such a feature supported >> by Linux at all? The problem is that this creates >> another race condition, as setting the offset register >> before an access may occur concurrently on two cores. >> Perhaps 256 MB is plenty on a 32-bit embedded device? > > GPUs tend to have their own paging scheme. > So don't need humongous windows. > I'm not sure how much space is really needed. > 32bit x86 reserve the top 1GB of physical address for PCI(e). I'm hoping 128 MB mem is enough. The two cards I have that are correctly detected request 8 KB. (I have other cards that are not enumerated at all... No idea why at the moment.) >> 4) The HW dev is considering the following fix. >> Instead of muxing the address spaces, provide smaller >> exclusive spaces. For example >> [0x5000_0000, 0x5400_0000] for config (64MB) >> [0x5400_0000, 0x5800_0000] for I/O (64MB) >> [0x5800_0000, 0x6000_0000] for mem (128MB) > > You almost certainly don't need more than 64k of IO. Good to know. > Config space isn't dense, you (probably) need 25 bits to get a 2nd bus number. > Even 256MB constrains you to 16 bus numbers. Unless I got the math wrong, it's 20 bits (1 MB) per bus. So 64 MB allows 64 buses. > Is this an ARM cpu inside an altera (now intel) fpga?? > There is a nasty bug in their PCIe to avalon bridge logic (fixed in quartus 16.1). The PCIe controller is from PLDA, and it's embedded in a SoC where the CPU is a multi-core ARM Cortex A9 MP. Regards.
