On 2021-02-22 15:47, Nicolas Saenz Julienne wrote:
Hi everyone, Raspberry Pi 4, a 64bit arm system on chip, contains a PCIe bus that can't handle 64bit accesses to its MMIO address space, in other words, writeq() has to be split into two distinct writel() operations. This isn't ideal, as it misrepresents PCI's promise of being able to treat device memory as regular memory, ultimately breaking a bunch of PCI device drivers[1]. I'd like to have a go at fixing this in a way that can be distributed in a generic distro without prejudice to other users. AFAIK there is no way to detect this limitation through generic PCIe capabilities, so one solution would be to expose it through firmware (devicetree in this case), and pass the limitations through 'struct device' so as for the drivers to choose the right access method in a way that doesn't affect performance much[2]. All in all, most of this doesn't need to be PCI-centric as the property could be applied to any MMIO bus.
It is indeed something that people can get wrong with internal buses as well - for example commit f2d9848aeb9f is such a workaround, also conveniently illustrating the case of significant functionality having to be disabled where the device *does* require 64-bit atomicity for correctness.
Working around kernel I/O accessors is all very well, but another concern for PCI in particular is when things like framebuffer memory can get mmap'ed into userspace (or even memremap'ed within the kernel). Even in AArch32, compiled code may result in 64-bit accesses being generated depending on how the CPU and interconnect handle LDRD/STRD/LDM/STM/etc., so it's basically not safe to ever let that happen at all.
Robin.
Thoughts? Opinions? Is it overkill just for a single SoC? Regards, Nicolas [1] https://github.com/raspberrypi/linux/issues/4158#issuecomment-782351510 [2] Things might get even weirder as the order in which the 32bit operations are performed might matter (low/high vs high/low). _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@xxxxxxxxxxxxxxxxxxx http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
