Am 14.01.19 um 20:13 schrieb Will Deacon: > On Mon, Jan 14, 2019 at 07:07:54PM +0000, Koenig, Christian wrote: >> Am 14.01.19 um 18:32 schrieb Ard Biesheuvel: >> - The reason remapping the CPU side as cacheable does work (which I >> did test) is because the GPU's uncacheable accesses (which I assume >> are made using the NoSnoop PCIe transaction attribute) are actually >> emitted as cacheable in some cases. >> . On my AMD Seattle, with or without SMMU (which is stage 2 only), I >> must use cacheable accesses from the CPU side or things are broken. >> This might be a h/w flaw, though. >> . On systems with stage 1+2 SMMUs, the driver uses stage 1 >> translations which always override the memory attributes to cacheable >> for DMA coherent devices. This is what is affecting the Cavium >> ThunderX2 (although it appears the attributes emitted by the RC may be >> incorrect as well.) >> >> The latter issue is a shortcoming in the SMMU driver that we have to >> fix, i.e., it should take care not to modify the incoming attributes >> of DMA coherent PCIe devices for NoSnoop to be able to work. >> >> So in summary, the mismatch appears to be between the CPU accessing >> the vmap region with non-cacheable attributes and the GPU accessing >> the same memory with cacheable attributes, resulting in a loss of >> coherency and lots of visible corruption. >> >> Actually it is the other way around. The CPU thinks some data is in the >> cache and the GPU only updates the system memory version because the >> snoop flag is not set. >> >> >> That doesn't seem to be what is happening. As far as we can tell from >> our experiments, all inbound transactions are always cacheable, and so >> the only way to make things work is to ensure that the CPU uses the >> same attributes. >> >> >> Ok that doesn't make any sense. If inbound transactions are cacheable or not is >> irrelevant when the CPU always uses uncached accesses. >> >> See on the PCIe side you have the snoop bit in the read/write transactions >> which tells the root hub if the device wants to snoop caches or not. >> >> When the CPU accesses some memory as cached then devices need to snoop the >> cache for coherent accesses. >> >> When the CPU accesses some memory as uncached then devices can disable snooping >> to improve performance, but when they don't do this it is mandated by the spec >> that this still works. > Which spec? The PCIe spec. The snoop bit (or rather the NoSnoop) in the transaction is perfectly optional IIRC. > The Arm architecture (and others including Power afaiu) doesn't > guarantee coherency when memory is accessed using mismatched cacheability > attributes. Well what exactly goes wrong on ARM? As far as I know Power doesn't really supports un-cached memory at all, except for a very very old and odd configuration with AGP. I mean in theory I agree that devices should use matching cacheability attributes, but in practice I know of quite a bunch of devices/engines which fails to do this correctly. Regards, Christian. > > Will _______________________________________________ dri-devel mailing list dri-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/dri-devel
