On Wed, 21 Jun 2023 12:45:35 +0300 Serge Semin <fancer.lancer@xxxxxxxxx> wrote: > > I thought that using a read will solve the issue like the gpio_nand driver > > (gpio_nand_dosync) > > AFAICS The io_sync dummy-read there is a workaround to fix the > bus-reordering within the SoC bus. In this case we have a PCIe bus > which is supposed to guarantee the strong order with the exception I > described above or unless there is a bug someplace in the PCIe fabric. > > > but I didn't thought of a cache that could return the value > > of the read even if the write doesn't fully happen. In the case of a cache > > how could we know that the write is done without using a delay? > > MMIO mapping is platform dependent and low-level driver dependent. > That's why I asked many times about the platform you are using and the > low-level driver that probes the eDMA engine. It would be also useful > to know what PCIe host controller is utilized too. > > Mainly MMIO spaces are mapped in a way to bypass the caching. But in > some cases it might be useful to map an MMIO space with additional > optimizations like Write-combining. For instance it could be > effectively done for the eDMA linked-list BAR mapping. Indeed why > would you need to send each linked-list byte/word/dword right away to > the device while you can combine them and send all together, then > flush the cache and only after that start the DMA transfer? Another > possible reason of the writes reordering could be in a way the PCIe > host outbound memory window (a memory region accesses to which are > translated to the PCIe bus transfers) is configured. For instance DW > PCIe Host controller outbound MW config CSR has a special flag which > enables setting a custom PCIe bus TLPs (packets) attribute. As I > mentioned above that attribute can affect the TLPs order: make it > relaxed or ID-based. > > Of course we can't reject a possibility of having some delays hidden > inside your device which may cause writes to the internal memory > landing after the writes to the CSRs. But that seems too exotic to be > considered as the real one for sure until the alternatives are > thoroughly checked. > > What I was trying to say that your problem can be caused by some much > more frequently met reason. If I were you I would have checked them > first and only then considered a workaround like you suggest. Thanks for you detailed answer, this was instructive. I will come back with more information if TLP flags are set. FYI the PCIe board I am currently working with is the one from Brainchip: Here is the driver: https://github.com/Brainchip-Inc/akida_dw_edma Köry
