Hi Sergey, On 10/07/2020 12.27, Serge Semin wrote: > Hello Peter > > On Fri, Jul 10, 2020 at 11:31:47AM +0300, Peter Ujfalusi wrote: >> >> >> On 10/07/2020 1.45, Serge Semin wrote: >>> Some devices may lack the support of the hardware accelerated SG list >>> entries automatic walking through and execution. In this case a burden of >>> the SG list traversal and DMA engine re-initialization lies on the >>> DMA engine driver (normally implemented by using a DMA transfer completion >>> IRQ to recharge the DMA device with a next SG list entry). But such >>> solution may not be suitable for some DMA consumers. In particular SPI >>> devices need both Tx and Rx DMA channels work synchronously in order >>> to avoid the Rx FIFO overflow. In case if Rx DMA channel is paused for >>> some time while the Tx DMA channel works implicitly pulling data into the >>> Rx FIFO, the later will be eventually overflown, which will cause the data >>> loss. So if SG list entries aren't automatically fetched by the DMA >>> engine, but are one-by-one manually selected for execution in the >>> ISRs/deferred work/etc., such problem will eventually happen due to the >>> non-deterministic latencies of the service execution. >> > >> It is not really the number of sg nents which is the problem, but the >> combination of total number of bytes _and_ the number of nents used to >> map them. > > No, in the case described above the latency only matters. Sure, the latency comes in (at least for EDMA) when moving from one sub-sg_list to the next one when needing to break the long list into smaller chunks. In the sub-completion we need to move to the next chunk. > The length of the SG > entries doesn't (at most of extents). The same is with the nents. If there are > more than one SG entries in the Rx SG-list handled with the non-deterministic > latency, the problem may happen at the moment of any Rx SG entry reload. You have latency in the hardware when progressing between the sg entries? > (In fact the DW DMAC driver is working with so called Linked-List entries, > which are used to map the SG-list entries into the list of items with length > less than or equal to the max block size the engine supports. But for > simplicity I call them the SG-list entries, since if dma_set_max_seg_size() > is specified the mapping will be uniform.) Right, I'll come back to the dma_set_max_seq_size() later for my cases ;) >> The EDMA from TI have similar limitation (we set the limit to 20 nents). >> Longer lists will be broken up to maximum of 20 segment transfers. >> This setup has bigger impact on audio (cyclic) as we need to limit the >> number of periods to not exceed this limit of 20. > > As I said the problem described above isn't about the number of entries, but > about how they are handled. If there is a latency, then the problem may and most > like will eventually happen. > > The complexity of the situation is that the DW DMAC driver may split a SG-list > entry up if its length exceeds max block size the engine supports. That's > why in order to fix the problem described in the patch log the DMA client driver > developed needs to take into account the next two points: > 1) Detect the maximum number of SG entries the DMA engine can handle without the > non-deterministic latency (you called it max_sg_nents_burst). So the client > drivers would know, that the DMA-channels responsible for Tx and Rx transfers > may be executed with latencies. This is really useful information for the clients. But in some cases they just don't care how the list is going to be processed, given if they have large enough internal FIFO to hold on while the DMA moves from one chunk to another. > 2) Make sure the length of each SG-list entry doesn't exceed the max block size > the DW DMAC supports. If some of them does, then the DW DMAC code will break > these SG-entries up into the smaller DMA Linked-list entries, which will get us > back to the re-submission latency problem described in the patch log. This > peculiarity is covered by calling dma_set_max_seg_size() method in the DW DMAC > driver (at least for SPI subsystem it works out-of-box). This is what does not work for me for neither EDMA or sDMA... The max_seq_size depends on the src/dst_addr_width the peripheral is configured. It is different for each DMA_SLAVE_BUSWIDTH_*_BYTES _and_ also for the burst used. It is on my to-do list to clean up and send this: https://github.com/omap-audio/linux-audio/commit/dd81de1a343810468b6e7a601e90a9161f46cab1 For EDMA the implementation is: https://github.com/omap-audio/linux-audio/commit/ee93c45667d0f250baa1e224e5d1266fae2e7c80 along the same lines for sDMA: https://github.com/omap-audio/linux-audio/commit/d6ac68692efcdac8ec233c70b802d37f594d6d4d What is obviously missing is that if the driver does not implement the device_get_max_len, then dma_get_max_seg_size() of the device should be returned. Other option would be: https://github.com/omap-audio/linux-audio/commit/95dda19c92a38437741850309374a2ca4ef1e361 I'm still hesitating which would be more generic to be acceptable. And to make things a bit more interesting the UDMA have also different dma_sg_len() supported depending on what mode the channel is used (TR or packet mode), but that's another story in itself. >> The sDMA on the other hand has different limits. Earlier versions >> without linking support can execute one SG chunk at the time and needs >> to reconfigure for the next one -> max_sg_nents is 1 for the older sDMAs... > > Yes, this is our case. Interesting. So you prefer that your clients will use dmaengine_prep_slave_single() if possible? >>> In order to let the DMA consumer know about the DMA device capabilities >>> regarding the hardware accelerated SG list traversal we introduce the >>> max_sg_list capability. It is supposed to be initialized by the DMA engine >>> driver with 0 if there is no limitation for the number of SG entries >>> atomically executed and with non-zero value if there is such constraints, >>> so the upper limit is determined by the number set to the property. >>> >>> Suggested-by: Andy Shevchenko <andriy.shevchenko@xxxxxxxxxxxxxxx> >>> Signed-off-by: Serge Semin <Sergey.Semin@xxxxxxxxxxxxxxxxxxxx> >>> Reviewed-by: Andy Shevchenko <andriy.shevchenko@xxxxxxxxxxxxxxx> >>> Cc: Alexey Malahov <Alexey.Malahov@xxxxxxxxxxxxxxxxxxxx> >>> Cc: Thomas Bogendoerfer <tsbogend@xxxxxxxxxxxxxxxx> >>> Cc: Arnd Bergmann <arnd@xxxxxxxx> >>> Cc: Rob Herring <robh+dt@xxxxxxxxxx> >>> Cc: linux-mips@xxxxxxxxxxxxxxx >>> Cc: devicetree@xxxxxxxxxxxxxxx >>> >>> --- >>> >>> Changelog v3: >>> - This is a new patch created as a result of the discussion with Vinud and >>> Andy in the framework of DW DMA burst and LLP capabilities. >>> >>> Changelog v4: >>> - Fix of->if typo. It should be definitely of. >>> --- >>> drivers/dma/dmaengine.c | 1 + >>> include/linux/dmaengine.h | 8 ++++++++ >>> 2 files changed, 9 insertions(+) >>> >>> diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c >>> index b332ffe52780..ad56ad58932c 100644 >>> --- a/drivers/dma/dmaengine.c >>> +++ b/drivers/dma/dmaengine.c >>> @@ -592,6 +592,7 @@ int dma_get_slave_caps(struct dma_chan *chan, struct dma_slave_caps *caps) >>> caps->directions = device->directions; >>> caps->min_burst = device->min_burst; >>> caps->max_burst = device->max_burst; >>> + caps->max_sg_nents = device->max_sg_nents; >>> caps->residue_granularity = device->residue_granularity; >>> caps->descriptor_reuse = device->descriptor_reuse; >>> caps->cmd_pause = !!device->device_pause; >>> diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h >>> index 0c7403b27133..a7e4d8dfdd19 100644 >>> --- a/include/linux/dmaengine.h >>> +++ b/include/linux/dmaengine.h >>> @@ -467,6 +467,9 @@ enum dma_residue_granularity { >>> * should be checked by controller as well >>> * @min_burst: min burst capability per-transfer >>> * @max_burst: max burst capability per-transfer >>> + * @max_sg_nents: max number of SG list entries executed in a single atomic >>> + * DMA tansaction with no intermediate IRQ for reinitialization. Zero >>> + * value means unlimited number of entries. >> > >> Without looking at the comment the name max_sg_nents implies that the >> DMA can not handle longer lists, but it is not really true. >> max_sg_nents_burst might be a bit cleaner for the first look? > > Seems reasonable. I also thought of a better naming so there wouldn't be need to > read a comment in order get a notion what exactly the parameter is responsible > for. Although at the moment of the patchset preparation nothing better got in my > mind. > > I like what you suggest, but I'd better make it "max_sg_burst" or "max_sg_chain". > What do you think? Since we should be able to handle longer lists and this is kind of a hint for clients that above this number of nents the list will be broken up to smaller 'bursts', which when traversing could cause latency. sg_chunk_len might be another candidate. > Vinod? > Andy (since you've already acked the patch)? > > -Sergey - Péter Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
