Hi,
On 17/12/2018 22:21, Niklas Cassel wrote:
> On Mon, Dec 17, 2018 at 06:19:32PM +0100, Gustavo Pimentel wrote:
>> Add Synopsys eDMA IP driver (version 0) to Linux kernel. This IP is generally
>> distributed with Synopsys PCIe EndPoint IP (depends of the use and licensing
>> agreement), which supports:
>> - legacy and unroll modes
>> - 16 independent and concurrent channels (8 write + 8 read)
>> - supports linked list (scatter-gather) transfer
>> - each linked list descriptor can transfer from 1 byte to 4 Gbytes
>> - PCIe EndPoint glue-logic
>>
>> Gustavo Pimentel (6):
>> dmaengine: Add Synopsys eDMA IP core driver
>> dmaengine: Add Synopsys eDMA IP version 0 support
>> dmaengine: Add Synopsys eDMA IP version 0 debugfs support
>> PCI: Add Synopsys endpoint EDDA Device id
>> dmaengine: Add Synopsys eDMA IP PCIe glue-logic
>> MAINTAINERS: Add Synopsys eDMA IP driver maintainer
>
> Hello Gustavo,
>
> Nice to see support for the embedded DMA controller used in the DWC PCIe
> controller.
Yes, I included you because you were from the beginning one of those who
requested this feature. :)
>
> Some kind of documentation in Documentation/ would be nice.
Yes, I thought so too, but under of DOCUMENTATION folder doesn't have anything
related with DMA drivers implementations either on topic of DMA or PCI.
I thought maybe putting on the dw-edma folder but that doesn't follow the
standard documentation flow.
>
>
> The eDMA can be used both while the PCIe controller is in EP and RC mode,
> right?
For now I can not give this certainty. My setup only have the eDMA on the EP
side and has the eDMA mapped on PCI BAR, which allows me to configure it through
PCI.
> I guess that this support is only to use the eDMA for RC mode?
For now this driver runs on the RC side (host/CPU) and configures the eDMA EP.
I've some plans to support eDMA configuration but on the EP side. I'm guessing
that also typical use case.
>
> Do you have any benchmarks to share? (Perhaps with different buffer sizes).
I'm currently working on the dw-edma-test driver (similar to dmatest driver)
that supports DEV_TO_MEM and MEM_TO_DEV directions instead of MEM_TO_MEM
(dmatest), and adapted to WRITE and READ channels with multiple threads. After
finishing this driver I'll able to do some benchmarks and comparisons. :)
Given your enthusiasm, when I'm finished we could do some brainstorm and define
some test cases / benchmarks, what do you think?
> Not sure what we will see, lower CPU utilization but more IRQs?
Probably yes. Let me explain you
With this implementation, you can transfer several chunks (a chunk is linked
list of burst elements).
Each burst basically defines the source, destination addresses and size to
transfer (from 1 Byte to 4 GB).
This linked list will be written on EP's RAM (mapped on a PCI BAR) which the HW
block will consume it. The linked list size is dependent of this EP's RAM size,
i.e. 8 KB will allow 340 elements (bursts).
On my currently setup I'm testing the driver implementation like this:
- 4 chunks
- each chunk is composed by 340 elements (bursts)
- each burst have a transfer size of 100 bytes (can be up to 4GB and the size
can be also different for each burst)
Chunk #0
|--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list)
Chunk #1
|--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list)
Chunk #2
|--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list)
Chunk #3
|--> Burst #0 --> ... --> Burst #34 -> IRQ (End transfer)
In total I'm writing ((4 * 340) + 35) * 100 = 139500 Bytes <=> 136 KB
> Have you thought about doing IRQ coalescing?
No. I'm only focus to implement the driver for now. After the implementation and
the benchmarks, maybe we could think about it. :)
>
> Usually we need dmas and dma-names in device tree, however, maybe that
> isn't needed here, since it is an embedded DMA controller.
> Don't we even need/want a DT property in the PCIe node, that says if we
> want to enable the eDMA or not?
That will be necessary in the future when we have eDMA in RC or when we
implement the driver to control the eDMA on the EP side.
>
> Usually a PCIe EP driver will call pci_ioremap_bar() and then use
> readl()/writel() directly.
> But since we now need to setup a DMA transfer, and wait for an IRQ,
> how does this work?
I gave a brief explanation in a previous comment in this email (see info about
the chunks and bursts)
I'm dividing this patch series like this:
- eDMA core + eDMA core v0 driver (implementing the interface with DMAengine
controller APIs and the interface with the eDMA HW block)
- eDMA PCIe glue-logic driver (attaches to EP and provides memory access to
eDMA core driver), for now is attach to Synopsys Vendor and Device ID and has a
platform data specific to our EP.
- eDMA Test driver (is a sample driver which aims to test the eDMA, but also
provides a example how an customer driver would use the eDMA driver, through the
DMAengine client APIs) (under developing, I hope to make it available before
xmas :))
Sounds logic?
>
> I have a board that has a CC_DMA_ENABLE set, so if you give me some
> detailed instructions how to test, I might be able to provide you with a
> Tested-by.
Ok, let me just finish the eDMA test and append it to the patch series.
I'm happy to help you with this. :)
Thanks!
Gustavo
>
>
> Kind regards,
> Niklas
>
>
>>
>> MAINTAINERS | 7 +
>> drivers/dma/Kconfig | 2 +
>> drivers/dma/Makefile | 1 +
>> drivers/dma/dw-edma/Kconfig | 18 +
>> drivers/dma/dw-edma/Makefile | 7 +
>> drivers/dma/dw-edma/dw-edma-core.c | 845 +++++++++++++++++++++++++++++++
>> drivers/dma/dw-edma/dw-edma-core.h | 147 ++++++
>> drivers/dma/dw-edma/dw-edma-pcie.c | 258 ++++++++++
>> drivers/dma/dw-edma/dw-edma-v0-core.c | 361 +++++++++++++
>> drivers/dma/dw-edma/dw-edma-v0-core.h | 26 +
>> drivers/dma/dw-edma/dw-edma-v0-debugfs.c | 357 +++++++++++++
>> drivers/dma/dw-edma/dw-edma-v0-debugfs.h | 24 +
>> drivers/dma/dw-edma/dw-edma-v0-regs.h | 145 ++++++
>> drivers/misc/pci_endpoint_test.c | 2 +-
>> include/linux/dma/edma.h | 43 ++
>> include/linux/pci_ids.h | 1 +
>> 16 files changed, 2243 insertions(+), 1 deletion(-)
>> create mode 100644 drivers/dma/dw-edma/Kconfig
>> create mode 100644 drivers/dma/dw-edma/Makefile
>> create mode 100644 drivers/dma/dw-edma/dw-edma-core.c
>> create mode 100644 drivers/dma/dw-edma/dw-edma-core.h
>> create mode 100644 drivers/dma/dw-edma/dw-edma-pcie.c
>> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.c
>> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.h
>> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.c
>> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.h
>> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-regs.h
>> create mode 100644 include/linux/dma/edma.h
>>
>> Signed-off-by: Gustavo Pimentel <gustavo.pimentel@xxxxxxxxxxxx>
>> Cc: Vinod Koul <vkoul@xxxxxxxxxx>
>> Cc: Andy Shevchenko <andriy.shevchenko@xxxxxxxxxxxxxxx>
>> Cc: Russell King <rmk+kernel@xxxxxxxxxxxxxxx>
>> Cc: Eugeniy Paltsev <paltsev@xxxxxxxxxxxx>
>> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@xxxxxxx>
>> Cc: Bjorn Helgaas <bhelgaas@xxxxxxxxxx>
>> Cc: Kishon Vijay Abraham I <kishon@xxxxxx>
>> Cc: Niklas Cassel <niklas.cassel@xxxxxxxxxx>
>> Cc: Joao Pinto <jpinto@xxxxxxxxxxxx>
>> Cc: Jose Abreu <jose.abreu@xxxxxxxxxxxx>
>> Cc: Luis Oliveira <lolivei@xxxxxxxxxxxx>
>> Cc: Vitor Soares <vitor.soares@xxxxxxxxxxxx>
>> Cc: Nelson Costa <nelson.costa@xxxxxxxxxxxx>
>> Cc: Pedro Sousa <pedrom.sousa@xxxxxxxxxxxx>
>>
>> --
>> 2.7.4
>>
