Hi Daniel,Le mar., mars 29 2022 at 10:54:43 +0200, Daniel Vetter <daniel@xxxxxxxx> a écrit :
On Mon, Feb 07, 2022 at 01:01:40PM +0000, Paul Cercueil wrote:Document the new DMABUF based API. v2: - Explicitly state that the new interface is optional and is not implemented by all drivers. - The IOCTLs can now only be called on the buffer FD returned by IIO_BUFFER_GET_FD_IOCTL.- Move the page up a bit in the index since it is core stuff and notdriver-specific. Signed-off-by: Paul Cercueil <paul@xxxxxxxxxxxxxxx> --- Documentation/driver-api/dma-buf.rst | 2 +Documentation/iio/dmabuf_api.rst | 94 ++++++++++++++++++++++++++++Documentation/iio/index.rst | 2 + 3 files changed, 98 insertions(+) create mode 100644 Documentation/iio/dmabuf_api.rstdiff --git a/Documentation/driver-api/dma-buf.rst b/Documentation/driver-api/dma-buf.rstindex 2cd7db82d9fe..d3c9b58d2706 100644 --- a/Documentation/driver-api/dma-buf.rst +++ b/Documentation/driver-api/dma-buf.rst @@ -1,3 +1,5 @@ +.. _dma-buf: + Buffer Sharing and Synchronization ==================================diff --git a/Documentation/iio/dmabuf_api.rst b/Documentation/iio/dmabuf_api.rstnew file mode 100644 index 000000000000..43bb2c1b9fdc --- /dev/null +++ b/Documentation/iio/dmabuf_api.rst @@ -0,0 +1,94 @@ +=================================== +High-speed DMABUF interface for IIO +=================================== + +1. Overview +=========== ++The Industrial I/O subsystem supports access to buffers through a file-based +interface, with read() and write() access calls through the IIO device's dev+node. ++It additionally supports a DMABUF based interface, where the userspace +application can allocate and append DMABUF objects to the buffer's queue. +This interface is however optional and is not available in all drivers.+ +The advantage of this DMABUF based interface vs. the read() +interface, is that it avoids an extra copy of the data between the +kernel and userspace. This is particularly useful for high-speed+devices which produce several megabytes or even gigabytes of data per+second. + +The data in this DMABUF interface is managed at the granularity of+DMABUF objects. Reducing the granularity from byte level to block level +is done to reduce the userspace-kernelspace synchronization overhead+since performing syscalls for each byte at a few Mbps is just not +feasible. ++This of course leads to a slightly increased latency. For this reason an +application can choose the size of the DMABUFs as well as how many it+allocates. E.g. two DMABUFs would be a traditional double buffering +scheme. But using a higher number might be necessary to avoid+underflow/overflow situations in the presence of scheduling latencies.So this reads a lot like reinventing io-uring with pre-registered O_DIRECT memory ranges. Except it's using dma-buf and hand-rolling a lot of piecesinstead of io-uring and O_DIRECT.
I don't see how io_uring would help us. It's an async I/O framework, does it allow us to access a kernel buffer without copying the data? Does it allow us to zero-copy the data to a network interface?
At least if the entire justification for dma-buf support is zero-copysupport between the driver and userspace it's _really_ not the right toolfor the job. dma-buf is for zero-copy between devices, with cpu accessfrom userpace (or kernel fwiw) being very much the exception (and oftenflat-out not supported at all).
We want both. Using dma-bufs for the driver/userspace interface is a convenience as we then have a unique API instead of two distinct ones.
Why should CPU access from userspace be the exception? It works fine for IIO dma-bufs. You keep warning about this being a terrible design, but I simply don't see it.
Cheers, -Paul
+ +2. User API +=========== + +``IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *)`` +---------------------------------------------------------------- ++Each call will allocate a new DMABUF object. The return value (if not +a negative errno value as error) will be the file descriptor of the new+DMABUF. + +``IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *)`` +-------------------------------------------------------- ++Place the DMABUF object into the queue pending for hardware process.+ +These two IOCTLs have to be performed on the IIO buffer's file +descriptor, obtained using the `IIO_BUFFER_GET_FD_IOCTL` ioctl. + +3. Usage +======== + +To access the data stored in a block by userspace the block must be+mapped to the process's memory. This is done by calling mmap() on the+DMABUF's file descriptor. + +Before accessing the data through the map, you must use the +DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the +DMA_BUF_SYNC_START flag, to make sure that the data is available.+This call may block until the hardware is done with this block. Once +you are done reading or writing the data, you must use this ioctl again+with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the +kernel's queue. ++If you need to know when the hardware is done with a DMABUF, you can+poll its file descriptor for the EPOLLOUT event. ++Finally, to destroy a DMABUF object, simply call close() on its file+descriptor. ++For more information about manipulating DMABUF objects, see: :ref:`dma-buf`.+ +A typical workflow for the new interface is: + + for block in blocks: + DMABUF_ALLOC block + mmap block + + enable buffer + + while !done + for block in blocks: + DMABUF_ENQUEUE block + + DMABUF_SYNC_START block + process data + DMABUF_SYNC_END block + + disable buffer + + for block in blocks: + close blockdiff --git a/Documentation/iio/index.rst b/Documentation/iio/index.rstindex 58b7a4ebac51..669deb67ddee 100644 --- a/Documentation/iio/index.rst +++ b/Documentation/iio/index.rst @@ -9,4 +9,6 @@ Industrial I/O iio_configfs + dmabuf_api + ep93xx_adc -- 2.34.1-- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
