Hi Gustavo, On Thu, Sep 07, 2017 at 03:42:11PM -0300, Gustavo Padovan wrote:
From: Gustavo Padovan <gustavo.padovan@xxxxxxxxxxxxx> Hi, Refer to the documentation on the first patch for the details. The previous iteration is here: https://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg118077.html The 2nd patch proposes an userspace API for fences, then on patch 3 we prepare to the addition of in-fences in patch 4, by introducing the infrastructure on vb2 to wait on an in-fence signal before queueing the buffer in the driver. Patch 5 fix uvc v4l2 event handling and patch 6 configure q->dev for vivid drivers to enable to subscribe and dequeue events on it. Patches 7-9 enables support to notify BUF_QUEUED events, the event send to userspace the out-fence fd and the index of the buffer that was queued. Patches 10-11 add support to mark queues as ordered. Finally patches 12 and 13 add more fence infrastructure to support out-fences, patch 13 exposes close_fd() and patch 14 adds support to out-fences. It only works for ordered queues for now, see open question at the end of the letter. Test tool can be found at: https://git.collabora.com/cgit/user/padovan/v4l2-test.git/ Main Changes ------------ * out-fences: change in behavior: the out-fence fd now comes out of the BUF_QUEUED event along with the buffer id.
The more I think about this, the more unfortunate it seems. Especially for our use-case (m2m engine which sits in front of the display processor to convert the format of some buffers), having to wait for the in-fence to signal before we can get an out-fence removes a lot of the advantages of having fences at all. Ideally, we'd like to queue up our m2m work (while the GPU is still rendering that buffer, holding the in-fence), immediately get the out-fence for the m2m work, and pass that to DRM as the in-fence for display. With the current behaviour we need to wait in userspace before we can pass the buffer to display. Wouldn't it be possible to enforce that the buffers aren't queued out-of-order in VB2? An easy way might be to (in qbuf) set a buffer's ->in_fence to be a fence_array of all the ->in_fences from the buffers before it in the queue (and its own). That would then naturally order the enqueue-ing in the driver, and allow you to return the out-fence immediately. This would also solve your output devices question from below - a buffer can never get queued in the driver until all of the buffers which were QBUF'd before it are queued in the driver. Cheers, -Brian
All other changes are recorded on the patches' commit messages. Open Questions -------------- * non-ordered devices, like m2m: I've been thinking a lot about those and one possibility is to have a way to tell userspace that the queue is not ordered and then associate the fence with the current buffer in QBUF instead of the next one to be queued. Of course, there won't be any ordering between the fences. But it may be enough for userspace to take advantage of Explicit Synchronization in such cases. Any thoughts? * OUTPUT devices and in-fence. If I understood OUTPUT devices correctly it is desirable to queue the buffers to the driver in the same order we received them from userspace. If that is correct, shouldn't we add some mechanism to prevent buffer whose fence signaled to jump ahead of others? Gustavo Padovan (14): [media] v4l: Document explicit synchronization behaviour [media] vb2: add explicit fence user API [media] vb2: check earlier if stream can be started [media] vb2: add in-fence support to QBUF [media] uvc: enable subscriptions to other events [media] vivid: assign the specific device to the vb2_queue->dev [media] v4l: add V4L2_EVENT_BUF_QUEUED event [media] vb2: add .buffer_queued() to notify queueing in the driver [media] v4l: add support to BUF_QUEUED event [media] vb2: add 'ordered' property to queues [media] vivid: mark vivid queues as ordered [media] vb2: add infrastructure to support out-fences fs/files: export close_fd() symbol [media] vb2: add out-fence support to QBUF Javier Martinez Canillas (1): [media] vb2: add videobuf2 dma-buf fence helpers Documentation/media/uapi/v4l/buffer.rst | 19 ++ Documentation/media/uapi/v4l/vidioc-dqevent.rst | 23 +++ Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 ++++ Documentation/media/videodev2.h.rst.exceptions | 1 + drivers/android/binder.c | 2 +- drivers/media/platform/vivid/vivid-core.c | 15 +- drivers/media/usb/cpia2/cpia2_v4l.c | 2 +- drivers/media/usb/uvc/uvc_v4l2.c | 2 +- drivers/media/v4l2-core/Kconfig | 1 + drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 4 +- drivers/media/v4l2-core/v4l2-ctrls.c | 6 +- drivers/media/v4l2-core/videobuf2-core.c | 221 ++++++++++++++++++++++-- drivers/media/v4l2-core/videobuf2-v4l2.c | 63 ++++++- fs/file.c | 5 +- fs/open.c | 2 +- include/linux/fdtable.h | 2 +- include/media/videobuf2-core.h | 63 ++++++- include/media/videobuf2-fence.h | 49 ++++++ include/uapi/linux/videodev2.h | 15 +- 19 files changed, 489 insertions(+), 37 deletions(-) create mode 100644 include/media/videobuf2-fence.h -- 2.13.5
