From: Hans Verkuil <hans.verkuil@xxxxxxxxx> Hi all, This is version 15 of the Request API series. The main changes compared to v14 are: - Squashed various patches from v14 together where applicable. - All comments to v13/v14 patches have been processed and code adapted where applicable. - Added new init_buffer vb2 buffer op to initialize struct vb2_v4l2_buffer (the request_fd field should be inited to -1). - The vb2 internal state PREPARED has been removed. Instead two new bool fields 'synced' and 'prepared' keep track of whether buffers are synced and/or prepared. This simplifies vb2 when it comes to keeping track of the buffer state and this is especially important when buffers can be in a request, but not yet queued. - After dequeueing a buffer that was part of a request the request_fd is filled in correctly, but it is also cleared (init_buffer) internally. So calling querybuf after the dqbuf will not have an associated request anymore. This makes sense since the buffer has no longer any relationship with a request after dqbuf was called. - Once you start queueing buffers in a request it is no longer allowed to queue buffers directly (i.e. without a request) until the queue is cancelled. This might be relaxed in the future, but for now this prevents mixing request and non-request buffers. - vim2m now uses delayed work instead of a timer. - It is no longer allowed to use requests for capture buffers of a memory-to-memory device. See this discussion: https://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg132202.html This series has been tested extensively with v4l2-compliance (modified to test requests) and the vim2m and vivid drivers. Regarding locking: There are two request locks used: req_queue_mutex (part of media_device) ensures high-level serialization of queueing/reiniting and canceling requests. It serializes STREAMON/OFF, close() and MEDIA_REQUEST_IOC_QUEUE. This is the top-level lock and should be taken before any others. The 'lock' spin_lock in struct media_request protects that structure and should be held for a short time only. When MEDIA_REQUEST_IOC_QUEUE is called it will validate and queue both control handlers and vb2 buffers. For control handlers the control handler lock will be taken, for vb2 buffers the vb2_queue lock will be taken. This requires that the lock field in vb2_queue is set, which is checked in videobuf2-v4l2.c. TODO: vim2m: the media topology is a bit bogus, this needs to be fixed (i.e. a proper HW entity should be added). But for now it is good enough for testing. Everything seemed to slip nicely into place while working on this, so I hope this is finally an implementation that we can proceed to upstream and build upon for complex camera pipelines in the future. This patch series is also available here: https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=reqv15 The patched v4l2-compliance is available here: https://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=request Regards, Hans Alexandre Courbot (2): videodev2.h: add request_fd field to v4l2_ext_controls Documentation: v4l: document request API Hans Verkuil (31): uapi/linux/media.h: add request API media-request: implement media requests media-request: add media_request_get_by_fd media-request: add media_request_object_find v4l2-device.h: add v4l2_device_supports_requests() helper v4l2-dev: lock req_queue_mutex v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev v4l2-ctrls: prepare internal structs for request API v4l2-ctrls: alloc memory for p_req v4l2-ctrls: use ref in helper instead of ctrl v4l2-ctrls: add core request support v4l2-ctrls: support g/s_ext_ctrls for requests vb2: store userspace data in vb2_v4l2_buffer davinci_vpfe: remove bogus vb2->state check vb2: drop VB2_BUF_STATE_PREPARED, use bool prepared/synced instead videodev2.h: Add request_fd field to v4l2_buffer vb2: add init_buffer buffer op videobuf2-core: embed media_request_object videobuf2-core: integrate with media requests videobuf2-v4l2: integrate with media requests videobuf2-core: add request helper functions videobuf2-v4l2: add vb2_request_queue/validate helpers videobuf2-core: add uses_requests flag videobuf2-v4l2: refuse qbuf if queue uses requests v4l2-mem2mem: add vb2_m2m_request_queue vim2m: add media device vim2m: use workqueue vim2m: support requests vivid: add mc vivid: add request support RFC: media-requests: add debugfs node Sakari Ailus (2): media: doc: Add media-request.h header to documentation build v4l2-mem2mem: Simplify exiting the function in v4l2_m2m_try_schedule Documentation/media/kapi/mc-core.rst | 2 + .../media/uapi/mediactl/media-controller.rst | 1 + .../media/uapi/mediactl/media-funcs.rst | 3 + .../uapi/mediactl/media-ioc-request-alloc.rst | 71 +++ .../uapi/mediactl/media-request-ioc-queue.rst | 50 ++ .../mediactl/media-request-ioc-reinit.rst | 51 ++ .../media/uapi/mediactl/request-api.rst | 219 ++++++++ Documentation/media/uapi/v4l/buffer.rst | 18 +- .../media/uapi/v4l/vidioc-g-ext-ctrls.rst | 28 +- Documentation/media/uapi/v4l/vidioc-qbuf.rst | 11 + .../media/videodev2.h.rst.exceptions | 1 + drivers/media/Makefile | 3 +- .../media/common/videobuf2/videobuf2-core.c | 244 +++++++-- .../media/common/videobuf2/videobuf2-v4l2.c | 492 +++++++++++------ drivers/media/dvb-core/dvb_vb2.c | 5 +- drivers/media/dvb-frontends/rtl2832_sdr.c | 5 +- drivers/media/media-device.c | 55 ++ drivers/media/media-devnode.c | 17 + drivers/media/media-request.c | 491 +++++++++++++++++ drivers/media/pci/bt8xx/bttv-driver.c | 2 +- drivers/media/pci/cx23885/cx23885-417.c | 2 +- drivers/media/pci/cx88/cx88-blackbird.c | 2 +- drivers/media/pci/cx88/cx88-video.c | 2 +- drivers/media/pci/saa7134/saa7134-empress.c | 4 +- drivers/media/pci/saa7134/saa7134-video.c | 2 +- .../media/platform/exynos4-is/fimc-capture.c | 2 +- drivers/media/platform/omap3isp/ispvideo.c | 4 +- drivers/media/platform/rcar-vin/rcar-core.c | 2 +- drivers/media/platform/rcar_drif.c | 2 +- .../media/platform/s3c-camif/camif-capture.c | 4 +- drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 4 +- drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 4 +- .../media/platform/soc_camera/soc_camera.c | 7 +- drivers/media/platform/vim2m.c | 93 +++- drivers/media/platform/vivid/vivid-core.c | 69 +++ drivers/media/platform/vivid/vivid-core.h | 8 + drivers/media/platform/vivid/vivid-ctrls.c | 46 +- .../media/platform/vivid/vivid-kthread-cap.c | 12 + .../media/platform/vivid/vivid-kthread-out.c | 12 + drivers/media/platform/vivid/vivid-sdr-cap.c | 16 + drivers/media/platform/vivid/vivid-vbi-cap.c | 10 + drivers/media/platform/vivid/vivid-vbi-out.c | 10 + drivers/media/platform/vivid/vivid-vid-cap.c | 10 + drivers/media/platform/vivid/vivid-vid-out.c | 10 + drivers/media/usb/cpia2/cpia2_v4l.c | 2 +- drivers/media/usb/cx231xx/cx231xx-417.c | 2 +- drivers/media/usb/cx231xx/cx231xx-video.c | 4 +- drivers/media/usb/msi2500/msi2500.c | 2 +- drivers/media/usb/tm6000/tm6000-video.c | 2 +- drivers/media/usb/uvc/uvc_queue.c | 5 +- drivers/media/usb/uvc/uvc_v4l2.c | 3 +- drivers/media/usb/uvc/uvcvideo.h | 1 + drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 14 +- drivers/media/v4l2-core/v4l2-ctrls.c | 501 +++++++++++++++++- drivers/media/v4l2-core/v4l2-dev.c | 13 + drivers/media/v4l2-core/v4l2-device.c | 3 +- drivers/media/v4l2-core/v4l2-ioctl.c | 44 +- drivers/media/v4l2-core/v4l2-mem2mem.c | 70 ++- drivers/media/v4l2-core/v4l2-subdev.c | 9 +- .../staging/media/davinci_vpfe/vpfe_video.c | 7 +- drivers/staging/media/imx/imx-media-dev.c | 2 +- drivers/staging/media/imx/imx-media-fim.c | 2 +- drivers/staging/media/omap4iss/iss_video.c | 3 +- drivers/usb/gadget/function/uvc_queue.c | 2 +- include/media/media-device.h | 33 ++ include/media/media-devnode.h | 4 + include/media/media-request.h | 380 +++++++++++++ include/media/v4l2-ctrls.h | 76 ++- include/media/v4l2-device.h | 11 + include/media/v4l2-mem2mem.h | 4 + include/media/videobuf2-core.h | 53 +- include/media/videobuf2-v4l2.h | 20 +- include/uapi/linux/media.h | 12 + include/uapi/linux/videodev2.h | 14 +- 74 files changed, 3036 insertions(+), 368 deletions(-) create mode 100644 Documentation/media/uapi/mediactl/media-ioc-request-alloc.rst create mode 100644 Documentation/media/uapi/mediactl/media-request-ioc-queue.rst create mode 100644 Documentation/media/uapi/mediactl/media-request-ioc-reinit.rst create mode 100644 Documentation/media/uapi/mediactl/request-api.rst create mode 100644 drivers/media/media-request.c create mode 100644 include/media/media-request.h -- 2.17.0
