> Hi Ming Qian, > > Thank you for working on this. > > Some high-level comments: > > First of all, it looks like this series is based on a nxp-kernel. I noticed references > to e.g. V4L2_COLORSPACE_GENERIC_FILM which doesn't exist in the mainline > kernel. The patch series really should be based on the mainline kernel, or > (preferred) the linux-media kernel > (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.linu > xtv.org%2Fmedia_tree.git%2F&data=04%7C01%7Cming.qian%40nxp.com > %7C3b39b6df4e6746d5e5c308d94c19dd55%7C686ea1d3bc2b4c6fa92cd99c > 5c301635%7C0%7C1%7C637624496237171229%7CUnknown%7CTWFpbGZs > b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0 > %3D%7C2000&sdata=FlBIfE5%2BT3vsO7urAeeic%2FslWCQIR0VhuUuhx8Z > Xizk%3D&reserved=0, > master branch). Hi Hans, The patches are indeed based on linux-media kernel, and the V4L2_COLORSPACE_GENERIC_FILM is added in the patch "media: v4l: add some definition of v4l2 colorspace/xfer_func/ycbcr_encoding", which is the second patch of these series. > > On 20/07/2021 03:43, Ming Qian wrote: > > Hi all, > > > > This patch series adds support for > > the imx8q video encoder and decoder > > via the VPU block present in imx8q platforms. > > Currently, support for IMX8QXP and IMX8QM is included. > > > > It features decoding for the following formats: > > - H.264 > > - HEVC > > - MPEG4 > > - MPEG2 > > - MJPEG > > - VC1 > > - VP8 > > - AVS > > > > It features encoding for the following formats: > > - H.264 > > > > The driver creates a separate device node for the encoder and decoder. > > > > Changelog: > > > > v4: > > - redefine the memory-region in devicetree bindings documentation > > - use v4l2's mechanism to implement synchronize queuing ioctl > > - remove the unnecessary mutex ioctl_sync > > - don't notify source change event if the parameters are same as > > previously established > > - add flag V4L2_FMT_FLAG_DYN_RESOLUTION to decoder's capture format > > > > v3: > > - don't make vpu device node a simple-bus > > - trigger probing vpu core in the driver > > - remove unnecessary vpu core index property > > > > v2: > > - fix dt bindings build error > > - split driver patch into several parts to avoid exceeding bytes limit > > > > Compliance > > ========== > > # v4l2-compliance -d /dev/video0 > > v4l2-compliance SHA: not available > > , 64 bits, 64-bit time_t > > Always compile v4l2-compliance from the git repo > (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.linu > xtv.org%2Fv4l-utils.git%2F&data=04%7C01%7Cming.qian%40nxp.com%7 > C3b39b6df4e6746d5e5c308d94c19dd55%7C686ea1d3bc2b4c6fa92cd99c5c3 > 01635%7C0%7C1%7C637624496237171229%7CUnknown%7CTWFpbGZsb3d > 8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3 > D%7C2000&sdata=%2FWh5ObGzfWN%2B2z4J9qhuM7bpA%2Fd6iaZbJw > QhMI%2FdzFI%3D&reserved=0). > Otherwise I cannot tell whether it is a recent version or if it is old. Since there is > no SHA I'm going with old. > OK, I'll use the latest v4l2-compliance to test it again. > > > > Compliance test for vpu B0 device /dev/video0: > > > > Driver Info: > > Driver name : vpu B0 > > Card type : imx vpu decoder > > Bus info : platform: imx8q-vpu > > Driver version : 5.10.35 > > Capabilities : 0x84204000 > > Video Memory-to-Memory Multiplanar > > Streaming > > Extended Pix Format > > Device Capabilities > > Device Caps : 0x04204000 > > Video Memory-to-Memory Multiplanar > > Streaming > > Extended Pix Format > > Hmm, v4l2-compliance should have detected a stateful decoder here. OK, I'll use the latest v4l2-compliance to test it again. > > > > > Required ioctls: > > test VIDIOC_QUERYCAP: OK > > > > Allow for multiple opens: > > test second /dev/video0 open: OK > > test VIDIOC_QUERYCAP: OK > > test VIDIOC_G/S_PRIORITY: OK > > test for unlimited opens: OK > > > > test invalid ioctls: OK > > Debug ioctls: > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > > test VIDIOC_LOG_STATUS: OK (Not Supported) > > > > Input ioctls: > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > > test VIDIOC_ENUMAUDIO: OK (Not Supported) > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > > test VIDIOC_G/S_AUDIO: OK (Not Supported) > > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > > > Output ioctls: > > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > > > Input/Output configuration ioctls: > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > > test VIDIOC_G/S_EDID: OK (Not Supported) > > > > Control ioctls: > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > > test VIDIOC_QUERYCTRL: OK > > test VIDIOC_G/S_CTRL: OK > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > > Standard Controls: 3 Private Controls: 2 > > > > Format ioctls: > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > > test VIDIOC_G/S_PARM: OK (Not Supported) > > test VIDIOC_G_FBUF: OK (Not Supported) > > test VIDIOC_G_FMT: OK > > test VIDIOC_TRY_FMT: OK > > test VIDIOC_S_FMT: OK > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > > test Cropping: OK > > test Composing: OK > > test Scaling: OK > > > > Codec ioctls: > > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > > test VIDIOC_(TRY_)DECODER_CMD: OK > > > > Buffer ioctls: > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > > test VIDIOC_EXPBUF: OK > > test Requests: OK (Not Supported) > > > > Total for vpu b0 device /dev/video0: 45, Succeeded: 45, Failed: 0, > > Warnings: 0 > > > > # v4l2-compliance -d /dev/video1 > > v4l2-compliance SHA: not available > > , 64 bits, 64-bit time_t > > > > Compliance test for imx vpu encoder device /dev/video1: > > > > Driver Info: > > Driver name : imx vpu encoder > > Card type : imx vpu encoder > > Bus info : platform: imx8q-vpu > > Driver version : 5.10.35 > > Capabilities : 0x84204000 > > Video Memory-to-Memory Multiplanar > > Streaming > > Extended Pix Format > > Device Capabilities > > Device Caps : 0x04204000 > > Video Memory-to-Memory Multiplanar > > Streaming > > Extended Pix Format > > Detected Stateful Encoder > > Here it properly detects a stateful encoder. > > Regards, > > Hans > OK, I'll use the latest v4l2-compliance to test it again. > > > > Required ioctls: > > test VIDIOC_QUERYCAP: OK > > > > Allow for multiple opens: > > test second /dev/video1 open: OK > > test VIDIOC_QUERYCAP: OK > > test VIDIOC_G/S_PRIORITY: OK > > test for unlimited opens: OK > > > > test invalid ioctls: OK > > Debug ioctls: > > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > > test VIDIOC_LOG_STATUS: OK (Not Supported) > > > > Input ioctls: > > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > > test VIDIOC_ENUMAUDIO: OK (Not Supported) > > test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) > > test VIDIOC_G/S_AUDIO: OK (Not Supported) > > Inputs: 0 Audio Inputs: 0 Tuners: 0 > > > > Output ioctls: > > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > > > Input/Output configuration ioctls: > > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) > > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > > test VIDIOC_G/S_EDID: OK (Not Supported) > > > > Control ioctls: > > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK > > test VIDIOC_QUERYCTRL: OK > > test VIDIOC_G/S_CTRL: OK > > test VIDIOC_G/S/TRY_EXT_CTRLS: OK > > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK > > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > > Standard Controls: 20 Private Controls: 0 > > > > Format ioctls: > > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > > test VIDIOC_G/S_PARM: OK > > test VIDIOC_G_FBUF: OK (Not Supported) > > test VIDIOC_G_FMT: OK > > test VIDIOC_TRY_FMT: OK > > test VIDIOC_S_FMT: OK > > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > > test Cropping: OK > > test Composing: OK (Not Supported) > > test Scaling: OK (Not Supported) > > > > Codec ioctls: > > test VIDIOC_(TRY_)ENCODER_CMD: OK > > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > > > Buffer ioctls: > > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > > test VIDIOC_EXPBUF: OK > > test Requests: OK (Not Supported) > > > > Total for imx vpu encoder device /dev/video1: 45, Succeeded: 45, > > Failed: 0, Warnings: 0 > > > > Ming Qian (13): > > dt-bindings: media: imx8q: add imx video codec bindings > > media: v4l: add some definition of v4l2 > > colorspace/xfer_func/ycbcr_encoding > > media: imx: imx8q: add imx8q vpu device driver > > media: imx: imx8q: add vpu core driver > > media: imx: imx8q: implement vpu core communication based on > mailbox > > media: imx: imx8q: add vpu v4l2 m2m support > > media: imx: imx8q: add v4l2 m2m vpu encoder stateful driver > > media: imx: imx8q: add v4l2 m2m vpu decoder stateful driver > > media: imx: imx8q: implement windsor encoder rpc interface > > media: imx: imx8q: implement malone decoder rpc interface > > ARM64: dts: freescale: imx8q: add imx vpu codec entries > > firmware: imx: scu-pd: imx8q: add vpu mu resources > > MAINTAINERS: add NXP IMX8Q VPU CODEC V4L2 driver entry > > > > .../bindings/media/nxp,imx8q-vpu.yaml | 178 ++ > > MAINTAINERS | 10 + > > .../arm64/boot/dts/freescale/imx8-ss-vpu.dtsi | 72 + > > arch/arm64/boot/dts/freescale/imx8qxp-mek.dts | 17 + > > arch/arm64/boot/dts/freescale/imx8qxp.dtsi | 26 + > > drivers/firmware/imx/scu-pd.c | 4 + > > drivers/media/platform/Kconfig | 2 + > > drivers/media/platform/Makefile | 2 + > > drivers/media/platform/imx/Kconfig | 19 + > > drivers/media/platform/imx/Makefile | 1 + > > drivers/media/platform/imx/vpu-8q/Makefile | 23 + > > drivers/media/platform/imx/vpu-8q/vdec.c | 1817 > +++++++++++++++++ > > drivers/media/platform/imx/vpu-8q/venc.c | 1395 +++++++++++++ > > drivers/media/platform/imx/vpu-8q/vpu.h | 343 ++++ > > drivers/media/platform/imx/vpu-8q/vpu_cmds.c | 446 ++++ > > drivers/media/platform/imx/vpu-8q/vpu_cmds.h | 34 + > > drivers/media/platform/imx/vpu-8q/vpu_codec.h | 77 + > > drivers/media/platform/imx/vpu-8q/vpu_color.c | 201 ++ > > drivers/media/platform/imx/vpu-8q/vpu_core.c | 919 +++++++++ > > drivers/media/platform/imx/vpu-8q/vpu_core.h | 25 + > > drivers/media/platform/imx/vpu-8q/vpu_dbg.c | 505 +++++ > > drivers/media/platform/imx/vpu-8q/vpu_defs.h | 194 ++ > > .../media/platform/imx/vpu-8q/vpu_dev_imx8q.c | 82 + > > drivers/media/platform/imx/vpu-8q/vpu_drv.c | 225 ++ > > .../media/platform/imx/vpu-8q/vpu_helpers.c | 405 ++++ > > .../media/platform/imx/vpu-8q/vpu_helpers.h | 80 + > > drivers/media/platform/imx/vpu-8q/vpu_imx8q.c | 227 ++ > > drivers/media/platform/imx/vpu-8q/vpu_imx8q.h | 125 ++ > > drivers/media/platform/imx/vpu-8q/vpu_log.h | 53 + > > .../media/platform/imx/vpu-8q/vpu_malone.c | 1744 > ++++++++++++++++ > > .../media/platform/imx/vpu-8q/vpu_malone.h | 51 + > > drivers/media/platform/imx/vpu-8q/vpu_mbox.c | 135 ++ > > drivers/media/platform/imx/vpu-8q/vpu_mbox.h | 25 + > > drivers/media/platform/imx/vpu-8q/vpu_msgs.c | 420 ++++ > > drivers/media/platform/imx/vpu-8q/vpu_msgs.h | 23 + > > drivers/media/platform/imx/vpu-8q/vpu_rpc.c | 266 +++ > > drivers/media/platform/imx/vpu-8q/vpu_rpc.h | 472 +++++ > > drivers/media/platform/imx/vpu-8q/vpu_v4l2.c | 662 ++++++ > > drivers/media/platform/imx/vpu-8q/vpu_v4l2.h | 53 + > > .../media/platform/imx/vpu-8q/vpu_windsor.c | 1253 ++++++++++++ > > .../media/platform/imx/vpu-8q/vpu_windsor.h | 48 + > > include/linux/imx_vpu.h | 19 + > > include/uapi/linux/imx_vpu.h | 120 ++ > > include/uapi/linux/videodev2.h | 30 + > > 44 files changed, 12828 insertions(+) create mode 100644 > > Documentation/devicetree/bindings/media/nxp,imx8q-vpu.yaml > > create mode 100644 arch/arm64/boot/dts/freescale/imx8-ss-vpu.dtsi > > create mode 100644 drivers/media/platform/imx/Kconfig > > create mode 100644 drivers/media/platform/imx/Makefile > > create mode 100644 drivers/media/platform/imx/vpu-8q/Makefile > > create mode 100644 drivers/media/platform/imx/vpu-8q/vdec.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/venc.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_cmds.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_cmds.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_codec.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_color.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_core.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_core.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_dbg.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_defs.h > > create mode 100644 > drivers/media/platform/imx/vpu-8q/vpu_dev_imx8q.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_drv.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_helpers.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_helpers.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_imx8q.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_imx8q.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_log.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_malone.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_malone.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_mbox.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_mbox.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_msgs.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_msgs.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_rpc.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_rpc.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_v4l2.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_v4l2.h > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_windsor.c > > create mode 100644 drivers/media/platform/imx/vpu-8q/vpu_windsor.h > > create mode 100644 include/linux/imx_vpu.h create mode 100644 > > include/uapi/linux/imx_vpu.h > >
