This series introduces the decoding infrastructure that will be used to add support for other codecs such as VP8, VP9 and H.264. There are a number of v4l2-compliance issues, which I will be addressing shortly. For those wanting to take a peep, I've pasted the v4l2-compliance issues here: http://ix.io/1AfJ. Minor issues aside, "release early" they say, so here it is! About the JPEG bounce buffer, we can probably get rid of it, but for now, it's fine to keep it as it won't affect video decoding. About the media controller topology change, which is no doubt the nastiest change in this series, it's important to mention that we need multiple video interface nodes: one video node for encoding, and one video node for decoding. Since the VPU is a single engine, it needs a single mem2mem device to serialize the codec jobs thru it, taking advantage of the whole mem2mem infrastructure. This works perfectly well, but requires a somewhat involved topology. For the encoder the graph goes like this: ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-enc-source │ │ /dev/video0 │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-enc-proc │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-enc-sink │ │ /dev/video0 │ └────────────────────────────────┘ For the decoder the graph goes like this: ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-dec-source │ │ /dev/video1 │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-dec-proc │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-dec-sink │ │ /dev/video1 │ └────────────────────────────────┘ Both are tied to the same media device, i.e. /dev/media0. Does this comply with the media controller specification? The pixel format helpers deserve a discussion of its own. Note that these helpers will be useful for probably most V4L drivers. See for instance, this recent bug caused by bad math in vivid's sizeimage calculations: https://patchwork.kernel.org/patch/10796201/ Finally, I have to thank Jonas Karlman, who did the initial MPEG-2 decoding work and also for getting mpv+ffmpeg to work using the Request API. This driver can be tested using mpv+ffmpeg for the video decoding side, and the Panfrost mesa driver for rendering. I should be posting instructions to set all of this up, and also will be submitting the support for H264, VP8 and VP9, hopefully very soon. Ezequiel Garcia (9): media: Introduce helpers to fill pixel format structs rockchip/vpu: Use pixel format helpers rockchip/vpu: Use v4l2_m2m_buf_copy_data rockchip/vpu: Cleanup macroblock alignment rockchip/vpu: Cleanup JPEG bounce buffer management rockchip/vpu: Open-code media controller register rockchip/vpu: Support the Request API rockchip/vpu: Add decoder boilerplate rockchip/vpu: Add support for non-standard controls Jonas Karlman (1): rockchip/vpu: Add support for MPEG-2 decoding drivers/media/v4l2-core/Makefile | 2 +- drivers/media/v4l2-core/v4l2-common.c | 71 +++ drivers/media/v4l2-core/v4l2-fourcc.c | 109 ++++ drivers/staging/media/rockchip/vpu/Makefile | 5 +- .../media/rockchip/vpu/rk3288_vpu_hw.c | 4 +- .../rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c | 4 +- .../media/rockchip/vpu/rk3399_vpu_hw.c | 61 +- .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c | 10 +- .../rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c | 263 +++++++++ .../staging/media/rockchip/vpu/rockchip_vpu.h | 115 +++- .../media/rockchip/vpu/rockchip_vpu_common.h | 10 + .../media/rockchip/vpu/rockchip_vpu_dec.c | 557 ++++++++++++++++++ .../media/rockchip/vpu/rockchip_vpu_drv.c | 426 ++++++++++++-- .../media/rockchip/vpu/rockchip_vpu_enc.c | 152 ++--- .../media/rockchip/vpu/rockchip_vpu_hw.h | 42 ++ .../media/rockchip/vpu/rockchip_vpu_jpeg.c | 25 + .../media/rockchip/vpu/rockchip_vpu_mpeg2.c | 61 ++ include/media/v4l2-common.h | 5 + include/media/v4l2-fourcc.h | 53 ++ 19 files changed, 1803 insertions(+), 172 deletions(-) create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c create mode 100644 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_dec.c create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c create mode 100644 include/media/v4l2-fourcc.h -- 2.20.1
