On 7/13/19 7:03 AM, Hans Verkuil wrote: > On 7/12/19 5:38 PM, André Almeida wrote: >> Hello, >> >> On 7/10/19 4:33 AM, Hans Verkuil wrote: >>> On 7/10/19 12:19 AM, Helen Koike wrote: >>>> Hi André, >>>> >>>> Thanks for the patches. >>>> >>>> On 7/2/19 12:47 PM, André Almeida wrote: >>>>> Hello, >>>>> >>>>> This patch adds a V4L2 output device on vimc, that comply with V4L2 API >>>>> for video output. If there is an output device and a capture device at the >>>>> same pipeline, one can get a video loopback pipeline feeding frames at >>>>> the output and then seeing them at the capture. It's possible to insert >>>>> vimc submodules at the pipeline to modify the image (e.g. a scaler). >>>>> >>>>> If one starts a streaming at the capture, with the output off, the >>>>> capture will display a noisy frame. If one starts a streaming at the >>>>> output with the capture off, the output will just consume the buffers, >>>>> without sending them to the pipeline. If both output and capture are >>>>> streaming, the loopback will happen. >>>> I understand why it is done like this in vivid, but I was wondering, if we >>>> have a pipeline like: >>>> output -> capture >>>> Shouldn't streaming from the capture just stalls if there is no frame >>>> available in the output (i.e. streaming in the output is off) ? But then I'm >>>> not sure what the framerate in the capture would mean. >>>> >>>> Hans, what do you think? >>> If you set up the pipeline like this: >>> >>> Video Output -> Scaler -> Video Capture >> >> If the capture will stall if there's no frame from the video output, how >> can I add support for this kind of pipeline at test-media? It would be >> required to send frames to the output device while running >> `v4l2-compliance` at the capture device to make testing possible. > > The compliance test doesn't support such devices at the moment. The implementation of the expected behavior can be found here: https://gitlab.collabora.com/tonyk/linux/tree/vimc/output-v2 > > I think a new option (or options) are needed to tell the compliance test > that the capture and output video devices together constitute an m2m device. I've reading the v4l-utils code base and I had a look at both m2m tests and capture/output tests, but I'm not sure how to implement this new option. How do you think it should be implemented? Should it resemble how v4l2-compliance tests vim2m? Something like this: v4l2-compliance -m platform:vim2m -z platform:vivid-002 -e vivid-002-vid-cap -s10 -P -a Thanks, André > > Regards, > > Hans > >> >> Thanks, >> André >> >>> Then this is a mem2mem device (except with two separate video devices) and >>> framerate doesn't apply anymore. And video capture will just stall if there >>> is no video output frame provided. >>> >>> It's how e.g. omap3isp works. >>> >>> Regards, >>> >>> Hans >>> >>>> Thanks, >>>> Helen >>>> >>>>> The patches 1 and 2 provide some ground to create the output >>>>> device. The patch 3 creates the device and modify how the vimc-streamer >>>>> was dealing with the s_stream callback on other vimc modules, to make >>>>> simpler implementing this callback at vimc-output. Patch 4 change the >>>>> behavior of the pipeline in order to be closer to a real life hardware. >>>>> Patches 5-7 updates the default pipeline and the documentation to >>>>> include the new output device. >>>>> >>>>> This is the result of v4l2-compliance after this patch series: >>>>> $ v4l2-compliance -m0 -s50 >>>>> Grand Total for vimc device /dev/media0: 476, Succeeded: 476, Failed: 0, >>>>> Warnings: 0 >>>>> >>>>> A git tree up to date with media-master and with this changes can be found >>>>> at: https://gitlab.collabora.com/tonyk/linux/tree/vimc/output >>>>> >>>>> In order to test it, one can follow these instructions: >>>>> >>>>> 1 - Configure the pipeline (requires v4l-utils): >>>>> >>>>> $ media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]' >>>>> $ media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]' >>>>> $ media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]' >>>>> $ media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]' >>>>> $ v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440 >>>>> $ v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81 >>>>> $ v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81 >>>>> $ v4l2-ctl -z platform:vimc -e "RGB/YUV Input" -v width=640,height=480 >>>>> >>>>> 2 - Use a userspace application: >>>>> 2.a gst-launch (requires gstreamer and gst-plugins-good): >>>>> >>>>> Feed frames into the output and grab from the capture (rescaled for >>>>> convenience): >>>>> >>>>> $ gst-launch-1.0 videotestsrc pattern=ball ! \ >>>>> video/x-raw,width=640,height=480,format=RGB \ >>>>> ! v4l2sink device=/dev/video2 v4l2src device=/dev/video3 ! \ >>>>> video/x-raw,width=1920,height=1440,format=RGB ! videoscale ! \ >>>>> video/x-raw,width=640,height=480 ! videoconvert ! ximagesink >>>>> >>>>> 2.b qv4l2 (requires v4l-utils): >>>>> >>>>> Open the output device: >>>>> >>>>> $ qv4l2 -d2 >>>>> >>>>> Open the capture device: >>>>> >>>>> $ qv4l2 -d3 >>>>> >>>>> Start the streaming at both, at any order. You can change the frame >>>>> content at "Test Pattern Generator" -> "Test Pattern" on the output. >>>>> >>>>> Thanks, >>>>> André >>>>> >>>>> André Almeida (7): >>>>> media: vimc: Create video module >>>>> media: vimc: video: Add write file operation >>>>> media: vimc: Create a V4L2 output device >>>>> media: vimc: Send null buffer through the pipeline >>>>> media: vimc: core: Add output device on the pipeline >>>>> media: vimc.dot: Update default topology diagram >>>>> media: vimc.rst: Add output device >>>>> >>>>> Documentation/media/v4l-drivers/vimc.dot | 4 +- >>>>> Documentation/media/v4l-drivers/vimc.rst | 12 +- >>>>> drivers/media/platform/vimc/Makefile | 4 +- >>>>> drivers/media/platform/vimc/vimc-capture.c | 356 +++---------------- >>>>> drivers/media/platform/vimc/vimc-common.h | 5 +- >>>>> drivers/media/platform/vimc/vimc-core.c | 7 +- >>>>> drivers/media/platform/vimc/vimc-debayer.c | 14 +- >>>>> drivers/media/platform/vimc/vimc-output.c | 362 ++++++++++++++++++++ >>>>> drivers/media/platform/vimc/vimc-scaler.c | 13 +- >>>>> drivers/media/platform/vimc/vimc-sensor.c | 10 +- >>>>> drivers/media/platform/vimc/vimc-streamer.c | 24 +- >>>>> drivers/media/platform/vimc/vimc-video.c | 273 +++++++++++++++ >>>>> drivers/media/platform/vimc/vimc-video.h | 130 +++++++ >>>>> 13 files changed, 849 insertions(+), 365 deletions(-) >>>>> create mode 100644 drivers/media/platform/vimc/vimc-output.c >>>>> create mode 100644 drivers/media/platform/vimc/vimc-video.c >>>>> create mode 100644 drivers/media/platform/vimc/vimc-video.h >>>>> >
