Hi Hans,
On Sun, Oct 20, 2019 at 07:26:20PM +0200, Hans Verkuil wrote:
> On 10/20/19 7:35 AM, Laurent Pinchart wrote:
> > Hi Adam,
> >
> > On Sat, Oct 19, 2019 at 11:14:03AM -0500, Adam Ford wrote:
> >> On Fri, Oct 18, 2019 at 4:17 PM Sakari Ailus <sakari.ailus@xxxxxx> wrote:
> >>> On Fri, Oct 18, 2019 at 04:10:23PM -0500, Adam Ford wrote:
> >>>> On Fri, Oct 18, 2019 at 4:00 PM Sakari Ailus <sakari.ailus@xxxxxx> wrote:
> >>>>> On Fri, Oct 18, 2019 at 02:38:57PM -0500, Adam Ford wrote:
> >>>>>> I have a DM3730 with a parallel mt9p031 sensor attached. I am trying
> >>>>>> to troubleshoot some issues with streaming video with G-streamer, but
> >>>>>> I I think the issue is in how the ISP driver reports the video info.
> >>>>>>
> >>>>>> I have the pipeline to grab from the resizer:
> >>>>>>
> >>>>>> media-ctl -v -V '"mt9p031 1-0048":0 [SGRBG8 1280x720
> >>>>>> (664,541)/1280x720], "OMAP3 ISP CCDC":2 [SGRBG8 1280x720], "OMAP3 ISP
> >>>>>> preview":1 [UYVY 1280x720], "OMAP3 ISP resizer":1 [UYVY 480x272]'
> >>>>>>
> >>>>>> This make /dev/video6 the output of the resizer and shows the format
> >>>>>> and resolution of the output of the resizer as:
> >>>>>>
> >>>>>> Setting up format UYVY8_1X16 480x272 on pad OMAP3 ISP resizer/1
> >>>>>> Format set: UYVY8_1X16 480x272
> >>>>>>
> >>>>>> I used 480x272 because it's the resolution of my LCD, and I was hoping
> >>>>>> the resizer would be able to scale this so the ARM would not need to
> >>>>>> do the work, and it appears to not have any issues with this
> >>>>>> resolution.
> >>>>>>
> >>>>>> However, if I query the video format, I don't get UYVY:
> >>>>>>
> >>>>>> # v4l2-ctl -d6 --list-formats-ext
> >>>>>> ioctl: VIDIOC_ENUM_FMT
> >>>>>> Type: Video Capture
> >>>>>>
> >>>>>> [0]: 'RGB3' (RGB3, emulated)
> >>>>>> [1]: 'BGR3' (BGR3, emulated)
> >>>>>> [2]: 'YU12' (YU12, emulated)
> >>>>>> [3]: 'YV12' (YV12, emulated)
> >>>>>>
> >>>>>> This becomes an issue when I attempt to stream video from my camera to
> >>>>>> anything, include fake sink:
> >>>>>>
> >>>>>> gst-launch-1.0 -v v4l2src device=/dev/video6 ! fakesink
> >>>>>> Tried to capture in RGB3, but device returned format UYVY
> >>>>>>
> >>>>>> So for some reason, when queried, it reports different values than
> >>>>>> UYVY, but when attempting to set the video capture to the listed
> >>>>>> formats, it returns an error.
> >>>>>>
> >>>>>> gst-launch-1.0 -v v4l2src device=/dev/video6 ! video/x-raw,
> >>>>>> format=UYVY ! fakesink
> >>>>>
> >>>>> I don't have any experience on v4l2src recently but I can comment on the
> >>>>> omap3isp driver.
> >>>>>
> >>>>> In general, the format the omap3isp may produce on a given video node
> >>>>> depends on the format of data which the block associated with the video
> >>>>> node is fed with.
> >>>>>
> >>>>> For instance, in case of the raw Bayer formats, the pixel order does not
> >>>>> change, and thus the pixel order remains all the way from the sensor to the
> >>>>> video node.
> >>>>
> >>>> From what I can tell, it looks like the output of the resizer is only
> >>>> capable of two formats,
> >>>> from ispresizer.c:
> >>>>
> >>>> /* resizer pixel formats */
> >>>> static const unsigned int resizer_formats[] = {
> >>>> MEDIA_BUS_FMT_UYVY8_1X16,
> >>>> MEDIA_BUS_FMT_YUYV8_1X16,
> >>>> };
> >>>>
> >>>> Also:
> >>>>
> >>>> * resizer_try_format - Handle try format by pad subdev method
> >>>> * @res : ISP resizer device
> >>>> * @cfg: V4L2 subdev pad configuration
> >>>> * @pad : pad num
> >>>> * @fmt : pointer to v4l2 format structure
> >>>> * @which : wanted subdev format
> >>>>
> >>>> switch (pad) {
> >>>> case RESZ_PAD_SINK:
> >>>> if (fmt->code != MEDIA_BUS_FMT_YUYV8_1X16 &&
> >>>> fmt->code != MEDIA_BUS_FMT_UYVY8_1X16)
> >>>> fmt->code = MEDIA_BUS_FMT_YUYV8_1X16;
> >>>>
> >>>> So it looks to me like if we're trying to do anything other than
> >>>> either of those,we set it to MEDIA_BUS_FMT_YUYV8_1X16
> >>>>
> >>>> Am I missing something?
> >>>
> >>> I guess for this particular video node there's no need for V4L2 extensions
> >>> to implement ENUM_FMT. Ideally the other nodes would support ENUM_FMT that
> >>> could provide meaningful information as well.
> >>
> >> I applied a variation of the patch in question, and I was able to both
> >> successfully use g-streamer-1.0 and I was able to see UYVY appear in
> >> the list. in fact, G-Streamer-1.0 was faster than FFPEG with less
> >> lag.
> >>
> >> ioctl: VIDIOC_ENUM_FMT
> >> Type: Video Capture
> >>
> >> [0]: 'UYVY' (UYVY 4:2:2)
> >> [1]: 'RGB3' (RGB3, emulated)
> >> [2]: 'BGR3' (BGR3, emulated)
> >> [3]: 'YU12' (YU12, emulated)
> >> [4]: 'YV12' (YV12, emulated)
> >>
> >> I don't know enough about VL2. It looks like all the V4L stuff is in
> >> the common isp files and not unique to the preview output, resizer
> >> output or the CCDC output. I don't have a CSI camera, so I cannot
> >> test anything.
> >>
> >> I checked all the video outputs, and they are all showing the same
> >> information. I realize the patch I found won't be accepted upstream
> >> as-is, but it would be nice to have some mechanism in place that can
> >> determine which output node is being used and somehow return the
> >> correct data for each node.
> >>
> >> I would like to do something to help improve this driver and/or make
> >> it more compatible with some of the V4L tools (like G-Streamer), so if
> >> someone has a recommendation on how we could move forward, I'm willing
> >> work on it.
> >
> > While I'm not totally opposed to implementing VIDIOC_ENUM_FMT for the
> > resizer video node, I'm not sure it would be the best solution to your
> > problem. Sure, it will fix an existing issue, but we already know that
> > it won't scale, as the other video nodes can't be supported the same
> > way.
> >
> > So far, our position was mostly that userspace should grow support for
> > MC-based devices where VIDIOC_ENUM_FMT isn't implemented. Maybe I'm
>
> Well, I disagree: implementing G/S/TRY_FMT without ENUM_FMT is out-of-spec.
> The v4l2-compliance utility would flunk any driver that tries that.
> Unfortunately, v4l2-compliance didn't exist (or was in its infancy) when
> omap3isp was written.
>
> I didn't even know that ENUM_FMT wasn't implemented for the omap3 ISP.
>
> It makes no sense either: the driver is smart enough to validate the
> pixelformat, but not smart enough to be able to enumerate the list of
> valid formats for the current pipeline configuration?
As noted earlier, there's a dependency to the source media bus format. That
needs to be specified in order to come up with the pixelformat that the
device can actually produce. This is not really a problem with driver
implementation but the lack of an API to get this done.
Also, this isn't specific to omap3isp either: any device where the capture
format depends on pipeline configuration has the same issue.
Since no better solutions have been suggested, I'd be in favour of
extending ENUM_FMT to cover MC-enabled drivers by using one of the reserved
fields for the source media bus code.
--
Regards,
Sakari Ailus
