On 2/5/19 11:16 AM, Tim Harvey wrote:
On Sat, Feb 2, 2019 at 11:10 AM Steve Longerbeam<slongerbeam@xxxxxxxxx> wrote:
<snip>
The *real* way to fix this would be to allow programmable encodings in
ipu-ic.c. But unfortunately the encodings are hardcoded (grep for
ic_csc_rgb2ycbcr in ipu-ic.c).
Ok, I saw that you went ahead and worked on this (thanks!) and that
you have a bt.709.v2 branch... is that one ready for testing?
Yes. I tried to test it too, but there is some regression in captured
images, it could be something in ov5640.c (I'm testing with the SabreSD
and the OV5640), or something else recently added. The regression looks
like a stride problem, I'm hoping it wasn't the recently introduced
compose window changes to relax width alignment. Have you tested with
those commits and a prpenc/vf pipeline?
<snip>
Also can we connect the mem2mem driver to the unused VDIC input in the
media controller so that we can use the VDIC to de-interlace content
captured from non IMX sources (ie PCI or USB capture devices)?
Exactly! That's something I have been working on. But it's difficult to
connect mem2mem to the unused VDIC IDMAC input pad because as of now the
v4l2 mem2mem internal API's do not allow connecting to *existing*
processing entities, and there are also issues with how sub-devices are
to deal with mem2mem contexts.
I do have a WIP branch that creates a video output device that connects
to the VDIC IDMAC input pad, which doesn't have the above issues. The
only drawback with that is how gstreamer can make use of such an output
device.
ok, keep me posted. Is it the output-vdic or mem2mem.v4-mc branch?
The output-vdic branch. It's almost ready to go, there is only some
strange issue with low and medium motion-compensation modes (captured
images show "snow").
I also noticed you have a add-fim-to-prpencvf branch... are you
working on adding FIM to ipu_ic_prp/enc still? That would be nice to
have to deal with sync loss/regain in analog decoders going through
VDIC de-interlacing.
I've been trying to get this working, I ran into some locking issues
when enabling lock debug options. Still trying to find a solution.
<snip>
This one (480i60Hz YUV via BT656 sensor->mux->csi->ic_prp->ic_prpenc)
still baffles me a bit but I've also found that any bt656 capture that
isn't specifically 720x480 (NTSC) or 720x576 (PAL) fails because of
the resolution checks in ipu_csi_init_interface() resulting in
'Unsupported interlaced video mode'. I'm not sure if
ipu_csi_set_bt_interlaced_codes() can be modified to support other
resolutions?
Well, Bt.656 only defines standard definition NTSC and PAL.
That is true. Do you know of any other sensors that use higher
resolutions with BT656 SAV/EAV encoding?
Nope all I have for testing is the NTSC/PAL ADV7180 on the SabreAuto.
The BT656 mode does work well
for the progressive modes up to 1080p30 (1080p60 exceeds the IMX6
pixel clock and can't be used).
I could dig into the bt656 spec to try and understand the various
codes that get stuffed into the IPUx_CSI0_CCIR_CODE_1/2/3 registers I
suppose but I'm still not sure I want to push up a device-tree config
that describes the tda1997x to CSI connection as 8bit BT656 as I would
prefer to describe it as 16bit YUV instead (as I may be close to
getting that format working well if we can get the IC able to output
rec709).
<snip>
# imx6q-gw54xx tda19971 720p 16bit YUV IPU1_CSI0
MODE1:sensor->mux->csi->ic_prp->ic_prpenc
# set sensor output pad to sensor source format
v4l2-ctl -d /dev/v4l-subdev15 --set-dv-bt-timings=query
# sensor format
media-ctl --get-v4l2 '"tda19971 2-0048":0'
[fmt:UYVY8_1X16/1280x720 field:none colorspace:rec709]
# get framerate
v4l2-ctl --device /dev/v4l-subdev15 --get-dv-timings
DV timings:
Active width: 1280
Active height: 720
Total width: 1650
Total height: 750
Frame format: progressive
Polarities: +vsync +hsync
Pixelclock: 74250000 Hz (60.00 frames per second)
Horizontal frontporch: 110
Horizontal sync: 40
Horizontal backporch: 220
Vertical frontporch: 5
Vertical sync: 5
Vertical backporch: 20
Standards: CTA-861
CTA-861 VIC: 0
Flags: framerate can be reduced by 1/1.001, CE-video, has CTA-861 VIC
# reset all links
media-ctl --reset
# setup links
media-ctl -l "'tda19971 2-0048':0 -> 'ipu1_csi0_mux':1[1]"
media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
# configure pads
media-ctl -V "'tda19971 2-0048':0 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x360 ]"
media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x360 ]"
media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x360 ]"
media-ctl --get-v4l2 '"ipu1_ic_prpenc":1'
# capture device
media-ctl -e 'ipu1_ic_prpenc capture'
/dev/video0
v4l2-ctl --device /dev/video0 --get-fmt-video
Format Video Capture:
Width/Height : 640/360
Pixel Format : 'UYVY' (UYVY 4:2:2)
Field : None
Bytes per Line : 1280
Size Image : 460800
Colorspace : Rec. 709
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Limited Range
Flags :
# capture 1 frame
v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw --stream-count=1
[ 125.966980] ipu1_ic_prpenc: pipeline start failed with -32
Do you know what the failure is here?
You are /2 downscaling in the CSI, but did not set the compose window at
the input pad, e.g:
media-ctl --set-v4l2 '"ipu1_csi0":0[compose:(0,0)/640x360]' # 1/2 scale
I don't think this is the issue. Note that I 'was' able to get the
same pipeline with the div-by-2 downscale without compose working with
720p bt656 YUV.
Here is the 720p 16bit YUV bus again with the compose:
# 720p60Hz YUV 16bit YUV bus
# imx6q-gw54xx tda19971 2-0048 IPU1_CSI0
MODE1:sensor->mux->csi->ic_prp->ic_prpenc
v4l2-ctl -d /dev/v4l-subdev15 --set-dv-bt-timings=query
BT timings set
# sensor format
media-ctl --get-v4l2 '"tda19971 2-0048":0'
[fmt:UYVY8_1X16/1280x720 field:none colorspace:rec709]
# reset all links
media-ctl --reset
# setup links
media-ctl -l "'tda19971 2-0048':0 -> 'ipu1_csi0_mux':1[1]"
media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
# configure pads
media-ctl -V "'tda19971 2-0048':0 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0':0 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0':0 [compose:(0,0)/640x360]"
media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x360 ]"
media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x360 ]"
media-ctl --get-v4l2 "'ipu1_csi0':0"
[fmt:UYVY8_1X16/1280x720@1/30 field:none
colorspace:rec709 xfer:709 ycbcr:601 quantization:lim-range
crop.bounds:(0,0)/1280x720
crop:(0,0)/1280x720
compose.bounds:(0,0)/1280x720
compose:(0,0)/640x360]
media-ctl --get-v4l2 "'ipu1_csi0':1"
[fmt:UYVY8_1X16/640x360@1/30 field:none
colorspace:rec709 xfer:709 ycbcr:601 quantization:lim-range]
media-ctl --get-v4l2 "'ipu1_ic_prp':1"
[fmt:AYUV8_1X32/640x480@1/30 field:none
Ah, ipu1_ic_prp:1 pad is 640x480, it looks like you missed setting
ipu1_csi0:1 pad format, so that ipu1_ic_prp pads are still set to
default formats.
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
media-ctl --get-v4l2 "'ipu1_ic_prpenc':1"
[fmt:AYUV8_1X32/640x360@1001/30000 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
media-ctl -e 'ipu1_ic_prpenc capture'
/dev/video0
v4l2-ctl --device /dev/video0 --get-fmt-video
Format Video Capture:
Width/Height : 640/360
Pixel Format : 'UYVY' (UYVY 4:2:2)
Field : None
Bytes per Line : 1280
Size Image : 460800
Colorspace : SMPTE 170M
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Limited Range
Flags :
v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw --stream-count=1
[ 1279.017148] ipu1_ic_prpenc: pipeline start failed with -32
And can you explain why I can't colorspace convert the following CSI
capture case?:
# imx6q-gw54xx tda19971 720p60 16-bit YUV sensor->mux->csi
# set sensor output pad to sensor source format
v4l2-ctl -d /dev/v4l-subdev15 --set-dv-bt-timings=query
# sensor format
media-ctl --get-v4l2 '"tda19971 2-0048":0'
[fmt:UYVY8_1X16/1280x720 field:none colorspace:srgb]
# reset all links
media-ctl --reset
# setup links
media-ctl -l "'tda19971 2-0048':0 -> 'ipu1_csi0_mux':1[1]"
media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
# configure pads
media-ctl -V "'tda19971 2-0048':0 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/1280x720 colorspace:rec709 ycbcr:709]"
media-ctl --get-v4l2 '"ipu1_csi0":2'
[fmt:UYVY8_1X16/1280x720@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
^^^ still itu601
I have found in my testing when dealing with a BT656 RGB colorspace
input which I need to convert ot rec709 I have to set the 'input' pad
of the csi which doesn't make sense and if I do this with a 16-bit RGB
colorspace (above) it jacks up the fmt:
media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/1280x720 colorspace:rec709 ycbcr:709]"
media-ctl --get-v4l2 '"ipu1_csi0":2'
[fmt:UYVY8_2X8/1280x720@1/30 field:none
colorspace:rec709 xfer:709 ycbcr:709 quantization:lim-range]
^^^ changed fmt because AYUV32 at CSI input is invalid so it defaults
fmt to YUVY8_2X8
So there is something I'm still doing wrong to setup CSC.
Sorry I guess I don't understand your question here. Correct the CSI
does not accept AYUV32 at its input pad.
What I was asking here is what is the correct way to configure for
ITU601 to REC709 CSC at the CSI? I would expect to set the REC709
format on the output pad of the CSI but it seems to require me to set
it on the input pad.
Right, the driver only allows setting colorimetry at the sink pads, and
propagates them to the source pads. So you'll need to set colorimetry
params at the CSI sink pad.
Consider:
# imx6q-gw54xx tda19971 720p60 16-bit RGB sensor->mux->csi
# set sensor output pad to sensor source format
v4l2-ctl -d /dev/v4l-subdev15 --set-dv-bt-timings=query
# sensor format
media-ctl --get-v4l2 '"tda19971 2-0048":0'
[fmt:UYVY8_1X16/1280x720 field:none colorspace:srgb]
# reset all links
media-ctl --reset
# setup links
media-ctl -l "'tda19971 2-0048':0 -> 'ipu1_csi0_mux':1[1]"
media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
# configure pads
media-ctl -V "'tda19971 2-0048':0 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY8_1X16/1280x720 field:none]"
media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/1280x720]"
media-ctl --get-v4l2 '"ipu1_csi0":2'
[fmt:UYVY8_1X16/1280x720@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
^^^ ITU601, but I want REC709
media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/1280x720 colorspace:rec709 ycbcr:709]"
media-ctl --get-v4l2 '"ipu1_csi0":2'
[fmt:UYVY8_1X16/1280x720@1/30 field:none
colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
^^^ No invalid argument but its still ITU601 at CSI output... but what
I've found is if I set it at the CSI input the ycbcr changes but the
fmt gets jacked up (as expected)
media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/1280x720 colorspace:rec709 ycbcr:709]"
[fmt:UYVY8_2X8/1280x720@1/30 field:none
colorspace:rec709 xfer:709 ycbcr:709 quantization:lim-range]
^^^ note we are now rec709 but fmt is UYVY8_2X8 because the CSI
doesn't accept UYVY8_1X16 so it deafults to UYVY8_2X8.
Why doesn't the CSI accept UYVY8_1X16 at its sink pad?
This doesn't work?:
media-ctl -V "'ipu1_csi0':0 [fmt:UYVY8_1X16/1280x720 colorspace:rec709 ycbcr:709]"
Steve
