On Mon, 29 Nov 2021 at 13:48, Adam Ford <aford173@xxxxxxxxx> wrote: > > On Tue, Nov 23, 2021 at 2:07 PM Nicolas Dufresne <nicolas@xxxxxxxxxxxx> wrote: > > > > Le lundi 22 novembre 2021 à 09:25 -0800, Tim Harvey a écrit : > > > On Sat, Nov 20, 2021 at 7:36 AM Adam Ford <aford173@xxxxxxxxx> wrote: > > > > > > > > On Fri, Nov 19, 2021 at 5:37 PM Adam Ford <aford173@xxxxxxxxx> wrote: > > > > > > > > > > On Fri, Nov 19, 2021 at 10:29 AM Nicolas Dufresne <nicolas@xxxxxxxxxxxx> wrote: > > > > > > > > > > > > Hi Adam, Tim, > > > > > > > > > > > > [...] > > > > > > > > > > Nicolas and Adam, > > > > > > > > > > > > > > > > > > > > For the H1 patches in this series: I've been able to test the IMX8MM > > > > > > > > > > H1 JPEG encode using GStreamer 1.18.5: > > > > > > > > > > $ gst-inspect-1.0 | grep -e "v4l2.*enc" > > > > > > > > > > video4linux2: v4l2jpegenc: V4L2 JPEG Encoder > > > > > > > > > > $ gst-launch-1.0 videotestsrc ! jpegenc ! rtpjpegpay ! udpsink > > > > > > > > > ^ v4l2jpegenc > > > > > > > > > > > > > > > > > > This is just a transcript error ? > > > > > > > > > > > > > > > > Nicolas, > > > > > > > > > > > > > > > > No! Thanks for catching my mistake. I was testing with software encode... ooops! > > > > > > > > > > > > > > > > 'gst-launch-1.0 videotestsrc ! v4l2jpegenc ! fakesink' actually hangs > > > > > > > > the board so likely a power-domain issue there? > > > > > > > > > > > > > > The v4l2-compliance tests fail on the h1 decoder with a hang, but I > > > > > > > think we're writing to registers which are not documented in the Mini > > > > > > > TRM. The Mini TRM doesn't explicitly show the JPEG encoding as a > > > > > > > feature, but some of the registers state JPEG, but because some of the > > > > > > > registers written for the H1 are not documented in the TRM. If those > > > > > > > registers are restricted or not in this SoC, I am concerned that it > > > > > > > might be related. I'll try to run some more tests this weekend to > > > > > > > check on the status of the power-domain stuff. > > > > > > > > > > > > To verify if the HW support JPEG encoding you can read SWREG63 bit 25. This is > > > > > > in the TRM, just not labelled properly. To mimic the decoding side, would be "HW > > > > > > synthesis config register X" with the bit labelled SW_ENC_JPEG_PROF (but > > > > > > PROF/profile is on or off). If your board hang while reading this, you likely > > > > > > didn't get the power bit right. > > > > > > > > > > > > IMX8 has an undocumented control block thing that we have been fighting with in > > > > > > imx8q, perhaps that's your issue. Few driver was proposed, we are still pending > > > > > > on NXP solution to be submitted (they asked us to wait, still waiting =)). > > > > > > > > > > Nicolas, > > > > > > > > > > Thanks for the suggestion to read offset FC. There was an attempt > > > > > made by Lucas Stach to develop a VPU blk-ctrl driver to coordinate the > > > > > power-domains with the GPC driver. Unfortunately, it does appear to > > > > > hang, so it might not be operating correctly. > > > > > > > > > > Lucas, > > > > > > > > > > Do you have any idea of stuff I can try to see if the power domain is > > > > > coming online correctly? > > > > > > > > > > [ 10.434727] imx-pgc imx-pgc-domain.6: request the vpumix domain to power up > > > > > [ 10.463647] imx-pgc imx-pgc-domain.6: request the vpumix ADB400 to power up > > > > > [ 10.517155] imx-pgc imx-pgc-domain.6: genpd vpumix success > > > > > [ 10.728927] vpu: set fuse bits to enable > > > > > [ 10.825500] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g1 GPC domain > > > > > [ 10.878986] imx-pgc imx-pgc-domain.7: request the vpu-g1 domain to power up > > > > > [ 10.932429] imx-pgc imx-pgc-domain.7: genpd vpu-g1 success > > > > > [ 10.971988] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g1 success > > > > > [ 11.004726] hantro-vpu 38300000.video-codec: registered > > > > > nxp,imx8mm-vpu-dec as /dev/video0 > > > > > [ 11.040760] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-g2 GPC domain > > > > > [ 11.066181] imx-pgc imx-pgc-domain.8: request the vpu-g2 domain to power up > > > > > [ 11.087887] imx-pgc imx-pgc-domain.8: genpd vpu-g2 success > > > > > [ 11.113808] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-g2 success > > > > > [ 11.139634] hantro-vpu 38310000.video-codec: registered > > > > > nxp,imx8mm-vpu-g2-dec as /dev/video1 > > > > > [ 11.156463] imx8m-blk-ctrl 38330000.blk-ctrl: power vpublk-h1 GPC domain > > > > > [ 11.170817] imx-pgc imx-pgc-domain.9: request the vpu-h1 domain to power up > > > > > [ 11.232990] imx-pgc imx-pgc-domain.9: genpd vpu-h1 success > > > > > [ 11.252546] imx8m-blk-ctrl 38330000.blk-ctrl: genpd vpublk-h1 success > > > > > [ 11.266152] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc > > > > > > > > > > <hang> > > > > > > > > > > adam > > > > > > > > > > > > > Nicolas, Tim, and Lucas, > > > > > > > > I think I have the hanging resolved in the power domains, and I'll be > > > > pushing the fix to the GPCv2. > > > > > > > > For the H1 Encoder, I added some debugging code to read the offset > > > > 0xfc and print some data based on the findings of that VPU-h1 offset. > > > > I basically check the various bits per the TRM to see if they are set > > > > and print some splat to indicate whether or not the function is > > > > supported. > > > > > > > > [ 8.861865] hantro-vpu 38320000.video-codec: Checking vpu->enc_base + 0xfc > > > > [ 8.870594] hantro-vpu 38320000.video-codec: Stabilization supported by HW > > > > [ 8.889341] hantro-vpu 38320000.video-codec: VP8 encoding supported by HW > > > > [ 8.899386] hantro-vpu 38320000.video-codec: H.264 encoding supported by HW > > > > [ 8.918171] hantro-vpu 38320000.video-codec: RGB to YUV conversion > > > > supported by HW > > > > [ 8.934067] hantro-vpu 38320000.video-codec: registered > > > > nxp,imx8mm-vpu-h1-enc as /dev/video2 > > > > > > > > Unfortunately, JPEG is not listed as supported. :-( > > > > > > Adam, > > > > > > Well not having JPEG encode support is unfortunate, and unexpected. Do > > > we not have hantro support yet for VP8/H264 encode? > > > > There is no mainline support yet. You can derive from RK3288 support using Google ChromeOS method (a v4l2 plugins that simulate in userspace a stateful encoder): > > > > - libv4l plugins / https://chromium.googlesource.com/chromiumos/third_party/libv4lplugins/+/refs/heads/master > > - Kernel Driver / https://chromium.googlesource.com/chromiumos/third_party/kernel/+/chromeos-4.4/drivers/media/platform/rockchip-vpu/ > > > > > > > > I haven't quite figured out how to build a modern mono-repo gstreamer > > > on the ubuntu 20.04 rootfs I'm using so I haven't been able to test > > > VPU encode/decode properly. I'll keep working on it when I'm back in > > > the office the following week. > > > > Did a quick test to make sure there isn't any ubuntu specific blockers, here's a > > dirty script that produce a minimal GStreamer, there was really nothing special > > compare to other meson projects. Note that I use --wrap-mode=nofallback to avoid > > letting GStreamer complete it's feature-set by downloading the planet. This > > already build quite a lot and could likely be made smaller by avoid plugins-good > > build-dep call, but then you need to check for v4l2odecs and video4linux devs > > (mostly gudev a glib udev binding). > > > > # Install ubuntu > > podman run -it --rm ubuntu:20.04 > > sed -i "s/# deb-src/deb-src/" /etc/apt/sources.list > > apt update > > apt build-dep gstreamer1.0-plugins-good > > apt install git python3-pip flex bison > > > > # Need a newer meson > > pip3 install --user meson > > export PATH=$PATH:~/.local/bin > > > > # Build GStreamer > > git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git > > cd gstreamer > > meson setup build --wrap-mode=nofallback > > ninja -C build > > > > # Run in-place > > ./gst-env.py > > gst-inspect-1.0 v4l2codecs > > gst-inspect 1.0 video4linux2 > > > Thanks for the suggestions. > > I downloaded what's in the master repo: > > [gst-main] root@localhost:~/gstreamer# gst-inspect-1.0 v4l2codecs > > ** (gst-plugin-scanner:7317): CRITICAL **: 10:29:51.847: can't find > gi.repository.Gst > Plugin Details: > Name v4l2codecs > Description V4L2 CODEC Accelerators plugin > Filename > /root/gstreamer/builddir/subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so > Version 1.19.3.1 > License LGPL > Source module gst-plugins-bad > Binary package GStreamer Bad Plug-ins git > Origin URL Unknown package origin > > v4l2slh264dec: V4L2 Stateless H.264 Video Decoder > v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder > v4l2slvp8alphadecodebin: VP8 Alpha Decoder > v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder > > 4 features: > +-- 4 elements > > [gst-main] root@localhost:~/gstreamer# gst-inspect-1.0 video4linux2 > Plugin Details: > Name video4linux2 > Description elements for Video 4 Linux > Filename > /root/gstreamer/builddir/subprojects/gst-plugins-good/sys/v4l2/libgstvideo4linux2.so > Version 1.19.3.1 > License LGPL > Source module gst-plugins-good > Binary package GStreamer Good Plug-ins git > Origin URL Unknown package origin > > v4l2deviceprovider: Video (video4linux2) Device Provider > v4l2jpegenc: V4L2 JPEG Encoder > v4l2radio: Radio (video4linux2) Tuner > v4l2sink: Video (video4linux2) Sink > v4l2src: Video (video4linux2) Source > > 5 features: > +-- 4 elements > +-- 1 device providers > > I still have the H1 encoder enabled, but I know JPEG isn't supported, > so I'm going to attempt to do some decoding and pipe to fakesink since > I don't have a functional display yet. > > gst-launch-1.0 -ev filesrc location=trailer_1080p_h264_mp3.avi ! > decodebin3 ! fakesink > > Redistribute latency... > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/v4l2slh264dec:v4l2slh264dec0.GstPad:src: > caps = video/x-raw, format=(string)NV12, width=(int)1920, > height=(int)1080, interlace-mode=(string)progressive, > multiview-mode=(string)mono, > multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, > pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1 > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0.GstGhostPad:video_0: > caps = video/x-raw, format=(string)NV12, width=(int)1920, > height=(int)1080, interlace-mode=(string)progressive, > multiview-mode=(string)mono, > multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, > pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1 > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0.GstGhostPad:video_0.GstProxyPad:proxypad6: > caps = video/x-raw, format=(string)NV12, width=(int)1920, > height=(int)1080, interlace-mode=(string)progressive, > multiview-mode=(string)mono, > multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, > pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1 > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/GstMultiQueue:multiqueue0: > min-interleave-time = 300000000 > Redistribute latency... > /GstPipeline:pipeline0/GstDecodebin3:decodebin3-0/v4l2slh264dec:v4l2slh264dec0.GstPad:sink: > caps = video/x-h264, variant=(string)itu, framerate=(fraction)25/1, > width=(int)1920, height=(int)1080, chroma-format=(string)4:2:0, > bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, > parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au, > profile=(string)high, level=(string)4, > codec_data=(buffer)01640028ffe1001a67640028acd940780227e584000003000400000300c83c60c65801000668ebe3cb22c0 > New clock: GstSystemClock > > And it appears to stream, because the counter increases. I haven't > checked the CPU utilization, but the fact that it shows v4l2slh264dec > is good. > > Is there a way to know if/how the decoder is using the proper VPU? I > assume if it wasn't using the proper one, it would fail, but was just > curious. > A few ways. You can check /proc/interrupts, which should have VPU activity. Or enable debug messages for the module, using the debug hantro parameter. V4L2 has debug messages that you can enable, see /sys/class/video4linux/video0/dev_debug. Instead of fakesink you can output to pngenc/jpegenc and check the output is visually correct. If at all possible, the proper way is to use Fluster, and report the score you get: https://github.com/fluendo/fluster It should be easy to use. > I think I'll redo the patch without the RFC and without the H1 encoder > unless anyone has any objections. I know I need to rebase on > linux-next anyway because the patches don't apply cleanly. Is there a > specific branch I should use? I don't know if this goes through > Shawn's IMX tree or the media tree (or a combination) > You should rebase on media's master branch: https://git.linuxtv.org/media_tree.git/log/ Thanks, Ezequiel
