Re: imx8mp mipi csi camera overlay: Unable to retrieve endpoint for port@1

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Jul 7, 2023 at 6:11 PM Adam Ford <aford173@xxxxxxxxx> wrote:
>
> On Fri, Jul 7, 2023 at 7:28 PM Tim Harvey <tharvey@xxxxxxxxxxxxx> wrote:
> >
> > Greetings,
> >
> > I have an imx219 camera connected to an imx8mp-venice-gw74xx via the
> > following details:
> > - camera is a RaspberryPi Camera v2 -
> > https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
> > - has its own on-board 24MHz osc so no clock is required from the baseboard
> > - pin 11 on the camera enables 1.8V and 2.8V LDO which is connected to
> > IMX8MP GPIO1_IO1 so we use that as a gpio regulator
> > - MIPI_CSI1 lanes 0 and 1 are used
> >
> > I'm using Linux 6.4 with Laurent's pending patch to add the MIPI CSI
> > DT nodes to imx8mp.dtsi [1] as follows:
> >
> > #include <dt-bindings/gpio/gpio.h>
> > #include "imx8mp-pinfunc.h"
> >
> > /dts-v1/;
> > /plugin/;
> >
> > &{/} {
> >         compatible = "gw,imx8mp-gw74xx", "fsl,imx8mp";
> >
> >         reg_cam: regulator-cam {
> >                 pinctrl-names = "default";
> >                 pinctrl-0 = <&pinctrl_reg_cam>;
> >                 compatible = "regulator-fixed";
> >                 regulator-name = "reg_cam";
> >                 gpio = <&gpio1 1 GPIO_ACTIVE_HIGH>;
> >                 enable-active-high;
> >                 regulator-min-microvolt = <1800000>;
> >                 regulator-max-microvolt = <1800000>;
> >         };
> >
> >         cam24m: cam24m {
> >                 compatible = "fixed-clock";
> >                 #clock-cells = <0>;
> >                 clock-frequency = <24000000>;
> >                 clock-output-names = "cam24m";
> >         };
> > };
> >
> > &i2c3 {
> >         #address-cells = <1>;
> >         #size-cells = <0>;
> >
> >         imx219: sensor@10 {
> >                 compatible = "sony,imx219";
> >                 reg = <0x10>;
> >                 clocks = <&cam24m>;
> >                 VDIG-supply = <&reg_cam>;
> >
> >                 port {
> >                         /* MIPI CSI-2 bus endpoint */
> >                         imx219_to_mipi_csi2: endpoint {
> >                                 remote-endpoint = <&mipi_csi_0_in>;
> >                                 clock-lanes = <0>;
> >                                 data-lanes = <1 2>;
> >                                 link-frequencies = /bits/ 64 <456000000>;
> >                         };
> >                 };
> >         };
> > };
> >
> > &mipi_csi_0 {
> >         status = "okay";
> >
> >         ports {
> >                 port@0 {
> >                         mipi_csi_0_in: endpoint {
> >                                 remote-endpoint = <&imx219_to_mipi_csi2>;
> >                                 data-lanes = <1 2>;
> >                         };
> >                 };
> >         };
> > };
> >
> > &iomuxc {
> >         pinctrl_reg_cam: regcamgrp {
> >                 fsl,pins = <
> >                         MX8MP_IOMUXC_GPIO1_IO01__GPIO1_IO01     0x41
> >                 >;
> >         };
> > };
> >
> > imx-mipi-csis fails to probe due to a missing port1 endpoint and I'm
> > not clear what to do with that:
> > imx-mipi-csis 32e40000.csi: Unable to retrieve endpoint for port@1
> > imx-mipi-csis: probe of 32e40000.csi failed with error -2
> >
> > Any suggestions?
>
> I think the port needs to point to an ISI node or a ISP node.
> Linux-next shows port@1 pointing to  isi_in_0.  In the patch you're
> referencing, it appears that node@1 is missing.
>
> Check out:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/arch/arm64/boot/dts/freescale/imx8mp.dtsi?h=next-20230707&id=9da15c4c850dd53309e07d5611f33655f8f8c05d
>
> See if that helps.
>
> adam
>

Hi Adam,

Thanks for the info. I didn't realize those patches were already in
linux-next and I missed the fact that the ISI was related to the CSI
(so I was missing the ISI patch and enabling that driver).

I'm now using 6.5.0-rc1 which has the patches that add CSI and ISI
nodes to the imx8mp.dtsi with the following dt fragment:
#include <dt-bindings/gpio/gpio.h>

#include "imx8mp-pinfunc.h"

/dts-v1/;
/plugin/;

&{/} {
        compatible = "gw,imx8mp-gw74xx", "fsl,imx8mp";

        reg_cam: regulator-cam {
                pinctrl-names = "default";
                pinctrl-0 = <&pinctrl_reg_cam>;
                compatible = "regulator-fixed";
                regulator-name = "reg_cam";
                gpio = <&gpio1 4 GPIO_ACTIVE_HIGH>;
                enable-active-high;
                regulator-min-microvolt = <1800000>;
                regulator-max-microvolt = <1800000>;
        };

        cam24m: cam24m {
                compatible = "fixed-clock";
                #clock-cells = <0>;
                clock-frequency = <24000000>;
                clock-output-names = "cam24m";
        };
};

&i2c4 {
        #address-cells = <1>;
        #size-cells = <0>;

        imx219: sensor@10 {
                compatible = "sony,imx219";
                reg = <0x10>;
                clocks = <&cam24m>;
                VDIG-supply = <&reg_cam>;

                port {
                        /* MIPI CSI-2 bus endpoint */
                        imx219_to_mipi_csi2: endpoint {
                                remote-endpoint = <&mipi_csi_0_in>;
                                clock-lanes = <0>;
                                data-lanes = <1 2>;
                                link-frequencies = /bits/ 64 <456000000>;
                        };
                };
        };
};

&isi_0 {
        status = "okay";
};

&mipi_csi_0 {
        status = "okay";

        ports {
                port@0 {
                        mipi_csi_0_in: endpoint {
                                remote-endpoint = <&imx219_to_mipi_csi2>;
                                data-lanes = <1 2>;
                        };
                };
        };
};

&iomuxc {
        pinctrl_reg_cam: regcamgrp {
                fsl,pins = <
                        MX8MP_IOMUXC_GPIO1_IO04__GPIO1_IO04     0x41
                >;
        };
};

Now I'm getting:
# dmesg | grep imx219
[    1.764111] imx219 2-0010: supply VANA not found, using dummy regulator
[    1.770953] imx219 2-0010: supply VDDL not found, using dummy regulator
[    5.520396] imx219 2-0010: Consider updating driver imx219 to match
on endpoints
# dmesg | grep mipi
[    2.080468] imx-mipi-csis 32e40000.csi: lanes: 2, freq: 500000000
# cat /sys/bus/media/devices/media*/model
FSL Capture Media Device
hantro-vpu
hantro-vpu
# cat /sys/class/video4linux/video*/name
mxc_isi.0.capture
mxc_isi.1.capture
mxc_isi.m2m
nxp,imx8mm-vpu-g1-dec
nxp,imx8mq-vpu-g2-dec
# media-ctl /dev/media0 -p
Media controller API version 6.5.0

Media device information
------------------------
driver          mxc-isi
model           FSL Capture Media Device
serial
bus info        platform:32e00000.isi
hw revision     0x0
driver version  6.5.0

Device topology
- entity 1: crossbar (5 pads, 4 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
                <- "csis-32e40000.csi":1 [ENABLED,IMMUTABLE]
        pad1: Sink
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
        pad2: Sink
                <- "mxc_isi.output":0 [ENABLED,IMMUTABLE]
        pad3: Source
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
                -> "mxc_isi.0":0 [ENABLED,IMMUTABLE]
        pad4: Source
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
                -> "mxc_isi.1":0 [ENABLED,IMMUTABLE]

- entity 7: mxc_isi.0 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:jpeg
xfer:srgb ycbcr:601 quantization:full-range
                 compose.bounds:(0,0)/1920x1080
                 compose:(0,0)/1920x1080]
                <- "crossbar":3 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:YUV8_1X24/1920x1080 field:none colorspace:jpeg
xfer:srgb ycbcr:601 quantization:full-range
                 crop.bounds:(0,0)/1920x1080
                 crop:(0,0)/1920x1080]
                -> "mxc_isi.0.capture":0 [ENABLED,IMMUTABLE]

- entity 10: mxc_isi.0.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
        pad0: Sink
                <- "mxc_isi.0":1 [ENABLED,IMMUTABLE]

- entity 18: mxc_isi.1 (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
        pad0: Sink
                [fmt:UYVY8_1X16/1920x1080 field:none colorspace:jpeg
xfer:srgb ycbcr:601 quantization:full-range
                 compose.bounds:(0,0)/1920x1080
                 compose:(0,0)/1920x1080]
                <- "crossbar":4 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:YUV8_1X24/1920x1080 field:none colorspace:jpeg
xfer:srgb ycbcr:601 quantization:full-range
                 crop.bounds:(0,0)/1920x1080
                 crop:(0,0)/1920x1080]
                -> "mxc_isi.1.capture":0 [ENABLED,IMMUTABLE]

- entity 21: mxc_isi.1.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
        pad0: Sink
                <- "mxc_isi.1":1 [ENABLED,IMMUTABLE]

- entity 29: mxc_isi.output (1 pad, 1 link)
             type Node subtype V4L flags 0
        pad0: Source
                -> "crossbar":2 [ENABLED,IMMUTABLE]

- entity 36: csis-32e40000.csi (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev3
        pad0: Sink
                [fmt:UYVY8_1X16/640x480 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
                <- "imx219 3-0010":0 []
        pad1: Source
                [fmt:UYVY8_1X16/640x480 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
                -> "crossbar":0 [ENABLED,IMMUTABLE]

- entity 41: imx219 3-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev4
        pad0: Source
                [fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range
                 crop.bounds:(8,8)/3280x2464
                 crop:(8,8)/3280x2464]
                -> "csis-32e40000.csi":0 []

# enable imx219 to csi link
media-ctl --device /dev/media0 --links "'imx219
3-0010':0->'csis-32e40000.csi':0[1]"

# v4l2-ctl --device /dev/video0 --all
Driver Info:
        Driver name      : mxc-isi
        Card type        : mxc-isi-cap
        Bus info         : platform:32e00000.isi
        Driver version   : 6.5.0
        Capabilities     : 0xa4201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : mxc-isi
        Model            : FSL Capture Media Device
        Serial           :
        Bus info         : platform:32e00000.isi
        Media version    : 6.5.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.5.0
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x0000000a (10)
        Name             : mxc_isi.0.capture
        Function         : V4L2 I/O
        Pad 0x0100000b   : 0: Sink
          Link 0x0200000e: from remote pad 0x1000009 of entity
'mxc_isi.0' (Video Pixel Formatter): Data, Enabled, Immutable
Priority: 2
Video input : 0 (mxc_isi.0.capture: ok)
Format Video Capture Multiplanar:
        Width/Height      : 640/480
        Pixel Format      : 'RGGB' (8-bit Bayer RGRG/GBGB)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : sRGB
        Transfer Function : sRGB
        YCbCr/HSV Encoding: ITU-R 601
        Quantization      : Limited Range
        Plane 0           :
           Bytes per Line : 640
           Size Image     : 307200

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                alpha_component 0x00980929 (int)    : min=0 max=255
step=1 default=0 value=0

# try to capture a frame
v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw --stream-count=1
                VIDIOC_STREAMON returned -1 (Broken pipe)
^^^ fails... not sure why

# try to use gstreamer
gst-launch-1.0 v4l2src device=/dev/video0 !
video/x-bayer,format=rggb,width=640,height=480,framerate=10/1 !
fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
to allocate required memory.
Additional debug info:
../sys/v4l2/gstv4l2src.c(759): gst_v4l2src_decide_allocation ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Internal data stream error.
Execution ended after 0:00:00.005517125
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Setting pipeline to NULL ...
Freeing pipeline ...
^^^ not sure what the memory allocation failure is about. It was
complaining about cma memory until I added cma=256M to the cmdline

I'm now at the point where I likely need help from the video4linux and
linux-media (cc'd) gurus.

best regards,

Tim



>
> >
> > Best regards,
> >
> > Tim
> > [1] https://patchwork.linuxtv.org/project/linux-media/patch/20230418153104.21337-2-laurent.pinchart@xxxxxxxxxxxxxxxx/




[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]

  Powered by Linux