Am Freitag, dem 21.05.2021 um 16:36 +0300 schrieb Laurent Pinchart:
> Hi Martin,
>
> On Fri, May 21, 2021 at 01:02:30PM +0200, Martin Kepplinger wrote:
> > Am Freitag, dem 21.05.2021 um 12:43 +0300 schrieb Laurent Pinchart:
> > > On Fri, May 21, 2021 at 11:25:20AM +0200, Martin Kepplinger
> > > wrote:
> > > > Am Donnerstag, dem 20.05.2021 um 15:37 +0300 schrieb Laurent
> > > > Pinchart:
> > > > > On Thu, May 20, 2021 at 12:54:27PM +0200, Martin Kepplinger
> > > > > wrote:
> > > > > > Am Mittwoch, dem 19.05.2021 um 18:46 +0300 schrieb Laurent
> > > > > > Pinchart:
> > > > > > > On Wed, May 19, 2021 at 05:21:11PM +0200, Martin
> > > > > > > Kepplinger wrote:
> > > > > > > > Am Mittwoch, dem 19.05.2021 um 04:14 +0300 schrieb
> > > > > > > > Laurent Pinchart:
> > > > > > > > > On Tue, May 18, 2021 at 04:39:00PM +0200, Martin
> > > > > > > > > Kepplinger wrote:
> > > > > > > > > > Am Sonntag, dem 16.05.2021 um 01:55 +0300 schrieb
> > > > > > > > > > Laurent Pinchart:
> > > > > > > > > > > On Tue, May 04, 2021 at 05:59:39PM +0200, Martin
> > > > > > > > > > > Kepplinger wrote:
> > > > >
> > > > > [snip]
> > > > >
> > > > > > I fixed mipi -> csi link. I had the DT port descriptions
> > > > > > for
> > > > > > mipi csi wrong.
> > > > >
> > > > > \o/
> > > > >
> > > > > > now, just because I think it makes sense, I do:
> > > > > >
> > > > > > media-ctl --set-v4l2 "'csi':0 [fmt:SGBRG10/640x480
> > > > > > colorspace:raw]"
> > > > > >
> > > > > > which now prints:
> > > > > >
> > > > > > Device topology
> > > > > > - entity 1: csi (2 pads, 2 links)
> > > > > > type V4L2 subdev subtype Unknown flags 0
> > > > > > device node name /dev/v4l-subdev0
> > > > > > pad0: Sink
> > > > > > [fmt:SGBRG10_1X10/640x480 field:none
> > > > > > colorspace:raw xfer:none ycbcr:601 quantization:full-range]
> > > > > > <- "imx8mq-mipi-csis.0":1
> > > > > > [ENABLED,IMMUTABLE]
> > > > > > pad1: Source
> > > > > > [fmt:SGBRG10_1X10/640x480 field:none
> > > > > > colorspace:raw xfer:none ycbcr:601 quantization:full-range]
> > > > > > -> "csi capture":0 [ENABLED,IMMUTABLE]
> > > > > >
> > > > > > - entity 4: csi capture (1 pad, 1 link)
> > > > > > type Node subtype V4L flags 0
> > > > > > device node name /dev/video1
> > > > > > pad0: Sink
> > > > > > <- "csi":1 [ENABLED,IMMUTABLE]
> > > > > >
> > > > > > - entity 10: imx8mq-mipi-csis.0 (2 pads, 2 links)
> > > > > > type V4L2 subdev subtype Unknown flags 0
> > > > > > device node name /dev/v4l-subdev1
> > > > > > pad0: Sink
> > > > > > <- "hi846 2-0020":0 []
> > > > > > pad1: Source
> > > > > > -> "csi":0 [ENABLED,IMMUTABLE]
> > > > >
> > > > > This subdev doesn't seem to report formats on its sink and
> > > > > source pads,
> > > > > which is weird. I've had a quick look at the .get_fmt() and
> > > > > .set_fmt()
> > > > > implementations in the code you've posted, and they're wrong.
> > > > > They
> > > > > shouldn't pass the calls to the source subdev with
> > > > > v4l2_subdev_call(),
> > > > > they should instead implement get and set format on this
> > > > > subdev. You can
> > > > > look at the imx7-mipi-csis driver to see how that's done.
> > > > > Once you'll
> > > > > have fixed this, you'll have to set the format on each pad
> > > > > with
> > > > > media-ctl to make sure formats through the pipeline match.
> > > > >
> > > > > The only location where you imx8mq-mipi-csis driver should
> > > > > use
> > > > > v4l2_subdev_call() is in .s_stream(), to propagate the
> > > > > operation to the
> > > > > source.
> > > > >
> > > > > By the way, I'd replace every occurence of "csis" with "csi2"
> > > > > in your
> > > > > driver. The name "csis" in the i.MX7 driver comes from the
> > > > > CSI-2 RX IP
> > > > > core that is named CSIS. That's not the case on the i.MX8QM.
> > > > >
> > > > > > - entity 15: hi846 2-0020 (1 pad, 1 link)
> > > > > > type V4L2 subdev subtype Sensor flags 0
> > > > > > device node name /dev/v4l-subdev2
> > > > > > pad0: Source
> > > > > > [fmt:SGBRG10_1X10/640x480 field:none
> > > > > > colorspace:raw]
> > > > > > -> "imx8mq-mipi-csis.0":0 []
> > > > >
> > > > > You need to enable this link, the following should do
> > > > >
> > > > > media-ctl -l "'hi846 2-0020':0 -> 'imx8mq-mipi-csis.0':0 [1]"
> > > >
> > > > ok makes sense, even though I basically just allow a set of
> > > > formats
> > > > without yet having to configure anything format-specific (I can
> > > > at
> > > > least use bits-per-pixel later, so it makes sense to have
> > > > them).
> > > > nevermind. I again append the current driver I use here.
> > > >
> > > > then I do:
> > > >
> > > > media-ctl --set-v4l2 "'csi':0 [fmt:SGBRG10/640x480
> > > > colorspace:raw]"
> > > > media-ctl --set-v4l2 "'imx8mq-mipi-csi2.0':0
> > > > [fmt:SGBRG10/640x480
> > > > colorspace:raw]"
> > > > media-ctl -l "'hi846 2-0020':0 -> 'imx8mq-mipi-csi2.0':0 [1]"
> > > >
> > > > which gets me:
> > > >
> > > > Device topology
> > > > - entity 1: csi (2 pads, 2 links)
> > > > type V4L2 subdev subtype Unknown flags 0
> > > > device node name /dev/v4l-subdev0
> > > > pad0: Sink
> > > > [fmt:SGBRG10_1X10/640x480 field:none
> > > > colorspace:raw xfer:none ycbcr:601 quantization:full-range]
> > > > <- "imx8mq-mipi-csi2.0":1 [ENABLED,IMMUTABLE]
> > > > pad1: Source
> > > > [fmt:SGBRG10_1X10/640x480 field:none
> > > > colorspace:raw xfer:none ycbcr:601 quantization:full-range]
> > > > -> "csi capture":0 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 4: csi capture (1 pad, 1 link)
> > > > type Node subtype V4L flags 0
> > > > device node name /dev/video0
> > > > pad0: Sink
> > > > <- "csi":1 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 10: imx8mq-mipi-csi2.0 (2 pads, 2 links)
> > > > type V4L2 subdev subtype Unknown flags 0
> > > > device node name /dev/v4l-subdev1
> > > > pad0: Sink
> > > > [fmt:SGBRG10_1X10/640x480]
> > > > <- "hi846 2-0020":0 [ENABLED]
> > > > pad1: Source
> > > > [fmt:SGBRG10_1X10/640x480]
> > > > -> "csi":0 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 15: hi846 2-0020 (1 pad, 1 link)
> > > > type V4L2 subdev subtype Sensor flags 0
> > > > device node name /dev/v4l-subdev2
> > > > pad0: Source
> > > > [fmt:SGBRG10_1X10/640x480 field:none
> > > > colorspace:raw]
> > > > -> "imx8mq-mipi-csi2.0":0 [ENABLED]
> > >
> > > This looks better.
> > >
> > > > but streaming still fails with:
> > > >
> > > > [ 352.255129] imx7-csi 30a90000.csi1_bridge: media bus code
> > > > not
> > > > compatible with the pixel format set on the video node: 1 != 0
> > >
> > > What is the capture command line ? Can you trace this (I assume
> > > the
> > > message is printed by capture_validate_fmt(), it's not present in
> > > mainline so I don't know what 1 and 0 correspond to, even though
> > > I
> > > suspect they would be IPUV3_COLORSPACE_* values) to see why it
> > > fails
> > > ?
> >
> > capture command:
> >
> > v4l2-ctl -d "/dev/v4l/by-path/platform-30a90000.csi1_bridge-video-
> > index0" --set-fmt-video=width=640,height=480 --stream-mmap --
> > stream-
> > to=test.raw --stream-count=1
> >
> > I'll have to continue after the weekend, but let's share some logs.
> > Yes, "1 != 0" is from capture_validate_fmt():
> >
> > priv->vdev.cc->cs != cc->cs
> >
> > When I print the format the imx_media_find_mbus_format() finds and
> > do:
> >
> > media-ctl --set-v4l2 "'csi':0 [fmt:SGBRG10/640x480 colorspace:raw]"
> >
> > I see:
> >
> > [ 184.251144] mc: media_release: Media Release
> > [ 184.254397] selected specific mbus code 0 for list nr 0 (fourcc
> > 0x59565955)
> > [ 184.264564] selected specific mbus code 0 for list nr 0 (fourcc
> > 0x59565955)
> > [ 184.274763] selected specific mbus code 0 for list nr 21 (fourcc
> > 0x36314247)
> > [ 184.285102] selected specific mbus code 0 for list nr 21 (fourcc
> > 0x36314247)
> > [ 184.295383] selected specific mbus code 0 for list nr 21 (fourcc
> > 0x36314247)
> > [ 184.305752] selected specific mbus code 0 for list nr 21 (fourcc
> > 0x36314247)
> >
> > 21 is the correct bayer format I want, but there's 0, so
> > "MEDIA_BUS_FMT_UYVY8_2X8" found the first 2 times. That is
> > IPUV3_COLORSPACE_YUV (1) while the correct Bayer format 21 is
> > IPUV3_COLORSPACE_RGB (0).
> >
> > so some format settings not yet correct.
>
> You need to specify the capture pixel format to v4l2-ctl. The driver
> defaults to YUYV (I think) otherwise. The CSI bridge will pad data
> with
> 0's on the right, so you need SGBRG16 (if I recall correctly, try
> SGRBG10 if it doesn't work).
>
right, now with:
v4l2-ctl -d "/dev/v4l/by-path/platform-30a90000.csi1_bridge-video-
index0" --set-fmt-video=width=640,height=480,pixelformat='GB16' --
stream-mmap --stream-to=test.raw --stream-count=1
indeed the colorspace mismatch is gone.
__media_pipeline_start returns broken pipe because of:
link validation failed for 'hi846 2-0020':0 -> 'imx8mq-mipi-csi2.0':0,
error -32:
imx7-csi 30a90000.csi1_bridge: walk: returning entity 'hi846 2-0020'
imx7-csi 30a90000.csi1_bridge: walk: returning entity 'imx8mq-mipi-
csi2.0'
hi846 2-0020: hi846: starting hi846_get_format
hi846 2-0020: Get format w=640 h=480 code=0x300e colorspace=0xb
imx7-csi 30a90000.csi1_bridge: v4l2_subdev_link_validate_default: field
does not match (source 1, sink 0)
imx7-csi 30a90000.csi1_bridge: v4l2_subdev_link_validate_default: link
was "hi846 2-0020":0 -> "imx8mq-mipi-csi2.0":0
imx7-csi 30a90000.csi1_bridge: link validation failed for 'hi846 2-
0020':0 -> 'imx8mq-mipi-csi2.0':0, error -32
...
imx7-csi 30a90000.csi1_bridge: pipeline start failed with -32
then I added the init_cfg() callback (that sets sink field to NONE) and
I could stream frames. So the v4l2/mc side of things looks ok, despite
the one thing:
Also, now when saying "--list-formats-ext" on the video node, it lists
all formats that the mipi driver supports (since it doesn't ask the
sensor anymore about that). That seems to be wrong.
Also, the frames are all black, so I'll need to debug the driver side
of things.
But I again append the current driver in case you want to have a look.
thanks so much for walking through this with me :) I won't forget the
dt properties and upstreaming - after I get correct frames.
martin
// SPDX-License-Identifier: GPL-2.0
/*
* Freescale i.MX8MQ SoC series MIPI-CSI receiver driver
*
* Copyright (C) 2021 Purism SPC
* Copyright (C) 2019 Linaro Ltd
* Copyright (C) 2015-2016 Freescale Semiconductor, Inc. All Rights Reserved.
* Copyright (C) 2011 - 2013 Samsung Electronics Co., Ltd.
*
*/
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/regmap.h>
#include <linux/mfd/syscon.h>
#include <linux/regulator/consumer.h>
#include <linux/reset.h>
#include <linux/spinlock.h>
#include <media/v4l2-common.h>
#include <media/v4l2-device.h>
#include <media/v4l2-fwnode.h>
#include <media/v4l2-mc.h>
#include <media/v4l2-subdev.h>
#define CSIS_DRIVER_NAME "imx8mq-mipi-csi2"
#define CSIS_SUBDEV_NAME CSIS_DRIVER_NAME
#define CSIS_PAD_SINK 0
#define CSIS_PAD_SOURCE 1
#define CSIS_PADS_NUM 2
#define MIPI_CSIS_DEF_PIX_WIDTH 640
#define MIPI_CSIS_DEF_PIX_HEIGHT 480
/* Register map definition */
/* i.MX8MQ CSI-2 controller CSR */
/* TODO 0x100, to dts? */
#define CSI2RX_CFG_NUM_LANES 0x100
#define CSI2RX_CFG_DISABLE_DATA_LANES 0x104
#define CSI2RX_BIT_ERR 0x108
#define CSI2RX_IRQ_STATUS 0x10C
#define CSI2RX_IRQ_MASK 0x110
#define CSI2RX_ULPS_STATUS 0x114
#define CSI2RX_PPI_ERRSOT_HS 0x118
#define CSI2RX_PPI_ERRSOTSYNC_HS 0x11C
#define CSI2RX_PPI_ERRESC 0x120
#define CSI2RX_PPI_ERRSYNCESC 0x124
#define CSI2RX_PPI_ERRCONTROL 0x128
#define CSI2RX_CFG_DISABLE_PAYLOAD_0 0x12C
#define CSI2RX_CFG_DISABLE_PAYLOAD_1 0x130
enum {
ST_POWERED = 1,
ST_STREAMING = 2,
ST_SUSPENDED = 4,
};
static const char * const mipi_csis_clk_id[] = {
"clk_core",
"clk_esc",
"clk_pxl",
"clk_clko2",
};
struct csis_imx8mq_hw_reset {
struct regmap *src;
u8 req_src;
u8 rst_val;
};
struct csis_imx8mq_phy_gpr {
struct regmap *gpr;
u8 req_src;
};
#define GPR_CSI2_1_RX_ENABLE BIT(13)
#define GPR_CSI2_1_VID_INTFC_ENB BIT(12)
#define GPR_CSI2_1_HSEL BIT(10)
#define GPR_CSI2_1_CONT_CLK_MODE BIT(8)
#define GPR_CSI2_1_S_PRG_RXHS_SETTLE(x) (((x) & 0x3F) << 2)
/*
* rxhs_settle[0] ... <720x480
* rxhs_settle[1] ... >720*480
*
* https://community.nxp.com/t5/i-MX-Processors/Explenation-for-HS-SETTLE-parameter-in-MIPI-CSI-D-PHY-registers/m-p/764275/highlight/true#M118744
*/
static u8 rxhs_settle[2] = { 0x14, 0x9 };
struct csi_state {
struct device *dev;
void __iomem *regs;
struct clk_bulk_data *clks;
struct reset_control *mrst;
struct regulator *mipi_phy_regulator;
u8 index;
struct v4l2_subdev sd;
struct media_pad pads[CSIS_PADS_NUM];
struct v4l2_async_notifier notifier;
struct v4l2_subdev *src_sd;
struct v4l2_fwnode_bus_mipi_csi2 bus;
u32 hs_settle;
u32 clk_settle;
struct mutex lock; /* Protect csis_fmt, format_mbus and state */
const struct csis_pix_format *csis_fmt;
struct v4l2_mbus_framefmt format_mbus;
u32 state;
struct csis_imx8mq_hw_reset hw_reset;
struct csis_imx8mq_phy_gpr phy_gpr;
u32 send_level;
};
/* -----------------------------------------------------------------------------
* Format helpers
*/
struct csis_pix_format {
u32 code;
u8 width;
};
static const struct csis_pix_format mipi_csis_formats[] = {
/* RAW (Bayer and greyscale) formats. */
{
.code = MEDIA_BUS_FMT_SBGGR8_1X8,
.width = 8,
}, {
.code = MEDIA_BUS_FMT_SGBRG8_1X8,
.width = 8,
}, {
.code = MEDIA_BUS_FMT_SGRBG8_1X8,
.width = 8,
}, {
.code = MEDIA_BUS_FMT_SRGGB8_1X8,
.width = 8,
}, {
.code = MEDIA_BUS_FMT_Y8_1X8,
.width = 8,
}, {
.code = MEDIA_BUS_FMT_SBGGR10_1X10,
.width = 10,
}, {
.code = MEDIA_BUS_FMT_SGBRG10_1X10,
.width = 10,
}, {
.code = MEDIA_BUS_FMT_SGRBG10_1X10,
.width = 10,
}, {
.code = MEDIA_BUS_FMT_SRGGB10_1X10,
.width = 10,
}, {
.code = MEDIA_BUS_FMT_Y10_1X10,
.width = 10,
}, {
.code = MEDIA_BUS_FMT_SBGGR12_1X12,
.width = 12,
}, {
.code = MEDIA_BUS_FMT_SGBRG12_1X12,
.width = 12,
}, {
.code = MEDIA_BUS_FMT_SGRBG12_1X12,
.width = 12,
}, {
.code = MEDIA_BUS_FMT_SRGGB12_1X12,
.width = 12,
}, {
.code = MEDIA_BUS_FMT_Y12_1X12,
.width = 12,
}, {
.code = MEDIA_BUS_FMT_SBGGR14_1X14,
.width = 14,
}, {
.code = MEDIA_BUS_FMT_SGBRG14_1X14,
.width = 14,
}, {
.code = MEDIA_BUS_FMT_SGRBG14_1X14,
.width = 14,
}, {
.code = MEDIA_BUS_FMT_SRGGB14_1X14,
.width = 14,
}
};
static const struct csis_pix_format *find_csis_format(u32 code)
{
unsigned int i;
for (i = 0; i < ARRAY_SIZE(mipi_csis_formats); i++)
if (code == mipi_csis_formats[i].code)
return &mipi_csis_formats[i];
return NULL;
}
/* -----------------------------------------------------------------------------
* Hardware configuration
*/
static inline u32 mipi_csis_read(struct csi_state *state, u32 reg)
{
return readl(state->regs + reg);
}
static inline void mipi_csis_write(struct csi_state *state, u32 reg, u32 val)
{
writel(val, state->regs + reg);
}
static void mipi_csis_sw_reset(struct csi_state *state)
{
struct device *dev = state->dev;
struct device_node *np = dev->of_node;
struct device_node *node;
phandle phandle;
u32 out_val[3];
int ret;
dev_dbg(dev, "%s: starting\n", __func__);
ret = of_property_read_u32_array(np, "csis-phy-reset", out_val, 3);
if (ret) {
dev_info(dev, "no csis-hw-reset property found: %d\n", ret);
return;
}
phandle = *out_val;
node = of_find_node_by_phandle(phandle);
if (!node) {
ret = PTR_ERR(node);
dev_dbg(dev, "not find src node by phandle: %d\n", ret);
}
state->hw_reset.src = syscon_node_to_regmap(node);
if (IS_ERR(state->hw_reset.src)) {
ret = PTR_ERR(state->hw_reset.src);
dev_err(dev, "failed to get src regmap: %d\n", ret);
}
of_node_put(node);
if (ret < 0)
return;
state->hw_reset.req_src = out_val[1];
state->hw_reset.rst_val = out_val[2];
/* reset imx8mq mipi phy */
regmap_update_bits(state->hw_reset.src,
state->hw_reset.req_src,
state->hw_reset.rst_val,
state->hw_reset.rst_val);
msleep(20);
dev_dbg(dev, "%s: done\n", __func__);
return;
}
static void mipi_csis_system_enable(struct csi_state *state, int on)
{
struct device *dev = state->dev;
struct device_node *np = dev->of_node;
struct device_node *node;
phandle phandle;
u32 out_val[2];
int ret;
if (!on) {
/* Disable Data lanes */
mipi_csis_write(state, CSI2RX_CFG_DISABLE_DATA_LANES, 0xf);
return;
}
ret = of_property_read_u32_array(np, "phy-gpr", out_val, 2);
if (ret) {
dev_info(dev, "no phy-gpr property found\n");
return;
}
phandle = *out_val;
node = of_find_node_by_phandle(phandle);
if (!node) {
dev_dbg(dev, "not find gpr node by phandle\n");
ret = PTR_ERR(node);
}
state->phy_gpr.gpr = syscon_node_to_regmap(node);
if (IS_ERR(state->phy_gpr.gpr)) {
dev_err(dev, "failed to get gpr regmap\n");
ret = PTR_ERR(state->phy_gpr.gpr);
}
of_node_put(node);
if (ret < 0)
return;
state->phy_gpr.req_src = out_val[1];
regmap_update_bits(state->phy_gpr.gpr,
state->phy_gpr.req_src,
0x3FFF,
GPR_CSI2_1_RX_ENABLE |
GPR_CSI2_1_VID_INTFC_ENB |
GPR_CSI2_1_HSEL |
GPR_CSI2_1_CONT_CLK_MODE |
GPR_CSI2_1_S_PRG_RXHS_SETTLE(state->
hs_settle));
dev_dbg(dev, "%s: hs_settle: 0x%X\n", __func__, state->hs_settle);
return;
}
static int mipi_csis_calculate_params(struct csi_state *state)
{
s64 link_freq;
/* Calculate the line rate from the pixel rate. */
link_freq = v4l2_get_link_freq(state->src_sd->ctrl_handler,
state->csis_fmt->width,
state->bus.num_data_lanes * 2);
if (link_freq < 0) {
dev_err(state->dev, "Unable to obtain link frequency: %d\n",
(int)link_freq);
return link_freq;
}
dev_err(state->dev, "link frequency calculated with %d bpp: %lld\n", state->csis_fmt->width, link_freq);
state->hs_settle = rxhs_settle[0];
return 0;
}
static void mipi_csis_set_params(struct csi_state *state)
{
int lanes = state->bus.num_data_lanes;
u32 val = 0;
int i;
/* Lanes */
mipi_csis_write(state, CSI2RX_CFG_NUM_LANES, lanes - 1);
for (i = 0; i < lanes; i++)
val |= (1 << i);
val = 0xF & ~val;
mipi_csis_write(state, CSI2RX_CFG_DISABLE_DATA_LANES, val);
dev_dbg(state->dev, "imx8mq: CSI2RX_CFG_DISABLE_DATA_LANES: 0x%X\n", val);
/* Mask interrupt */
// Don't let ULPS (ultra-low power status) interrupts flood
mipi_csis_write(state, CSI2RX_IRQ_MASK, 0x1ff);
mipi_csis_write(state, 0x180, 1);
/* vid_vc */
mipi_csis_write(state, 0x184, 1);
mipi_csis_write(state, 0x188, state->send_level);
}
static int mipi_csis_clk_enable(struct csi_state *state)
{
return clk_bulk_prepare_enable(ARRAY_SIZE(mipi_csis_clk_id), state->clks);
}
static void mipi_csis_clk_disable(struct csi_state *state)
{
clk_bulk_disable_unprepare(ARRAY_SIZE(mipi_csis_clk_id), state->clks);
}
static int mipi_csis_clk_get(struct csi_state *state)
{
unsigned int i;
int ret;
state->clks = devm_kcalloc(state->dev, ARRAY_SIZE(mipi_csis_clk_id),
sizeof(*state->clks), GFP_KERNEL);
if (!state->clks)
return -ENOMEM;
for (i = 0; i < ARRAY_SIZE(mipi_csis_clk_id); i++)
state->clks[i].id = mipi_csis_clk_id[i];
ret = devm_clk_bulk_get(state->dev, ARRAY_SIZE(mipi_csis_clk_id),
state->clks);
return ret;
}
static void mipi_csis_start_stream(struct csi_state *state)
{
mipi_csis_sw_reset(state);
mipi_csis_set_params(state);
mipi_csis_system_enable(state, true);
}
static void mipi_csis_stop_stream(struct csi_state *state)
{
mipi_csis_system_enable(state, false);
}
/* -----------------------------------------------------------------------------
* V4L2 subdev operations
*/
static struct csi_state *mipi_sd_to_csis_state(struct v4l2_subdev *sdev)
{
return container_of(sdev, struct csi_state, sd);
}
static int mipi_csis_s_stream(struct v4l2_subdev *sd, int enable)
{
struct csi_state *state = mipi_sd_to_csis_state(sd);
int ret;
mipi_csis_write(state, CSI2RX_IRQ_MASK, 0x008);
dev_dbg(state->dev, "%s: enable: %d\n", __func__, enable);
if (enable) {
ret = mipi_csis_calculate_params(state);
if (ret < 0)
return ret;
ret = pm_runtime_get_sync(state->dev);
if (ret < 0) {
pm_runtime_put_noidle(state->dev);
return ret;
}
ret = v4l2_subdev_call(state->src_sd, core, s_power, 1);
if (ret < 0 && ret != -ENOIOCTLCMD)
goto done;
}
mutex_lock(&state->lock);
if (enable) {
if (state->state & ST_SUSPENDED) {
ret = -EBUSY;
goto unlock;
}
mipi_csis_start_stream(state);
ret = v4l2_subdev_call(state->src_sd, video, s_stream, 1);
if (ret < 0)
goto unlock;
state->state |= ST_STREAMING;
} else {
v4l2_subdev_call(state->src_sd, video, s_stream, 0);
ret = v4l2_subdev_call(state->src_sd, core, s_power, 0);
if (ret == -ENOIOCTLCMD)
ret = 0;
mipi_csis_stop_stream(state);
state->state &= ~ST_STREAMING;
}
unlock:
mutex_unlock(&state->lock);
done:
if (!enable || ret < 0)
pm_runtime_put(state->dev);
return ret;
}
static struct v4l2_mbus_framefmt *
mipi_csis_get_format(struct csi_state *state,
struct v4l2_subdev_pad_config *cfg,
enum v4l2_subdev_format_whence which,
unsigned int pad)
{
if (which == V4L2_SUBDEV_FORMAT_TRY)
return v4l2_subdev_get_try_format(&state->sd, cfg, pad);
return &state->format_mbus;
}
static int mipi_csi2_init_cfg(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg)
{
struct csi_state *state = mipi_sd_to_csis_state(sd);
struct v4l2_mbus_framefmt *fmt_sink;
struct v4l2_mbus_framefmt *fmt_source;
enum v4l2_subdev_format_whence which;
which = cfg ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
fmt_sink = mipi_csis_get_format(state, cfg, which, CSIS_PAD_SINK);
fmt_sink->code = MEDIA_BUS_FMT_SGBRG10_1X10;
fmt_sink->width = MIPI_CSIS_DEF_PIX_WIDTH;
fmt_sink->height = MIPI_CSIS_DEF_PIX_HEIGHT;
fmt_sink->field = V4L2_FIELD_NONE;
fmt_sink->colorspace = V4L2_COLORSPACE_RAW;
fmt_sink->xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(fmt_sink->colorspace);
fmt_sink->ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(fmt_sink->colorspace);
fmt_sink->quantization =
V4L2_MAP_QUANTIZATION_DEFAULT(false, fmt_sink->colorspace,
fmt_sink->ycbcr_enc);
/*
* When called from mipi_csis_subdev_init() to initialize the active
* configuration, cfg is NULL, which indicates there's no source pad
* configuration to set.
*/
if (!cfg)
return 0;
fmt_source = mipi_csis_get_format(state, cfg, which, CSIS_PAD_SOURCE);
*fmt_source = *fmt_sink;
return 0;
}
static int mipi_csis_get_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat)
{
struct csi_state *state = mipi_sd_to_csis_state(sd);
struct v4l2_mbus_framefmt *fmt;
fmt = mipi_csis_get_format(state, cfg, sdformat->which, sdformat->pad);
mutex_lock(&state->lock);
sdformat->format = *fmt;
mutex_unlock(&state->lock);
return 0;
}
static int mipi_csis_enum_mbus_code(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_mbus_code_enum *code)
{
struct csi_state *state = mipi_sd_to_csis_state(sd);
/*
* We can't transcode in any way, the source format is identical
* to the sink format.
*/
if (code->pad == CSIS_PAD_SOURCE) {
struct v4l2_mbus_framefmt *fmt;
if (code->index > 0)
return -EINVAL;
fmt = mipi_csis_get_format(state, cfg, code->which, code->pad);
code->code = fmt->code;
return 0;
}
if (code->pad != CSIS_PAD_SINK)
return -EINVAL;
if (code->index >= ARRAY_SIZE(mipi_csis_formats))
return -EINVAL;
code->code = mipi_csis_formats[code->index].code;
return 0;
}
static int mipi_csis_set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat)
{
struct csi_state *state = mipi_sd_to_csis_state(sd);
struct csis_pix_format const *csis_fmt;
struct v4l2_mbus_framefmt *fmt;
/*
* The CSIS can't transcode in any way, the source format can't be
* modified.
*/
if (sdformat->pad == CSIS_PAD_SOURCE)
return mipi_csis_get_fmt(sd, cfg, sdformat);
if (sdformat->pad != CSIS_PAD_SINK)
return -EINVAL;
dev_err(state->dev, "trying to find format for code %d\n",
sdformat->format.code);
csis_fmt = find_csis_format(sdformat->format.code);
if (!csis_fmt) {
dev_err(state->dev, "no format found based on code %d\n",
sdformat->format.code);
csis_fmt = &mipi_csis_formats[0];
}
fmt = mipi_csis_get_format(state, cfg, sdformat->which, sdformat->pad);
mutex_lock(&state->lock);
fmt->code = csis_fmt->code;
fmt->width = sdformat->format.width;
fmt->height = sdformat->format.height;
sdformat->format = *fmt;
/* Propagate the format from sink to source. */
fmt = mipi_csis_get_format(state, cfg, sdformat->which,
CSIS_PAD_SOURCE);
*fmt = sdformat->format;
/* Store the CSI2 format descriptor for active formats. */
if (sdformat->which == V4L2_SUBDEV_FORMAT_ACTIVE)
state->csis_fmt = csis_fmt;
mutex_unlock(&state->lock);
if (sdformat->format.width * sdformat->format.height > 720 * 480) {
state->hs_settle = rxhs_settle[1];
} else {
state->hs_settle = rxhs_settle[0];
}
state->send_level = 64;
dev_dbg(state->dev,
"%s: format %dx%d send_level %d hs_settle 0x%X\n", __func__,
sdformat->format.width, sdformat->format.height,
state->send_level, state->hs_settle);
return 0;
}
static int mipi_csis_log_status(struct v4l2_subdev *sd)
{
struct csi_state *state = mipi_sd_to_csis_state(sd);
mutex_lock(&state->lock);
mutex_unlock(&state->lock);
return 0;
}
static const struct v4l2_subdev_core_ops mipi_csis_core_ops = {
.log_status = mipi_csis_log_status,
};
static const struct v4l2_subdev_video_ops mipi_csis_video_ops = {
.s_stream = mipi_csis_s_stream,
};
static const struct v4l2_subdev_pad_ops mipi_csis_pad_ops = {
.init_cfg = mipi_csi2_init_cfg,
.enum_mbus_code = mipi_csis_enum_mbus_code,
.get_fmt = mipi_csis_get_fmt,
.set_fmt = mipi_csis_set_fmt,
};
static const struct v4l2_subdev_ops mipi_csis_subdev_ops = {
.core = &mipi_csis_core_ops,
.video = &mipi_csis_video_ops,
.pad = &mipi_csis_pad_ops,
};
/* -----------------------------------------------------------------------------
* Media entity operations
*/
static int mipi_csis_link_setup(struct media_entity *entity,
const struct media_pad *local_pad,
const struct media_pad *remote_pad, u32 flags)
{
struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
struct csi_state *state = mipi_sd_to_csis_state(sd);
struct v4l2_subdev *remote_sd;
dev_dbg(state->dev, "link setup %s -> %s", remote_pad->entity->name,
local_pad->entity->name);
/* We only care about the link to the source. */
if (!(local_pad->flags & MEDIA_PAD_FL_SINK))
return 0;
remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
if (flags & MEDIA_LNK_FL_ENABLED) {
if (state->src_sd)
return -EBUSY;
state->src_sd = remote_sd;
} else {
state->src_sd = NULL;
}
return 0;
}
static const struct media_entity_operations mipi_csis_entity_ops = {
.link_setup = mipi_csis_link_setup,
.link_validate = v4l2_subdev_link_validate,
.get_fwnode_pad = v4l2_subdev_get_fwnode_pad_1_to_1,
};
/* -----------------------------------------------------------------------------
* Async subdev notifier
*/
static struct csi_state *
mipi_notifier_to_csis_state(struct v4l2_async_notifier *n)
{
return container_of(n, struct csi_state, notifier);
}
static int mipi_csis_notify_bound(struct v4l2_async_notifier *notifier,
struct v4l2_subdev *sd,
struct v4l2_async_subdev *asd)
{
struct csi_state *state = mipi_notifier_to_csis_state(notifier);
struct media_pad *sink = &state->sd.entity.pads[CSIS_PAD_SINK];
return v4l2_create_fwnode_links_to_pad(sd, sink, 0);
}
static const struct v4l2_async_notifier_operations mipi_csis_notify_ops = {
.bound = mipi_csis_notify_bound,
};
static int mipi_csis_async_register(struct csi_state *state)
{
struct v4l2_fwnode_endpoint vep = {
.bus_type = V4L2_MBUS_CSI2_DPHY,
};
struct v4l2_async_subdev *asd;
struct fwnode_handle *ep;
unsigned int i;
int ret;
v4l2_async_notifier_init(&state->notifier);
ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(state->dev), 0, 0,
FWNODE_GRAPH_ENDPOINT_NEXT);
if (!ep)
return -ENOTCONN;
ret = v4l2_fwnode_endpoint_parse(ep, &vep);
if (ret)
goto err_parse;
for (i = 0; i < vep.bus.mipi_csi2.num_data_lanes; ++i) {
if (vep.bus.mipi_csi2.data_lanes[i] != i + 1) {
dev_err(state->dev,
"data lanes reordering is not supported");
goto err_parse;
}
}
state->bus = vep.bus.mipi_csi2;
dev_dbg(state->dev, "data lanes: %d\n", state->bus.num_data_lanes);
dev_dbg(state->dev, "flags: 0x%08x\n", state->bus.flags);
asd = v4l2_async_notifier_add_fwnode_remote_subdev(
&state->notifier, ep, struct v4l2_async_subdev);
if (IS_ERR(asd)) {
ret = PTR_ERR(asd);
goto err_parse;
}
fwnode_handle_put(ep);
state->notifier.ops = &mipi_csis_notify_ops;
ret = v4l2_async_subdev_notifier_register(&state->sd, &state->notifier);
if (ret)
return ret;
return v4l2_async_register_subdev(&state->sd);
err_parse:
fwnode_handle_put(ep);
return ret;
}
/* -----------------------------------------------------------------------------
* Suspend/resume
*/
static int mipi_csis_pm_suspend(struct device *dev, bool runtime)
{
struct v4l2_subdev *sd = dev_get_drvdata(dev);
struct csi_state *state = mipi_sd_to_csis_state(sd);
int ret = 0;
mutex_lock(&state->lock);
if (state->state & ST_POWERED) {
mipi_csis_stop_stream(state);
mipi_csis_clk_disable(state);
state->state &= ~ST_POWERED;
if (!runtime)
state->state |= ST_SUSPENDED;
}
mutex_unlock(&state->lock);
return ret ? -EAGAIN : 0;
}
static int mipi_csis_pm_resume(struct device *dev, bool runtime)
{
struct v4l2_subdev *sd = dev_get_drvdata(dev);
struct csi_state *state = mipi_sd_to_csis_state(sd);
int ret = 0;
mutex_lock(&state->lock);
if (!runtime && !(state->state & ST_SUSPENDED))
goto unlock;
if (!(state->state & ST_POWERED)) {
state->state |= ST_POWERED;
mipi_csis_clk_enable(state);
}
if (state->state & ST_STREAMING)
mipi_csis_start_stream(state);
state->state &= ~ST_SUSPENDED;
unlock:
mutex_unlock(&state->lock);
return ret ? -EAGAIN : 0;
}
static int __maybe_unused mipi_csis_suspend(struct device *dev)
{
return mipi_csis_pm_suspend(dev, false);
}
static int __maybe_unused mipi_csis_resume(struct device *dev)
{
return mipi_csis_pm_resume(dev, false);
}
static int __maybe_unused mipi_csis_runtime_suspend(struct device *dev)
{
return mipi_csis_pm_suspend(dev, true);
}
static int __maybe_unused mipi_csis_runtime_resume(struct device *dev)
{
return mipi_csis_pm_resume(dev, true);
}
static const struct dev_pm_ops mipi_csis_pm_ops = {
SET_RUNTIME_PM_OPS(mipi_csis_runtime_suspend, mipi_csis_runtime_resume,
NULL)
SET_SYSTEM_SLEEP_PM_OPS(mipi_csis_suspend, mipi_csis_resume)
};
/* -----------------------------------------------------------------------------
* Probe/remove & platform driver
*/
static int mipi_csis_subdev_init(struct csi_state *state)
{
struct v4l2_subdev *sd = &state->sd;
v4l2_subdev_init(sd, &mipi_csis_subdev_ops);
sd->owner = THIS_MODULE;
snprintf(sd->name, sizeof(sd->name), "%s.%d",
CSIS_SUBDEV_NAME, state->index);
sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
sd->ctrl_handler = NULL;
sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
sd->entity.ops = &mipi_csis_entity_ops;
sd->dev = state->dev;
state->csis_fmt = &mipi_csis_formats[0];
mipi_csi2_init_cfg(sd, NULL);
state->pads[CSIS_PAD_SINK].flags = MEDIA_PAD_FL_SINK
| MEDIA_PAD_FL_MUST_CONNECT;
state->pads[CSIS_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE
| MEDIA_PAD_FL_MUST_CONNECT;
return media_entity_pads_init(&sd->entity, CSIS_PADS_NUM,
state->pads);
}
/* TODO needed? */
static int mipi_csis_parse_dt(struct csi_state *state)
{
struct device_node *node = state->dev->of_node;
return 0;
}
static int mipi_csis_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct csi_state *state;
int ret;
state = devm_kzalloc(dev, sizeof(*state), GFP_KERNEL);
if (!state)
return -ENOMEM;
mutex_init(&state->lock);
state->dev = dev;
/* Parse DT properties. */
ret = mipi_csis_parse_dt(state);
if (ret < 0) {
dev_err(dev, "Failed to parse device tree: %d\n", ret);
return ret;
}
/* Acquire resources. */
state->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(state->regs))
return PTR_ERR(state->regs);
ret = mipi_csis_clk_get(state);
if (ret < 0)
return ret;
ret = mipi_csis_clk_enable(state);
if (ret < 0) {
dev_err(state->dev, "failed to enable clocks: %d\n", ret);
return ret;
}
/* Initialize and register the subdev. */
ret = mipi_csis_subdev_init(state);
if (ret < 0)
goto disable_clock;
platform_set_drvdata(pdev, &state->sd);
ret = mipi_csis_async_register(state);
if (ret < 0) {
dev_err(dev, "async register failed: %d\n", ret);
goto cleanup;
}
/* Enable runtime PM. */
pm_runtime_enable(dev);
if (!pm_runtime_enabled(dev)) {
ret = mipi_csis_pm_resume(dev, true);
if (ret < 0)
goto cleanup;
}
dev_info(dev, "lanes: %d\n", state->bus.num_data_lanes);
return 0;
cleanup:
media_entity_cleanup(&state->sd.entity);
v4l2_async_notifier_unregister(&state->notifier);
v4l2_async_notifier_cleanup(&state->notifier);
v4l2_async_unregister_subdev(&state->sd);
disable_clock:
mipi_csis_clk_disable(state);
mutex_destroy(&state->lock);
return ret;
}
static int mipi_csis_remove(struct platform_device *pdev)
{
struct v4l2_subdev *sd = platform_get_drvdata(pdev);
struct csi_state *state = mipi_sd_to_csis_state(sd);
v4l2_async_notifier_unregister(&state->notifier);
v4l2_async_notifier_cleanup(&state->notifier);
v4l2_async_unregister_subdev(&state->sd);
pm_runtime_disable(&pdev->dev);
mipi_csis_pm_suspend(&pdev->dev, true);
mipi_csis_clk_disable(state);
media_entity_cleanup(&state->sd.entity);
mutex_destroy(&state->lock);
pm_runtime_set_suspended(&pdev->dev);
return 0;
}
static const struct of_device_id mipi_csis_of_match[] = {
{ .compatible = "fsl,imx8mq-mipi-csi2",},
{ /* sentinel */ },
};
MODULE_DEVICE_TABLE(of, mipi_csis_of_match);
static struct platform_driver mipi_csis_driver = {
.probe = mipi_csis_probe,
.remove = mipi_csis_remove,
.driver = {
.of_match_table = mipi_csis_of_match,
.name = CSIS_DRIVER_NAME,
.pm = &mipi_csis_pm_ops,
},
};
module_platform_driver(mipi_csis_driver);
MODULE_DESCRIPTION("i.MX8MQ MIPI CSI-2 receiver driver");
MODULE_LICENSE("GPL v2");
MODULE_ALIAS("platform:imx8mq-mipi-csi2");
