Hi Bingbu, Claus,
On 8/21/23 12:07, Claus Stovgaard wrote:
> Bingbu
>
> On Mon, 2023-08-21 at 08:55 +0200, Claus Stovgaard wrote:
>> Bingbu
>>
>> On Mon, 2023-08-21 at 14:22 +0800, Bingbu Cao wrote:
>>>
>>> Claus,
>>>
>>>
>>> On 8/21/23 11:14 AM, Bingbu Cao wrote:
>>>
>>>
>>> I see that the ivsc driver has not been in master branch. Before
>>> that,
>>> could you try several hack to check whether camera can work on
>>> master?
>>>
>>> https://github.com/bingbucao/linux/commits/ipu_dev
>>>
>>> 7ebff51284d9 media: ov01a10: hack ivsc to make camera can work
>>> 01cc9f3d1b61 i2c: ljca: Call acpi_dev_clear_dependencies()
>>> 92e5d122e105 vsc: Defer firmware loading to avoid long probing time
>>> 5f5d5f0df06b driver: ivsc: add intel ivsc driver
>>> 0f4819dec533 Revert "gpio: Add support for Intel LJCA USB GPIO
>>> driver"
>>
>> Thanks for your quick reply.
>>
>> I was missing understanding of ivsc when I wrote the mail yesterday.
>> Got some basic understanding yesterday after I wrote, and big thanks
>> for confirming it, and also thanks for your ipu_dev branch. Has just
>> cloned it, and is building as I write.
>>
>> Just fyi, I was trying to hack something together yesterday, and got
>> further, but not yet working.
>>
>> My hack was to combine the out-of-tree ivsc drivers and firmware from
>>
>> * https://github.com/intel/ivsc-firmware.git
>> * https://github.com/intel/ivsc-driver.git
>>
>> Though noticed that I need some changes to the sensor driver so was
>> also building all the drivers from ipu6-drivers (with minor changes
>> to
>> get_pages) as out-of-tree modules.
>>
>> * https://github.com/intel/ipu6-drivers.git ;
>>
>> Here I used everything beside media/pci/*.ko files. I could see the
>> sensor and got further, but was missing the last.
>>
>> Looking forward to try your branch. Looks much cleaner, and would be
>> nice to get working :)
>>
>
> I got it to work on Dell XPS 9320.
> With some minor changes compared to your guide in Documentation/admin-
> guide/media/ipu6-isys.rst
>
> [root@xps-1 ]# uname -a
> Linux xps-1 6.5.0-rc7-g7ebff51284d9 #1 SMP PREEMPT_DYNAMIC Mon Aug 21
> 09:02:20 CEST 2023 x86_64 GNU/Linux
>
> [root@xps-1 ]# media-ctl -d /dev/media0 -p | tail -n10
>
> - entity 2149: ov01a10 16-0036 (1 pad, 1 link)
> type V4L2 subdev subtype Sensor flags 0
> device node name /dev/v4l-subdev4
> pad0: Source
> [fmt:SBGGR10_1X10/1280x800 field:none colorspace:raw
> crop.bounds:(0,0)/1296x816
> crop:(8,8)/1280x800]
> -> "Intel IPU6 CSI2 2":0 []
>
> So i2c is 16-0036 - and we use it for setup like your guide.
>
> export MDEV=/dev/media0
>
> media-ctl -d $MDEV -l "\"ov01a10 17-0036\":0 -> \"Intel IPU6 CSI2
> 2\":0[1]"
>
> media-ctl -d $MDEV -V "\"ov01a10 17-0036\":0 [fmt:SBGGR10/1280x800]"
> media-ctl -d $MDEV -V "\"Intel IPU6 CSI2 2\":0 [fmt:SBGGR10/1280x800]"
> media-ctl -d $MDEV -V "\"Intel IPU6 CSI2 2\":1 [fmt:SBGGR10/1280x800]"
>
> media-ctl -d $MDEV -l "\"ov01a10 17-0036\":0 -> \"Intel IPU6 CSI2
> 2\":0[1]"
> media-ctl -d $MDEV -l "\"Intel IPU6 CSI2 2\":1 ->\"Intel IPU6 ISYS
> Capture 0\":0[5]"
>
> Though yavta does not work in the way as described in the guide.
>
> [root@xps-1 ]# yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --
> file=/tmp/frame-#.bin -f SBGGR10 /dev/video0
> Device /dev/video0 opened.
> Device `ipu6' on `PCI:0000:00:05.0' (driver 'isys') supports video,
> capture, with mplanes.
> Video format set: SBGGR10 (30314742) 1280x800 field none, 1 planes:
> * Stride 2560, buffer size 2050560
> Video format: SBGGR10 (30314742) 1280x800 field none, 1 planes:
> * Stride 2560, buffer size 2050560
> Unable to request buffers: Invalid argument (22).
>
>
> So I changed to use v4l2-ctl
>
> [root@xps-1 ]# v4l2-ctl -d /dev/video0 --set-fmt-video
> width=1280,height=800,pixelformat=BG10 --stream-mmap --stream-count=1 -
> -stream-to=frame.bin
>
> With this I created raw data in BG10 format, and later used a small
> python script with numpy and opencv to look at the data.
>
> #!/usr/bin/env python3
> # Demosaicing Bayer Raw image
>
> import cv2
> import numpy as np
>
> width = 1280
> height = 800
>
> with open("frame.bin", "rb") as rawimg:
> # Read the bayer data
> data = np.fromfile(rawimg, np.uint16, width * height)
> bayer = np.reshape(data, (height, width))
>
> # Just a offset gain to be able to see something
> for x in range(0, len(bayer)):
> for y in range(0, len(bayer[0])):
> bayer[x, y] = (bayer[x,y] << 8)
>
> rgb = cv2.cvtColor(bayer, cv2.COLOR_BayerBGGR2RGB)
>
> cv2.imshow('rgb', rgb)
> cv2.waitKey()
> cv2.destroyAllWindows()
>
>
> Thanks for the help, and now we know what is needed to make it work on
> top of yesterdays rc7
Bingbu, thank you for the series. Claus, thank you for the python
test-script.
I've just given this a test-run on top of a recent checkout
of media-staging/master, so on top of the drivers/media
changes headed for 6.6 .
And with the attached changes + the ov2740 changes from
the github ipu6-drevers repo I got this working on
a lenovo thinkpad x1 yoga with an ov2740 driver.
I've attached the necessary changes to adjust the new ipu6
code for the v4l2-async changes which are queued up for
kernel 6.6 .
Regards,
Hans
From 37a18f7684340b8655b6f0c712b15b4bc6966a3d Mon Sep 17 00:00:00 2001
From: Hans de Goede <hdegoede@xxxxxxxxxx>
Date: Thu, 31 Aug 2023 14:48:38 +0200
Subject: [PATCH] media: ipu6: Adjust for (pending) 6.6 v4l2-async API changes
Adjust the ipu6 code to work with the (pending) 6.6 v4l2-async API changes.
Signed-off-by: Hans de Goede <hdegoede@xxxxxxxxxx>
---
.../media/pci/intel/ipu6/ipu6-isys-mcd-phy.c | 8 +--
drivers/media/pci/intel/ipu6/ipu6-isys.c | 65 ++++++++++++-------
drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +-
drivers/media/pci/intel/ipu6/ipu6.c | 5 +-
4 files changed, 48 insertions(+), 32 deletions(-)
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-mcd-phy.c b/drivers/media/pci/intel/ipu6/ipu6-isys-mcd-phy.c
index 226d647d1da0..562e8a830cc3 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-mcd-phy.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-mcd-phy.c
@@ -583,12 +583,12 @@ static int ipu6_isys_mcd_phy_common_init(struct ipu6_isys *isys)
struct ipu6_device *isp = adev->isp;
void __iomem *isp_base = isp->base;
struct sensor_async_subdev *s_asd;
- struct v4l2_async_subdev *asd;
+ struct v4l2_async_connection *asd;
void __iomem *phy_base;
unsigned int phy_id;
unsigned int i;
- list_for_each_entry(asd, &isys->notifier.asd_list, asd_list) {
+ list_for_each_entry(asd, &isys->notifier.done_list, asc_entry) {
s_asd = container_of(asd, struct sensor_async_subdev, asd);
phy_id = s_asd->csi2.port / 4;
phy_base = isp_base + IPU6_ISYS_MCD_PHY_BASE(phy_id);
@@ -641,12 +641,12 @@ static int ipu6_isys_mcd_phy_config(struct ipu6_isys *isys)
struct ipu6_device *isp = adev->isp;
void __iomem *isp_base = isp->base;
struct sensor_async_subdev *s_asd;
+ struct v4l2_async_connection *asd;
struct ipu6_isys_csi2_config cfg;
- struct v4l2_async_subdev *asd;
unsigned int i, phy_port, phy_id;
void __iomem *phy_base;
- list_for_each_entry(asd, &isys->notifier.asd_list, asd_list) {
+ list_for_each_entry(asd, &isys->notifier.done_list, asc_entry) {
s_asd = container_of(asd, struct sensor_async_subdev, asd);
cfg.port = s_asd->csi2.port;
cfg.nlanes = s_asd->csi2.nlanes;
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.c b/drivers/media/pci/intel/ipu6/ipu6-isys.c
index c5db58f12c93..df81e81a1029 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.c
@@ -664,7 +664,7 @@ static int isys_iwake_watermark_cleanup(struct ipu6_isys *isys)
/* The .bound() notifier callback when a match is found */
static int isys_notifier_bound(struct v4l2_async_notifier *notifier,
struct v4l2_subdev *sd,
- struct v4l2_async_subdev *asd)
+ struct v4l2_async_connection *asd)
{
struct ipu6_isys *isys =
container_of(notifier, struct ipu6_isys, notifier);
@@ -691,44 +691,59 @@ static const struct v4l2_async_notifier_operations isys_async_ops = {
.complete = isys_notifier_complete,
};
-static int isys_fwnode_parse(struct device *dev,
- struct v4l2_fwnode_endpoint *vep,
- struct v4l2_async_subdev *asd)
-{
- struct sensor_async_subdev *s_asd =
- container_of(asd, struct sensor_async_subdev, asd);
-
- s_asd->csi2.port = vep->base.port;
- s_asd->csi2.nlanes = vep->bus.mipi_csi2.num_data_lanes;
-
- return 0;
-}
-
static int isys_notifier_init(struct ipu6_isys *isys)
{
- size_t asd_struct_size = sizeof(struct sensor_async_subdev);
+ const struct ipu6_isys_internal_csi2_pdata *csi2 =
+ &isys->pdata->ipdata->csi2;
struct device *dev = &isys->adev->auxdev.dev;
- struct ipu6_device *isp = isys->adev->isp;
+ unsigned int i;
int ret;
- v4l2_async_nf_init(&isys->notifier);
- ret = v4l2_async_nf_parse_fwnode_endpoints(&isp->pdev->dev,
- &isys->notifier,
- asd_struct_size,
- isys_fwnode_parse);
- if (ret < 0) {
- dev_err(dev, "parse fwnode endpoints failed: %d\n", ret);
+ v4l2_async_nf_init(&isys->notifier, &isys->v4l2_dev);
+ for (i = 0; i < csi2->nports; i++) {
+ struct v4l2_fwnode_endpoint vep = {
+ .bus_type = V4L2_MBUS_CSI2_DPHY
+ };
+ struct sensor_async_subdev *s_asd;
+ struct fwnode_handle *ep;
+
+ ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev->parent), i, 0,
+ FWNODE_GRAPH_ENDPOINT_NEXT);
+ if (!ep)
+ continue;
+
+ ret = v4l2_fwnode_endpoint_parse(ep, &vep);
+ if (ret)
+ goto err_parse;
+
+ s_asd = v4l2_async_nf_add_fwnode_remote(&isys->notifier, ep,
+ struct
+ sensor_async_subdev);
+ if (IS_ERR(s_asd)) {
+ ret = PTR_ERR(s_asd);
+ goto err_parse;
+ }
+
+ s_asd->csi2.port = vep.base.port;
+ s_asd->csi2.nlanes = vep.bus.mipi_csi2.num_data_lanes;
+
+ fwnode_handle_put(ep);
+
+ continue;
+
+err_parse:
+ fwnode_handle_put(ep);
return ret;
}
- if (list_empty(&isys->notifier.asd_list)) {
+ if (list_empty(&isys->notifier.waiting_list)) {
/* isys probe could continue with async subdevs missing */
dev_warn(dev, "no subdev found in graph\n");
return 0;
}
isys->notifier.ops = &isys_async_ops;
- ret = v4l2_async_nf_register(&isys->v4l2_dev, &isys->notifier);
+ ret = v4l2_async_nf_register(&isys->notifier);
if (ret) {
dev_err(dev, "failed to register async notifier : %d\n", ret);
v4l2_async_nf_cleanup(&isys->notifier);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h
index ef0115914297..5c23ad3e5dea 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h
@@ -94,7 +94,7 @@ struct ipu6_isys_csi2_config {
};
struct sensor_async_subdev {
- struct v4l2_async_subdev asd;
+ struct v4l2_async_connection asd;
struct ipu6_isys_csi2_config csi2;
};
diff --git a/drivers/media/pci/intel/ipu6/ipu6.c b/drivers/media/pci/intel/ipu6/ipu6.c
index 7d5f465a9e71..0c63873baa5c 100644
--- a/drivers/media/pci/intel/ipu6/ipu6.c
+++ b/drivers/media/pci/intel/ipu6/ipu6.c
@@ -14,6 +14,8 @@
#include <linux/pm_runtime.h>
#include <linux/timer.h>
+#include <media/ipu-bridge.h>
+
#include "ipu6.h"
#include "ipu6-bus.h"
#include "ipu6-buttress.h"
@@ -24,7 +26,6 @@
#include "ipu6-platform-buttress-regs.h"
#include "ipu6-platform-isys-csi2-reg.h"
#include "ipu6-platform-regs.h"
-#include "../ipu-bridge.h"
#define IPU6_PCI_BAR 0
@@ -430,7 +431,7 @@ ipu6_isys_init(struct pci_dev *pdev, struct device *parent,
return ERR_PTR(-EINVAL);
}
- ret = ipu_bridge_init(pdev);
+ ret = ipu_bridge_init(&pdev->dev, ipu_bridge_parse_ssdb);
if (ret) {
dev_err_probe(&pdev->dev, ret,
"IPU6 bridge init failed\n");
--
2.41.0
