Hi All,
On 8/31/23 23:24, Hans de Goede wrote:
> Hi Bingbu, Claus,
>
> On 8/21/23 12:07, Claus Stovgaard wrote:
>> Bingbu
>>
>> On Mon, 2023-08-21 at 08:55 +0200, Claus Stovgaard wrote:
>>> Bingbu
>>>
>>> On Mon, 2023-08-21 at 14:22 +0800, Bingbu Cao wrote:
>>>>
>>>> Claus,
>>>>
>>>>
>>>> On 8/21/23 11:14 AM, Bingbu Cao wrote:
>>>>
>>>>
>>>> I see that the ivsc driver has not been in master branch. Before
>>>> that,
>>>> could you try several hack to check whether camera can work on
>>>> master?
>>>>
>>>> https://github.com/bingbucao/linux/commits/ipu_dev
>>>>
>>>> 7ebff51284d9 media: ov01a10: hack ivsc to make camera can work
>>>> 01cc9f3d1b61 i2c: ljca: Call acpi_dev_clear_dependencies()
>>>> 92e5d122e105 vsc: Defer firmware loading to avoid long probing time
>>>> 5f5d5f0df06b driver: ivsc: add intel ivsc driver
>>>> 0f4819dec533 Revert "gpio: Add support for Intel LJCA USB GPIO
>>>> driver"
>>>
>>> Thanks for your quick reply.
>>>
>>> I was missing understanding of ivsc when I wrote the mail yesterday.
>>> Got some basic understanding yesterday after I wrote, and big thanks
>>> for confirming it, and also thanks for your ipu_dev branch. Has just
>>> cloned it, and is building as I write.
>>>
>>> Just fyi, I was trying to hack something together yesterday, and got
>>> further, but not yet working.
>>>
>>> My hack was to combine the out-of-tree ivsc drivers and firmware from
>>>
>>> * https://github.com/intel/ivsc-firmware.git
>>> * https://github.com/intel/ivsc-driver.git
>>>
>>> Though noticed that I need some changes to the sensor driver so was
>>> also building all the drivers from ipu6-drivers (with minor changes
>>> to
>>> get_pages) as out-of-tree modules.
>>>
>>> * https://github.com/intel/ipu6-drivers.git ;
>>>
>>> Here I used everything beside media/pci/*.ko files. I could see the
>>> sensor and got further, but was missing the last.
>>>
>>> Looking forward to try your branch. Looks much cleaner, and would be
>>> nice to get working :)
>>>
>>
>> I got it to work on Dell XPS 9320.
>> With some minor changes compared to your guide in Documentation/admin-
>> guide/media/ipu6-isys.rst
>>
>> [root@xps-1 ]# uname -a
>> Linux xps-1 6.5.0-rc7-g7ebff51284d9 #1 SMP PREEMPT_DYNAMIC Mon Aug 21
>> 09:02:20 CEST 2023 x86_64 GNU/Linux
>>
>> [root@xps-1 ]# media-ctl -d /dev/media0 -p | tail -n10
>>
>> - entity 2149: ov01a10 16-0036 (1 pad, 1 link)
>> type V4L2 subdev subtype Sensor flags 0
>> device node name /dev/v4l-subdev4
>> pad0: Source
>> [fmt:SBGGR10_1X10/1280x800 field:none colorspace:raw
>> crop.bounds:(0,0)/1296x816
>> crop:(8,8)/1280x800]
>> -> "Intel IPU6 CSI2 2":0 []
>>
>> So i2c is 16-0036 - and we use it for setup like your guide.
>>
>> export MDEV=/dev/media0
>>
>> media-ctl -d $MDEV -l "\"ov01a10 17-0036\":0 -> \"Intel IPU6 CSI2
>> 2\":0[1]"
>>
>> media-ctl -d $MDEV -V "\"ov01a10 17-0036\":0 [fmt:SBGGR10/1280x800]"
>> media-ctl -d $MDEV -V "\"Intel IPU6 CSI2 2\":0 [fmt:SBGGR10/1280x800]"
>> media-ctl -d $MDEV -V "\"Intel IPU6 CSI2 2\":1 [fmt:SBGGR10/1280x800]"
>>
>> media-ctl -d $MDEV -l "\"ov01a10 17-0036\":0 -> \"Intel IPU6 CSI2
>> 2\":0[1]"
>> media-ctl -d $MDEV -l "\"Intel IPU6 CSI2 2\":1 ->\"Intel IPU6 ISYS
>> Capture 0\":0[5]"
>>
>> Though yavta does not work in the way as described in the guide.
>>
>> [root@xps-1 ]# yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --
>> file=/tmp/frame-#.bin -f SBGGR10 /dev/video0
>> Device /dev/video0 opened.
>> Device `ipu6' on `PCI:0000:00:05.0' (driver 'isys') supports video,
>> capture, with mplanes.
>> Video format set: SBGGR10 (30314742) 1280x800 field none, 1 planes:
>> * Stride 2560, buffer size 2050560
>> Video format: SBGGR10 (30314742) 1280x800 field none, 1 planes:
>> * Stride 2560, buffer size 2050560
>> Unable to request buffers: Invalid argument (22).
>>
>>
>> So I changed to use v4l2-ctl
>>
>> [root@xps-1 ]# v4l2-ctl -d /dev/video0 --set-fmt-video
>> width=1280,height=800,pixelformat=BG10 --stream-mmap --stream-count=1 -
>> -stream-to=frame.bin
>>
>> With this I created raw data in BG10 format, and later used a small
>> python script with numpy and opencv to look at the data.
>>
>> #!/usr/bin/env python3
>> # Demosaicing Bayer Raw image
>>
>> import cv2
>> import numpy as np
>>
>> width = 1280
>> height = 800
>>
>> with open("frame.bin", "rb") as rawimg:
>> # Read the bayer data
>> data = np.fromfile(rawimg, np.uint16, width * height)
>> bayer = np.reshape(data, (height, width))
>>
>> # Just a offset gain to be able to see something
>> for x in range(0, len(bayer)):
>> for y in range(0, len(bayer[0])):
>> bayer[x, y] = (bayer[x,y] << 8)
>>
>> rgb = cv2.cvtColor(bayer, cv2.COLOR_BayerBGGR2RGB)
>>
>> cv2.imshow('rgb', rgb)
>> cv2.waitKey()
>> cv2.destroyAllWindows()
>>
>>
>> Thanks for the help, and now we know what is needed to make it work on
>> top of yesterdays rc7
>
>
> Bingbu, thank you for the series. Claus, thank you for the python
> test-script.
>
> I've just given this a test-run on top of a recent checkout
> of media-staging/master, so on top of the drivers/media
> changes headed for 6.6 .
>
> And with the attached changes + the ov2740 changes from
> the github ipu6-drevers repo I got this working on
> a lenovo thinkpad x1 yoga with an ov2740 driver.
>
> I've attached the necessary changes to adjust the new ipu6
> code for the v4l2-async changes which are queued up for
> kernel 6.6 .
Attached is one more patch which fixes an oops when
using lockdep.
With both patches applied this is:
Tested-by: Hans de Goede <hdegoede@xxxxxxxxxx>
Regards,
Hans
From 80f82bc3b45fc20e152a9a3dceeb08d5773e50ad Mon Sep 17 00:00:00 2001
From: Hans de Goede <hdegoede@xxxxxxxxxx>
Date: Sat, 2 Sep 2023 16:34:09 +0200
Subject: [PATCH] media: ipu6-bus: Set pm-domain later
Move the dev_pm_domain_set() call to after the auxiliary_device_init()
call so that auxdev->dev.power.lock has been initialized.
This fixes the following lockdep warning / backtrace:
[ 61.381538] INFO: trying to register non-static key.
[ 61.381540] The code is fine but needs lockdep annotation, or maybe
[ 61.381541] you didn't initialize this object before use?
[ 61.381542] turning off the locking correctness validator.
[ 61.381544] CPU: 8 PID: 1837 Comm: (udev-worker) Tainted: G E 6.5.0+ #28
[ 61.381547] Hardware name: LENOVO 21HQSIT025/21HQSIT025, BIOS N3XET37W (1.12 ) 04/18/2023
[ 61.381548] Call Trace:
[ 61.381550] <TASK>
[ 61.381552] dump_stack_lvl+0x57/0x90
[ 61.381558] register_lock_class+0x480/0x490
[ 61.381563] ? __lock_acquire+0x405/0x2190
[ 61.381568] __lock_acquire+0x76/0x2190
[ 61.381572] lock_acquire+0xc4/0x290
[ 61.381575] ? device_pm_check_callbacks+0x1d/0x100
[ 61.381578] _raw_spin_lock_irqsave+0x47/0x70
[ 61.381581] ? device_pm_check_callbacks+0x1d/0x100
[ 61.381582] device_pm_check_callbacks+0x1d/0x100
[ 61.381585] ipu6_bus_initialize_device+0xfa/0x160 [intel_ipu6]
[ 61.381591] ipu6_configure_spc+0xbe5/0x13a0 [intel_ipu6]
<snip>
Signed-off-by: Hans de Goede <hdegoede@xxxxxxxxxx>
---
drivers/media/pci/intel/ipu6/ipu6-bus.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/media/pci/intel/ipu6/ipu6-bus.c b/drivers/media/pci/intel/ipu6/ipu6-bus.c
index 0e58accf0654..bb9cb2c67a1e 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-bus.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-bus.c
@@ -110,7 +110,6 @@ ipu6_bus_initialize_device(struct pci_dev *pdev, struct device *parent,
auxdev->dev.dma_mask = &adev->dma_mask;
auxdev->dev.dma_parms = pdev->dev.dma_parms;
auxdev->dev.coherent_dma_mask = adev->dma_mask;
- dev_pm_domain_set(&auxdev->dev, &ipu6_bus_pm_domain);
ret = auxiliary_device_init(auxdev);
if (ret < 0) {
@@ -120,6 +119,8 @@ ipu6_bus_initialize_device(struct pci_dev *pdev, struct device *parent,
return ERR_PTR(ret);
}
+ dev_pm_domain_set(&auxdev->dev, &ipu6_bus_pm_domain);
+
pm_runtime_forbid(&adev->auxdev.dev);
pm_runtime_enable(&adev->auxdev.dev);
--
2.41.0
