Hi Martin, On Tue, Feb 23, 2021 at 10:04:44AM +0100, Martin Kepplinger wrote: > On 22.12.20 16:55, Martin Kepplinger wrote: > > On 22.12.20 16:35, Laurent Pinchart wrote: > >> On Tue, Dec 22, 2020 at 03:06:28PM +0100, Martin Kepplinger wrote: > >>> On 10.12.20 17:04, Laurent Pinchart wrote: > >>>> On Thu, Dec 10, 2020 at 02:12:30PM +0100, Martin Kepplinger wrote: > >>>>> On 10.12.20 13:17, Laurent Pinchart wrote: > >>>>>> On Thu, Dec 10, 2020 at 11:24:00AM +0200, Laurent Pinchart wrote: > >>>>>>> On Thu, Dec 10, 2020 at 09:17:48AM +0100, Martin Kepplinger wrote: > >>>>>>>> hi, > >>>>>>>> > >>>>>>>> TL;DR: did you use the NXP "mx6s_capture" csi bridge driver with > >>>>>>>> other > >>>>>>>> cameras? > >>>>>>> > >>>>>>> I've recently worked on camera support for i.MX8MM (whose camera IP > >>>>>>> cores are, if not identical, very similar to the i.MX8MQ's). The > >>>>>>> least I > >>>>>>> can say is that it was painful :-( > >>>>>>> > >>>>>>> I'm using an MT9M114 sensor, which can produce RAW8, RAW10 and > >>>>>>> YUV and > >>>>>>> has a CSI-2 interface. My first use case is to capture RAW10, which > >>>>>>> isn't supported by the mx6s_capture driver. > >>>>> > >>>>> so did you successfully use the NXP mx6s_capture driver with that > >>>>> sensor too? > >>>> > >>>> I haven't tried. The mx6s_capture driver and the mainline driver share a > >>>> similar architecture, as they came from the same code base. They have > >>>> diverged, with the mainline driver receiving bug fixes and new > >>>> features, and my large no-yet-upstreamed patch series adds lots of fixes > >>>> that are required for the mt9m114 driver to work with the driver. For > >>>> that reason, I've considered that porting the mt9m114 driver to the NXP > >>>> mx6s_capture, and fixing the same issues in the mx6s_capture driver than > >>>> were present in the mainline driver, would require lots of time for > >>>> likely very little gain. > >>>> > >>>> That being said, I've compared the two drivers, and I haven't seen > >>>> anything in mx6s_capture that would address the specific issues I'm > >>>> facing. I've ordered yesterday an i.MX8MM EVK with an OV5640 camera > >>>> module, and I will test those with the mainline driver. If they don't > >>>> work, and assuming the NXP kernel driver works on that platform, I'll > >>>> have two code bases to compare. > >>>> > >>>>>>>> I try to use a CSI2 camera (hi846 I'm writing a driver for) on imx8mq: > >>>>>>>> Using NXP's CSI bridge driver > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/mx6s_capture.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> as well as the CSI driver itself: > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/imx8/mxc-mipi-csi2_yav.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> works fine when using the ov5640 camera with this driver: > >>>>>>>> https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_mipi_v2.c?h=imx_5.4.0_8dxlphantom_er > >>>>>>>> > >>>>>>>> > >>>>>>>> (I realize there is a CSI bridge driver in staging, but that need more > >>>>>>>> work to be actually used. Of course after this the goal is to fix and > >>>>>>>> use it; and mainline a CSI phy driver too.) > >>>>>>> > >>>>>>> I have lots of patches for this driver, which I've developed on i.MX7D > >>>>>>> for a separate project. I'd like to mainline them, but this is blocked > >>>>>>> by one last issue that I haven't been able to solve yet. In a nutshell, > >>>>>>> the CSI writes two consecutive frames in each buffer, overflowing the > >>>>>>> allocated memory. The registers that control the buffer size seem to be > >>>>>>> programmed correctly as far as I can tell. I've reported this issue to > >>>>>>> NXP but haven't received any feedback yet. > >>>>> > >>>>> that's different from where I am. So you don't get any interrupt (EOF or > >>>>> other) when only *one* frame is written into one buffer? > >>>> > >>>> I don't get any FB1_DMA_DONE or FB2_DMA_DONE interrupt at least. I > >>>> haven't checked other interrupts. I thus get half of the frame rate. If > >>>> I configure the CSI_CSIIMAG_PARA register with half of the actual image > >>>> height, buffers then contain a single image and the frame rate goes to > >>>> the expected value. That gives me a workaround, but without > >>>> understanding the root cause, I'm worried that just halving the height > >>>> would introduce other breakages. > >>> > >>> Hi Laurent, > >>> > >>> Thanks a lot, exchanging things this way alone is much appreciated. I'll > >>> give you detailed feedback on the staging drivers later - I tried > >>> running them, they probe but still a (gstreamer) stream would not start. > >>> But most all of your addition make a lot of sense. > >>> > >>> Just so you know, to emphasize that once more: I'm still on the nxp > >>> drivers (yes, they are very similar) just because the ov56 cam works > >>> with them over here. > >> > >> I've recently bought an i.MX8MM EVK, with the OV5647 camera module, and > >> when I try to capture images, with the NXP kernel, I just don't get any. > >> *sigh* I'll have to debug that. > > > > oh you have it already. very good. > > > > I guess the nxp kernel directly *should* work, but I use a kernel based > > on v5.9.X and include the drivers like so: > > https://source.puri.sm/Librem5/linux-next/-/commit/ed8f567432d776d21cb7160c03e3272826a5f316 > > > > https://source.puri.sm/Librem5/linux-next/-/commit/8d93a2cfd485d404741560dcb2f22da548f54ebc > > > >>> The main problem I have is that, as soon as I try > >>> to use a different pixelformat than V4L2_PIX_FMT_YUYV that basically > >>> nothing works anymore. userspace stops to "be interested" pretty soon. > >>> > >>> If I do (wrongly) use that pixelformat, I do get (only) one frame (dma) > >>> completed, the usual way. > >>> > >>> And when I break the ov56 by using a wrong pixelformat in mx6s_capture > >>> (nxp bridge driver), I get equally get only one frame (dma) completed > >>> there. > >>> > >>> How can I make this thing use a different pixelformat (by setting the > >>> media bus format in the sensor driver)? > >> > >> All the components have to support the format you want to capture. This > >> includes the CSI driver, the MIPI_CSI driver, and the OV5647 driver. The > >> pixel format only matters in the CSI driver, for the other two it's the > >> media bus format that matters. To capture raw bayer data, you'll first > >> need to extend the OV5640 driver to support the corresponding media bus > >> format (and of course configure the sensor to output that format), then > >> the MIPI_CSI driver to support that media bus format as well, and > >> finally the CSI driver. > > > > ok, the sensor I work with outputs raw bayer only, and that sounds like > > I should pay more attention to the mipi csi driver. > > > > that links to my experiments I occasionally update: > > https://source.puri.sm/martin.kepplinger/linux-next/-/commits/hi846 > > > >>> thanks again for sharing your code early. have a nice rest of the year, > > hi Laurent, how are you? Am I right in that you tried to run the ov5640 > mainline driver with the CSI bridge staging driver on imx8mq? Which mipi > driver would you then use? I've run it with the staging driver on i.MX8MP, not i.MX8MQ. The MQ seems to have the same CSI bridge, but a different CSI-2 receiver (and PHY). It would be great if you could contribute a driver for that :-) > Did you have progress in that area and in resolving things in order to > push some of your changes out? I've sent a large patch series recently, if you could review it it would speed up integration ;-) > I'm still struggling with the different sensor I want to use on imx8mq > (hi846) and I still use the nxp bsp drivers for mipi and csi bridge: > > I was able to add and fix a few things and get some kind of image at > least, but there's still definitely some sizes or timings wrong (in > every resolution mode I think). 1280x720 shows it clearly as the image > looks "almost" normal only when I resize it to 916 pixel width. Also, it > "shifts" with later frames. For a few images of this, see > > https://source.puri.sm/Librem5/linux-next/-/issues/43#note_143875 > > I just wanted to describe the above just in case you've seen something > like that during your experiments with sensors and can point me to > places to look at. I do think that the SoC side should be ok now (but > still now 100% sure), but many sensors are similar and maybe you have > ideas there too. I've seen similar (but not identical) issues that were due to incorrect Ths_settle values in the D-PHY. You could start by trying different values there. This seems to be controlled on i.MX8MQ by CSI2_1_S_PRG_RXHS_SETTLE in IOMUXC_GPR_GPR34 or CSI2_2_S_PRG_RXHS_SETTLE in IOMUXC_GPR_GPR41, but the value of the fields are not properly documented :-S The i.MX8MM suffered from the same issue, but a table with register values for different clock frequencies was posted by NXP on their support forum. Maybe the same information could be available there for i.M8XMQ ? > anyways, thanks for looking at this and as soon as I have a satisfying > image with my current setup, I'll switch to the staging csi driver. But > there's no driver for the mipi ("phy") in staging for imx8mq, right? -- Regards, Laurent Pinchart
