Hi, On 2/16/23 22:34, Laurent Pinchart wrote: > Hi Hans, > > On Thu, Feb 16, 2023 at 04:47:51PM +0100, Hans de Goede wrote: >> On 2/16/23 15:48, Laurent Pinchart wrote: >>> On Thu, Feb 16, 2023 at 03:20:08PM +0100, Hans de Goede wrote: >>>> Hi All, >>>> >>>> I have been looking into moving the sensor registration for atomisp2 >>>> over to v4l2-aysnc similar to how >>>> drivers/media/pci/intel/ipu3/cio2-bridge.c does things. >>>> >>>> Together with some other smaller changes this should allow the atomisp >>>> code use standard sensor drivers instead of having their own fork of >>>> these drivers. >>>> >>>> While looking into this I realized that the current architecture of >>>> the atomisp2 code where it registers 8 /dev/video# nodes + many >>>> v4l2-subdevs is getting in the way of doing this. At a minimum the >>>> current convoluted media-ctl graph makes it harder then necessary to >>>> make this change. >>>> >>>> So this makes me realize that it probably is time to make some changes >>>> to the atomisp-code to remove a bunch of somewhat obscure (and >>>> untested / unused) features. I have been thinking about removing these >>>> for a long time already since they also get in the way of a bunch of >>>> other things like allowing the /dev/video# nodes to be opened multiple >>>> times. >>>> >>>> So my plan is to reduce the feature set to make atomisp work as more >>>> or less a standard webcam (with front/back sensors) which is how most >>>> hw is using it and also is how all our (my) current testing uses it. >>>> >>>> This means reducing the graph to a single /dev/video0 output node + 2 >>>> subdevs for the sensors I might put one more node in the graph for >>>> selecting between the 3 CSI ports, or those could be 3 possible >>>> sources for /dev/video0. >>> >>> Could you briefly summarize the hardware architecture, and in particular >>> what building blocks are present, and how they're connected ? That will >>> help with the discussion. >> >> I can try, but it is complicated. The atomisp appears to mainly be >> some coprocessor thing (with I guess some hw-accel blocks on the side) >> the way it works from the driver's pov is that the firmware file really >> contains a a whole bunch of different binaries to run on the co-processor, >> with a table describing the binaries including supported input and >> output formats. >> >> Each binary represents a complete camera pipeline, going from >> directly reading from the CSI receiver on one end to DMA-ing >> the fully finished ready to consume buffers in the requested >> destination fmt on the other end. The driver picks a binary >> based on the requested input + output formats and then uploads >> + starts that. >> >> So basically it is one big black box, where we hookup a >> sensor on one side and then on the other end say give my YUYV >> or YU12, or ... There are of course a whole bunch of >> processing parameters we can set like lens shading correction >> tables (format unknown), etc. But basically it is still >> just a black box. >> >> So from a mediactl pov as I see it the whole thing is a single >> node in the graph. > > Do you mean a single entity for the ISP ? I'd go for > > sensor subdev -> CSI-2 RX subdev -> ISP subdev -> video device > > Is that what you meant ? Yes although I'm not sure having "CSI-2 RX subdev" in there as a separate node makes much sense given how blackbox-y the entire working of the pipeline is. At least I'm not aware of any way to e.g. skip the ISP and get raw bayer frames directly out of the CSI-2 receiver. Regards, Hans
