Hi Sylwester, On Wednesday 01 February 2012 12:41:44 Sylwester Nawrocki wrote: > On 02/01/2012 11:00 AM, Sakari Ailus wrote: > >> Some camera sensors generate data formats that cannot be described using > >> current convention of the media bus pixel code naming. > >> > >> For instance, interleaved JPEG data and raw VYUY. Moreover interleaving > >> is rather vendor specific, IOW I imagine there might be many ways of how > >> the interleaving algorithm is designed. > > > > Is that truly interleaved, or is that e.g. first yuv and then jpeg? > > Interleaving the two sounds quite strange to me. > > It's truly interleaved. There might be some chance for yuv/jpeg one after > the other, but the interleaved format needs to be supported. > > >> I'm wondering how to handle this. For sure such an image format will > >> need a new vendor-specific fourcc. Should we have also vendor specific > >> media bus code ? > >> > >> I would like to avoid vendor specific media bus codes as much as > >> possible. For instance defining something like > >> > >> V4L2_MBUS_FMT_VYUY_JPEG_1X8 > >> > >> for interleaved VYUY and JPEG data might do, except it doesn't tell > >> anything about how the data is interleaved. > >> > >> So maybe we could add some code describing interleaving (xxxx) > >> > >> V4L2_MBUS_FMT_xxxx_VYUY_JPEG_1X8 > >> > >> or just the sensor name instead ? > > > > If that format is truly vendor specific, I think a vendor or sensor > > specific media bus code / 4cc would be the way to go. On the other hand, > > you must be prepared to handle these formats in your ISP driver, too. > > Yes, I don't see an issue in adding a support for a new format in > ISP/bridge driver, it needs to know anyway e.g. what MIPI-CSI data type > corresponds to the data from sensor. > > > I'd guess that all the ISP would do to such formats is to write them to > > memory since I don't see much use for either in ISPs --- both typically > > are output of the ISP. > > Yep, correct. In fact in those cases the sensor has complicated ISP built > in, so everything a bridge have to do is to pass data over to user space. > > Also non-image data might need to be passed to user space as well. > > > I think we will need to consider use cases where the sensors produce > > other data than just the plain image: I've heard of a sensor producing > > both (consecutively, I understand) and there are sensors that produce > > metadata as well. For those, we need to specify the format of the full > > frame, not just the image data part of it --- which we have called > > "frame" at least up to this point. > > Yes, moreover such formats partly determine data layout in memory, rather > than really just a format on a video bus. > > > If the case is that the ISP needs this kind of information from the > > sensor driver to be able to handle this kind of data, i.e. to write the > > JPEG and YUV to separate memory locations, I'm proposing to start > > working on this > > It's not the case here, it would involve unnecessary copying in kernel > space. Even in case of whole consequitive data planes contiguous buffer is > needed. And it's not easy to split because the border between the data > planes cannot be arbitrarily aligned. > > > now rather than creating a single hardware-specific solution. > > Yes, I'm attempting rather generic approach, even just for that reason that > there are multiple Samsung sensors that output hybrid data. I've seen Sony > sensor doing that as well. Do all those sensors interleave the data in the same way ? This sounds quite hackish and vendor-specific to me, I'm not sure if we should try to generalize that. Maybe vendor-specific media bus format codes would be the way to go. I don't expect ISPs to understand the format, they will likely be configured in pass-through mode. Instead of adding explicit support for all those weird formats to all ISP drivers, it might make sense to add a "binary blob" media bus code to be used by the ISP. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
