Hello, On Tuesday 24 May 2016 19:26:32 Sakari Ailus wrote: > On Tue, May 24, 2016 at 05:36:42PM +0200, Hans Verkuil wrote: > > On 05/24/2016 05:28 PM, Sakari Ailus wrote: > > > Hi Hans, > > > > > >> Should it be mentioned here that changing the video format might change > > >> the buffersize? In case the buffersize is always a multiple of the > > >> width? > > > > > > Isn't that the case in general, as with pixel formats? buffersize could > > > also be something else than a multiple of width (there's no width for > > > metadata formats) due to e.g. padding required by hardware. > > > > Well, I don't think it is obvious that the metadata buffersize depends on > > the video width. Perhaps developers who are experienced with CSI know > > this, but if you know little or nothing about CSI, then it can be > > unexpected (hey, that was the case for me!). > > > > I think it doesn't hurt to mention this relation. > > Ah, I think I misunderstood you first. > > Typically the metadata width is the same as the image data width, that's > true. And it's how the hardware works. This is still visible in the media > bus format and the solution belongs rather to how multiple streams over a > single link are supported. Let me clarify on this. In the general case there's no concept of metadata width when stored in memory. The two most common use cases for metadata store register values (or similar) information, or statistics. The former is just a byte stream in some kind of TLV (Type Length Value) format. The latter a set of values or arrays computed either on the full image or on subwindows, possibly laid out as a grid. When transported over a bus, however, metadata can sometimes have a width and height. That's the case for CSI-2, which is a line-oriented format. Metadata then need to be broken into chunks transmitted in a CSI-2 line packet, even if they don't correspond to a line of an image. The line width on the bus is just the number of bytes transmitted in a single packet, which could be chosen freely (within the range allowed by CSI-2). In practice, to simplify the implementation, the line width is chosen to be identical to the line width of the image frames that the metadata correspond to, but that's not a requirement of either CSI-2 or the metadata format itself. We thus need to expose metadata width and height on subdevs to ensure proper configuration of the transmitter and receiver, but that's not strictly mandatory on video nodes. The metadata buffer size itself doesn't depend on the width and height of the corresponding image frames. A histogram using 64 bins on 3 components will be stored exactly the same way regardless of whether it's computed on a VGA or 1080p frame. The buffer size depends on the configuration of the metadata source only, which in the case of the histogram generator in the VSP would include a control that decides whether to compute the histogram with 64 bins or 256 bins (the latter needs a 4 times larger buffer). For metadata computed on a variable number of subwindows the buffer size will depend on the number of subwindows, which will in turn be possibly influenced by the size of the image. It could make sense to use fewer subwindows to compute AF data on a VGA image than on a 4k image. That is not however a requirement, and there's no direct mapping between image size and metadata size, the number of subwindows being usually configured by userspace. I hope this clarifies the problem. Please let me know if you have additional questions or thoughts, and if you see anything that should be worded differently in the documentation (or even structures that should get new fields). > It's just that setting the image media bus format affects the metadata media > bus format. I guess that could be mentioned albeit it's hardware specific, > on some sensors metadata width is independent of the image width. Even then > this is not where I'd put it. I'd get back to the topic when documenting > how the API for multiple streams over a single link works. -- Regards, Laurent Pinchart
