> Hello, > > we are currently working on a chip that requires a separate buffer for > each > plane in a frame. Our hardware requires, those plane buffers not to be > placed > immediately one after another. > > There is no support for such buffers in V4L2 yet and we would like to > propose > a solution for this problem. > > > Purpose and requirements > ========================= > Currently, the v4l2_buffer struct supports only contiguous memory buffers, > i.e. one frame has to fit in one, contiguous physical buffer. A driver > receives and passes back to the userspace (e.g. when mmap()ing) only one > pointer (offset). > > Our hardware requires two physically separate buffers for Y and CbCr > components, > which must be placed in two different memory banks. > A similar problem was also expressed by Jun Nie in a recent discussion on > this > list: > http://article.gmane.org/gmane.linux.drivers.video-input-infrastructure/10462. > > That proposal included a hardcoded 3-plane format. > There also was a requirement for per-plane stride as well and although we > have > not included it in this proposal, it could be easily incorporated into > this > design (see comments below). > > We would like to add support for a more general type of buffers: n-plane > buffers. > No changes should be made to break the existing API. > > > > Proposed extensions > ==================== > The proposed extensions to the framework are as follows: > > 1. Add two new memory types: > > enum v4l2_memory { > V4L2_MEMORY_MMAP = 1, > V4L2_MEMORY_USERPTR = 2, > V4L2_MEMORY_OVERLAY = 3, > + V4L2_MEMORY_MULTI_USERPTR = 4, > + V4L2_MEMORY_MULTI_MMAP = 5, > }; > > The new types would be used to identify multi-planar buffers. > > > 2. Modify the buffer struct (no change to size): > > struct v4l2_buffer { > /* ... */ > union { > __u32 offset; > unsigned long userptr; > + unsigned long multi_info_ptr; > } m; > /* ... */ > }; > > > 3. The multi_info_ptr would contain a userspace pointer to a structure > further > describing the buffer: > > + struct v4l2_multiplane_info { > + __u32 count; Rather than introducing this new struct, perhaps it would be better to reuse the v4l2_buffer length field as a 'count' for multiplanes. That length field is currently unused for multiplane formats. > + struct v4l2_plane[0]; > + }; > > Where the v4l2_plane array would contain count elements: > > + struct v4l2_plane { > + __u32 parent_index; > + __u32 bytesused; > + union { > + __u32 offset; > + unsigned long userptr; > + } m; > + __u32 flags; > + __u32 length; > + __u32 reserved; Make this reserved[4]. > + }; > > parent_index - index of the parent v4l2_buffer Why do we need this index? > > offset, userptr, bytesused, length - same as in v4l2_buffer struct but for > current frame > > flags - one flag currently: V4L2_PLANE_FLAG_MAPPED > (or reuse V4L2_BUF_FLAG_MAPPED for that) Isn't this just a copy of the v4l2_buffer flags? Why do we need it again? > > A stride field could also be added if there is a need for one. > > > > How this would work > =================== > > ------------------------------------------------------------------------------- > 1. Formats > ------------------------------------------------------------------------------- > No need to change the format API, although new formats for such buffers > may be > needed and added, as required. > > > ------------------------------------------------------------------------------- > 2. Requesting, querying and mapping buffers > ------------------------------------------------------------------------------- > No changes to existing applications/drivers required. > > A driver (and the videobuffer framework components) willing to support > multi-plane buffers would have to be made aware of the new memory types: > > > VIDIOC_REQBUFS: > --------------- > > - MULTI_MMAP: > > * application: pass the new memory type and count of multi-plane buffers > (not plane count) normally > > * driver: fills in count as usual, being the number of actually > allocated > buffers (i.e. 1 for each multi-plane buffer, not each plane) > > - MULTI_USERPTR: > * no changes > > > VIDIOC_QUERYBUFS: > ----------------- > - MULTI_MMAP: > > * application: pass a v4l2_buffer struct as usual, but with the new memory > type and a userspace pointer (in multi_info_ptr) to an instance of > v4l2_multiplane_info structure. The structure and the embedded > v4l2_plane[] array has to be preallocated in userspace and have count > set > to the required number of planes. > > * driver fills offset fields in each v4l2_plane struct, analogically to > offsets in "normal" v4l2_buffers. > > > - MULTI_USERPTR: > n/a > > > > mmap() > ----------------- > Basically just like in normal buffer case, but with planes instead of > buffers > and one mmap() call per each plane. > > - application calls mmap count times (one for each plane), passing the > offsets > provided in v4l2_plane structs > > - there is no need for those calls to be in any particular order. > > > - driver (videobuffer framework) should store an array of planes > internally - just like it does with v4l2_buffers - and match offsets in > that array to those provided in mmap. > > - a plane gets marked as mapped (V4L2_PLANE_FLAG_MAPPED flag) after > a successful mmap. A buffer changes state to mapped (V4L2_BUF_FLAG_MAPPED) > only if all of its planes are mapped. > > - matching planes with buffers can be done using the parent_index member > > > ------------------------------------------------------------------------------- > 3. Queuing and dequeuing buffers, buffer usage > ------------------------------------------------------------------------------- > > No real changes have to be made to be made to the v4l2 framework, the > buffers > get queued and dequeud as usual. Only access to the new type differs, but > not much - in practice, just handle more pointers than one. > > As for the videobuffer framework, additional function(s) to acquire > addresses > to each plane will have to be added and it should be made aware of planes. > But the overall mechanism remains mostly unchanged. > > > > Comments are welcome, especially other requirements that we might not have > considered. I like this. It looks like a clean solution to this problem. Regards, Hans > > > Best regards > -- > Pawel Osciak > Linux Platform Group > Samsung Poland R&D Center > > -- > To unsubscribe from this list: send the line "unsubscribe linux-media" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
