Hello Karthik, On Tuesday 22 December 2015 05:30:52 karthik poduval wrote: > I have been wanting to share these thoughts for the group but was > waiting for the right time which I think is now since Guennadi brought > up this discussion. > > For the Amazon Fire phone 4 corner camera, here is how we passed > metadata from driver to application (which was a CV client requiring > per frame metadata). > > We took an unused field in struct v4l2_buffer (__u32 reserved in this > case) and used it to pass in a pointer to a user space metadata object > (i.e. struct app_metadata) to the driver via the VIDIOC_DQBUF ioctl > call. > > struct v4l2_buffer for reference. > http://lxr.free-electrons.com/source/include/uapi/linux/videodev2.h#L836 > > The driver copied its local copy of the metadata object to the > userspace metadata object using the copy_to_user primitive offered by > the kernel. > > Here is how we handled the metadata in the driver code. > https://github.com/Fire-Phone/android_kernel_amazon_kodiak/blob/master/drive > rs/media/platform/msm/camera_v2/camera/camera.c#L235 > > This was done before HAL V3 was available. With HAL V3, the metadata > object can be the HAL v3 metadata buffer. Non Android devices can use > custom metadata format (like the one we used). > > With this approach, the metadata always accompanies the frame data as > it's available along with the frame buffer inside struct v4l2_buffer > from the VIDIOC_DQBUF ioctl call. > > If the community likes this idea, the v4l2_buffer can now be > officially modified to contain a pointer to user space metadata object > v4l2_buffer.metadata and then metadata format and size can be agreed > upon between application and driver. > Thoughts ? I see several issues with that approach. The first one is that the meta-data buffer is only passed at DQBUF time. Drivers thus need to copy data using the CPU instead of capturing meta-data directly to the buffer through DMA. If the meta-data size is small the performance impact can be negligible, but that might not be true in the general case. A second issue is that the approach isn't generic enough in my opinion. If we want to attach additional data buffers to a v4l2_buffer I agree with Sakari that we should design a multi-part buffer API in order to not limit the implementation to meta-data, but support other kind of information such as statistics for instance. Finally, it might be beneficial in some use cases to pass meta-data to userspace before the end of the frame (assuming meta-data is available earlier of course). That's certainly true for statistics, use cases for meta-data are less clear to me though. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
