On Wed, Oct 20, 2010 at 4:14 PM, Hans Verkuil <hverkuil@xxxxxxxxx> wrote: > On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: >> Hello, >> >> As I promissed I continue the development of the VideoBuf2 at Samsung >> until Pawel finds some spare time to help us. This is a third version of >> the framework. Besides the minor bugfixes here and there I've added a >> complete read() callback emulator. This emulator provides 2 types of >> read() operation - 'streaming' and 'one shot'. It is suitable to replace >> both videobuf_read_stream() and videobuf_read_one() methods from the old >> videobuf. > > One thing I never understood: what is the point of supporting 'one shot' read > mode? Why not support just streaming? Does anyone know? > > Another question: how hard is it to support write mode as well? I think > vb2 should support both. I suspect that once we have a read emulator it isn't > difficult to make a write emulator too. One thing consideration is that with this implementation we can't meet merge window for 2.6.37. How do you think? Thank you, Kyungmin Park > > A last remark: the locking has changed recently in videobuf due to the work > done on eliminating the BKL. It's probably a good idea to incorporate those > changes as well in vb2. > > Regards, > > Hans > >> >> Taking into account the size of the patches and the number of lines I've >> changed, I've decided to keep the Pawel signed-off attribute and >> authorship to correctly credit him. If this is against kernel rules, >> feel free to let me know. >> >> Best regards >> -- >> Marek Szyprowski >> Samsung Poland R&D Center >> >> >> Changes since V2: >> ================= >> - added read() emulator (see patch #5/7) >> - fixed lack of parentheses in macro definitions (caused side effects >> in some places) >> - added a separate check for VM_READ or VM_WRITE in vb2_mmap() >> - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines >> - updated vivi driver with the new read() emulator >> >> Changes since V1: >> ================= >> - removed drv_lock, added start_streaming and stop_streaming callbacks >> >> >> Here is the original Videobuf2 introduction prepared by Pawel: >> ======================================================================= >> >> These patches add a new driver framework for Video for Linux 2 driver >> - Videobuf2. >> >> Videobuf2 is intended as a replacement for videobuf, the current driver >> framework, which will be referred to as "videobuf1" for the remainder >> of this document. >> >> ================================ >> What is videobuf2? >> ================================ >> Videobuf2 is a Video for Linux 2 API-compatible driver framework for >> multimedia devices. It acts as an intermediate layer between userspace >> applications and device drivers. It also provides low-level, modular >> memory management functions for drivers. >> >> Videobuf2 eases driver development, reduces drivers' code size and aids in >> proper and consistent implementation of V4L2 API in drivers. >> >> Videobuf2 memory management backend is fully modular. This allows custom >> memory management routines for devices and platforms with non-standard >> memory management requirements to be plugged in, without changing the >> high-level buffer management functions and API. >> >> The framework provides: >> - implementations of streaming I/O V4L2 ioctls and file operations >> - high-level video buffer, video queue and state management functions >> - video buffer memory allocation and management >> >> ================================ >> Why a new framework? >> ================================ >> There have been many discussions in the V4L2 community about the feasibility >> of writing a new framework, as opposed to fixing the existing one. It has been >> agreed though that: >> - videobuf1 has major flaws and an attempt to fix it would end up in rewriting >> most of the code >> - many drivers depend on videobuf1 and since the changes would be major, >> an effort to adapt and test them all would not be realistically possible >> >> Due to the problems with videobuf most new drivers cannot use it. This leads >> to code replication and overcomplicated drivers. >> >> ================================ >> What is wrong with videobuf1? >> ================================ >> There are many problems with the current videobuf implementation. During a V4L2 >> mini-summit in Helsinki in June 2010, two presentations were delivered >> on this topic: >> - Laurent Pinchart "videobuf - the good, the bad and the ugly" >> http://linuxtv.org/downloads/presentations/summit_jun_2010/20100614-v4l2_summit-videobuf.pdf >> - Pawel Osciak "Future of the videobuf framework" >> http://linuxtv.org/downloads/presentations/summit_jun_2010/Videobuf_Helsinki_June2010.pdf >> >> These presentations highlighted many problems with videobuf. The most prominent >> include: >> >> - V4L2 API violations and wrong memory management design >> - it is impossible to pause streaming (buffers are freed on streamoff) >> - VIDIOC_REQBUFS(0) does not free memory >> - it is impossible to reallocate memory with VIDIOC_REQBUFS >> - video memory is allocated on mmap, qbuf or even on page fault, >> freed on unmap, streamoff or explicitly by drivers >> - per-buffer waitqueues >> - not extensible enough and thus not ready for new platforms and uses, >> especially considering embedded multimedia devices >> - very hard to add new memory handling routines and custom memory allocators >> - no or poor support for handling cache coherency, IOMMUs, >> - poor flexibility - only one do-it-all function for handling memory pinning, >> cache, sg-list creation, etc... >> - unused fields, code duplication, vague/inconsistent naming, obscure usage in >> some places... >> >> Many driver authors expressed their frustration with videobuf. Developers >> acknowledge its merits and would like to use it, but due mostly to its >> inflexible memory allocation schemes they are unable to do so. >> >> ================================ >> Main goals of the redesign >> ================================ >> - correct V4L2 API implementation, fixing videobuf1 problems and shortcomings >> - full separation between queue management and memory management >> - fully flexible, pluggable memory allocators and memory handling routines >> - more specialized driver callbacks, called at different points >> - support for new V4L2 API extensions, such as multi-planar video buffers >> >> ================================ >> Driver callbacks >> ================================ >> Driver callbacks have been redesigned for symmetry: >> - buf_init - called once, after memory is allocated or after a new USERPTR >> buffer is queued; can be used e.g. to pin pages, verify contiguity, set up >> IOMMU mappings, etc. >> - buf_prepare - called on each QBUF; can be used e.g. for cache sync, copying >> to bounce buffers, etc. >> - buf_finish - called on each DQBUF; can be used e.g. for cache sync, copying >> back from bounce buffers, etc. >> - buf_cleanup - called before freeing/releasing memory; can be used e.g. for >> unmapping memory, etc. >> >> The remaining driver callbacks have been slightly redesigned: >> - queue_negotiate - now incorporates multi-planar extensions; drivers return >> required number of buffers and planes per buffer >> - plane_setup - drivers return plane sizes >> Those two callbacks replace the old buf_setup. >> >> - buf_queue - basically stays the same >> >> ================================ >> Memory allocators and handling >> ================================ >> Memory handling has been designed to allow more customization than in the >> original videobuf. For this memory allocation ops have been slightly redesigned, >> and have become fully replaceable and an allocator context struct have been >> introduced. >> >> Allocator context is intended to provide memory operations to videobuf and also >> for storing allocator private data, if required, although simpler allocators >> do not have to use this feature. Private data can be added by embedding the >> context struct inside their own structures: >> >> struct vb2_alloc_ctx { >> const struct vb2_mem_ops *mem_ops; >> }; >> >> struct vb2_foo_alloc_conf { >> struct vb2_alloc_ctx alloc_ctx; >> /* Allocator private data here */ >> }; >> >> Moreover, a buffer context structure concept has been introduced. Allocators >> return their own, custom, per-buffer structures on every allocation. This >> structure is then used as a "cookie" and passed to other memory handling >> methods called for its corresponding buffer. >> >> Memory operations, stored in the allocator context, can be replaced if >> needed by drivers on a per-function basis and functions from other allocators >> or drivers can be reused as well. A full list with documentation can be found >> in the videobuf2-core.h file. >> >> It is also possible, although not required, to assign different contexts per >> plane. This may be useful for drivers that need to use different memory types >> for different planes. An example may be a driver that stores video data in the >> first plane, which has to be allocated from a device-accessible memory area, >> and metadata in the second plane, which does not have to be stored in >> a device-accessible memory. >> >> An good example of integrating a more advanced allocator, the recently discussed >> on this list CMA (contiguous memory allocator), can be found in videobuf2-cma.*. >> >> ================================ >> Other changes >> ================================ >> The changes described above are the main changes in videobuf2. Most of the core >> API has remained the same or very similar, although its implementation has been >> fully rewritten. Some more visible changes include: >> >> - Memory is now properly allocated on REQBUFS and can be freed and reallocated >> there as well. >> - It is now possible to pause and resume streaming with streamon/streamoff, >> without freeing the buffers. >> - V4L2 API-related, userspace-visible metadata, such as inputs, timestamps, etc. >> are no longer stored in videobuf buffer structure, but in an actual >> v4l2_buffer struct (idea borrowed from Laurent Pinchart). I felt that driver >> authors would prefer to use V4L2 API-based structures of videobuf custom >> structures where possible. It also eases copying v4l2_buffer-related data >> from/to userspace. >> - Buffers do not include waitqueues anymore. One, global, per-queue waitqueue >> for done buffers has been introduced instead. Per-buffer waitqueues were not >> very useful and introduced additional complications in code. >> With this, drivers have gained the ability of dequeuing buffers out-of-order >> as well. >> - Buffer states are not handled jointly by both videobuf and driver anymore, >> I felt it was not required and confusing for driver authors >> - Some fields that were less useful have been removed and naming of others >> have been changed to better reflect their function. >> - Other then reqbufs, ioctl implementations have remained almost the same >> and behave in the same way, >> >> >> Please see documentation in videobuf2-core.c and videobuf2-core.h for more >> details and the patch porting vivi to videobuf2 for how to port existing >> drivers to videobuf2. >> >> This is a preliminary version intended for review, but has already been tested >> for multiple drivers and different memory handling implementations. DMA-SG >> implementation is not included yet. >> >> The CMA allocator patches are attached for reference, to show how a more >> complicated allocator can be integrated into the framework. >> >> Any comments will be very much appreciated! >> >> Best regards, >> Pawel Osciak >> Linux Platform Group >> Samsung Poland R&D Center >> >> ======================================================================= >> >> Patch summary: >> >> Pawel Osciak (6): >> v4l: add videobuf2 Video for Linux 2 driver framework >> v4l: videobuf2: add generic memory handling routines >> v4l: videobuf2: add vmalloc allocator >> v4l: videobuf2: add DMA coherent allocator >> v4l: vivi: port to videobuf2 >> v4l: videobuf2: add CMA allocator >> >> Marek Szyprowski (1): >> v4l: videobuf2: add read() emulator >> >> drivers/media/video/Kconfig | 23 +- >> drivers/media/video/Makefile | 6 + >> drivers/media/video/videobuf2-cma.c | 250 ++++ >> drivers/media/video/videobuf2-core.c | 1726 ++++++++++++++++++++++++++ >> drivers/media/video/videobuf2-dma-coherent.c | 208 ++++ >> drivers/media/video/videobuf2-memops.c | 199 +++ >> drivers/media/video/videobuf2-vmalloc.c | 177 +++ >> drivers/media/video/vivi.c | 348 +++--- >> include/media/videobuf2-cma.h | 25 + >> include/media/videobuf2-core.h | 392 ++++++ >> include/media/videobuf2-dma-coherent.h | 21 + >> include/media/videobuf2-memops.h | 31 + >> include/media/videobuf2-vmalloc.h | 16 + >> 13 files changed, 3258 insertions(+), 164 deletions(-) >> create mode 100644 drivers/media/video/videobuf2-cma.c >> create mode 100644 drivers/media/video/videobuf2-core.c >> create mode 100644 drivers/media/video/videobuf2-dma-coherent.c >> create mode 100644 drivers/media/video/videobuf2-memops.c >> create mode 100644 drivers/media/video/videobuf2-vmalloc.c >> create mode 100644 include/media/videobuf2-cma.h >> create mode 100644 include/media/videobuf2-core.h >> create mode 100644 include/media/videobuf2-dma-coherent.h >> create mode 100644 include/media/videobuf2-memops.h >> create mode 100644 include/media/videobuf2-vmalloc.h >> >> > > -- > Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco > -- > To unsubscribe from this list: send the line "unsubscribe linux-media" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
