Re: [Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension - Final spec published!

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Tom,

hmm, I wonder if it was a bug/oversight for the YUV capabilities of
this extension to not depend on OES_EGL_image_external (which
unfortunately, doesn't seem to have a GL counterpart)?

I think this currently implies that you could sample from an imported
YUV eglimg using (for example) sampler2D in GL or GLES, which I think
was not the intention.

BR,
-R

On Mon, Feb 25, 2013 at 6:54 AM, Tom Cooksey <tom.cooksey@xxxxxxx> wrote:
> Hi All,
>
> The final spec has had enum values assigned and been published on Khronos:
>
> http://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_image_dma_buf_import.txt
>
> Thanks to all who've provided input.
>
>
> Cheers,
>
> Tom
>
>
>
>> -----Original Message-----
>> From: mesa-dev-bounces+tom.cooksey=arm.com@xxxxxxxxxxxxxxxxxxxxx [mailto:mesa-dev-
>> bounces+tom.cooksey=arm.com@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Tom Cooksey
>> Sent: 04 October 2012 13:10
>> To: mesa-dev@xxxxxxxxxxxxxxxxxxxxx; linaro-mm-sig@xxxxxxxxxxxxxxxx; dri-
>> devel@xxxxxxxxxxxxxxxxxxxxx; linux-media@xxxxxxxxxxxxxxx
>> Subject: [Mesa-dev] [RFC] New dma_buf -> EGLImage EGL extension - New draft!
>>
>> Hi All,
>>
>> After receiving a fair bit of feedback (thanks!), I've updated the
>> EGL_EXT_image_dma_buf_import spec
>> and expanded it to resolve a number of the issues. Please find the latest draft below and let
>> me
>> know any additional feedback you might have, either on the lists or by private e-mail - I
>> don't mind
>> which.
>>
>> I think the only remaining issue now is if we need a mechanism whereby an application can
>> query
>> which drm_fourcc.h formats EGL supports or if just failing with EGL_BAD_MATCH when the
>> application
>> has use one EGL doesn't support is sufficient. Any thoughts?
>>
>>
>> Cheers,
>>
>> Tom
>>
>>
>> --------------------8<--------------------
>>
>>
>> Name
>>
>>     EXT_image_dma_buf_import
>>
>> Name Strings
>>
>>     EGL_EXT_image_dma_buf_import
>>
>> Contributors
>>
>>     Jesse Barker
>>     Rob Clark
>>     Tom Cooksey
>>
>> Contacts
>>
>>     Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
>>     Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
>>
>> Status
>>
>>     DRAFT
>>
>> Version
>>
>>     Version 4, October 04, 2012
>>
>> Number
>>
>>     EGL Extension ???
>>
>> Dependencies
>>
>>     EGL 1.2 is required.
>>
>>     EGL_KHR_image_base is required.
>>
>>     The EGL implementation must be running on a Linux kernel supporting the
>>     dma_buf buffer sharing mechanism.
>>
>>     This extension is written against the wording of the EGL 1.2 Specification.
>>
>> Overview
>>
>>     This extension allows creating an EGLImage from a Linux dma_buf file
>>     descriptor or multiple file descriptors in the case of multi-plane YUV
>>     images.
>>
>> New Types
>>
>>     None
>>
>> New Procedures and Functions
>>
>>     None
>>
>> New Tokens
>>
>>     Accepted by the <target> parameter of eglCreateImageKHR:
>>
>>         EGL_LINUX_DMA_BUF_EXT
>>
>>     Accepted as an attribute in the <attrib_list> parameter of
>>     eglCreateImageKHR:
>>
>>         EGL_LINUX_DRM_FOURCC_EXT
>>         EGL_DMA_BUF_PLANE0_FD_EXT
>>         EGL_DMA_BUF_PLANE0_OFFSET_EXT
>>         EGL_DMA_BUF_PLANE0_PITCH_EXT
>>         EGL_DMA_BUF_PLANE1_FD_EXT
>>         EGL_DMA_BUF_PLANE1_OFFSET_EXT
>>         EGL_DMA_BUF_PLANE1_PITCH_EXT
>>         EGL_DMA_BUF_PLANE2_FD_EXT
>>         EGL_DMA_BUF_PLANE2_OFFSET_EXT
>>         EGL_DMA_BUF_PLANE2_PITCH_EXT
>>         EGL_YUV_COLOR_SPACE_HINT_EXT
>>         EGL_SAMPLE_RANGE_HINT_EXT
>>         EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT
>>         EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT
>>
>>     Accepted as the value for the EGL_YUV_COLOR_SPACE_HINT_EXT attribute:
>>
>>         EGL_ITU_REC601_EXT
>>         EGL_ITU_REC709_EXT
>>         EGL_ITU_REC2020_EXT
>>
>>     Accepted as the value for the EGL_SAMPLE_RANGE_HINT_EXT attribute:
>>
>>         EGL_YUV_FULL_RANGE_EXT
>>         EGL_YUV_NARROW_RANGE_EXT
>>
>>     Accepted as the value for the EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
>>     EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes:
>>
>>         EGL_YUV_CHROMA_SITING_0_EXT
>>         EGL_YUV_CHROMA_SITING_0_5_EXT
>>
>>
>> Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
>>
>>     Add to section 2.5.1 "EGLImage Specification" (as defined by the
>>     EGL_KHR_image_base specification), in the description of
>>     eglCreateImageKHR:
>>
>>    "Values accepted for <target> are listed in Table aaa, below.
>>
>>       +-------------------------+--------------------------------------------+
>>       |  <target>               |  Notes                                     |
>>       +-------------------------+--------------------------------------------+
>>       |  EGL_LINUX_DMA_BUF_EXT  |   Used for EGLImages imported from Linux   |
>>       |                         |   dma_buf file descriptors                 |
>>       +-------------------------+--------------------------------------------+
>>        Table aaa.  Legal values for eglCreateImageKHR <target> parameter
>>
>>     ...
>>
>>     If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display, <ctx>
>>     must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
>>     EGLClientBuffer. The details of the image is specified by the attributes
>>     passed into eglCreateImageKHR. Required attributes and their values are as
>>     follows:
>>
>>         * EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in pixels
>>
>>         * EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as specified
>>           by drm_fourcc.h and used as the pixel_format parameter of the
>>           drm_mode_fb_cmd2 ioctl.
>>
>>         * EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0 of
>>           the image.
>>
>>         * EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
>>           dma_buf of the first sample in plane 0, in bytes.
>>
>>         * EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the start of
>>           subsequent rows of samples in plane 0. May have special meaning for
>>           non-linear formats.
>>
>>     For images in an RGB color-space or those using a single-plane YUV format,
>>     only the first plane's file descriptor, offset & pitch should be specified.
>>     For semi-planar YUV formats, the chroma samples are stored in plane 1 and
>>     for fully planar formats, U-samples are stored in plane 1 and V-samples are
>>     stored in plane 2. Planes 1 & 2 are specified by the following attributes,
>>     which have the same meanings as defined above for plane 0:
>>
>>         * EGL_DMA_BUF_PLANE1_FD_EXT
>>         * EGL_DMA_BUF_PLANE1_OFFSET_EXT
>>         * EGL_DMA_BUF_PLANE1_PITCH_EXT
>>         * EGL_DMA_BUF_PLANE2_FD_EXT
>>         * EGL_DMA_BUF_PLANE2_OFFSET_EXT
>>         * EGL_DMA_BUF_PLANE2_PITCH_EXT
>>
>>     In addition to the above required attributes, the application may also
>>     provide hints as to how the data should be interpreted by the GL. If any of
>>     these hints are not specified, the GL will guess based on the pixel format
>>     passed as the EGL_LINUX_DRM_FOURCC_EXT attribute or may fall-back to some
>>     default value. Not all GLs will be able to support all combinations of
>>     these hints and are free to use whatever settings they choose to achieve
>>     the closest possible match.
>>
>>         * EGL_YUV_COLOR_SPACE_HINT_EXT: The color-space the data is in. Only
>>           relevant for images in a YUV format, ignored when specified for an
>>           image in an RGB format. Accepted values are:
>>           EGL_ITU_REC601_EXT, EGL_ITU_REC709_EXT & EGL_ITU_REC2020_EXT.
>>
>>         * EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
>>           EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT: Where chroma samples are
>>           sited relative to luma samples when the image is in a sub-sampled
>>           format. When the image is not using chroma sub-sampling, the luma and
>>           chroma samples are assumed to be co-sited. Siting is split into the
>>           vertical and horizontal and is in a fixed range. A siting of zero
>>           means the first luma sample is taken from the same position in that
>>           dimension as the chroma sample. This is best illustrated in the
>>           diagram below:
>>
>>                  (0.5, 0.5)        (0.0, 0.5)        (0.0, 0.0)
>>                 +   +   +   +     +   +   +   +     *   +   *   +
>>                   x       x       x       x
>>                 +   +   +   +     +   +   +   +     +   +   +   +
>>
>>                 +   +   +   +     +   +   +   +     *   +   *   +
>>                   x       x       x       x
>>                 +   +   +   +     +   +   +   +     +   +   +   +
>>
>>             Luma samples (+), Chroma samples (x) Chrome & Luma samples (*)
>>
>>           Note this attribute is ignored for RGB images and non sub-sampled
>>           YUV images. Accepted values are: EGL_YUV_CHROMA_SITING_0_EXT (0.0)
>>           & EGL_YUV_CHROMA_SITING_0_5_EXT (0.5)
>>
>>         * EGL_SAMPLE_RANGE_HINT_EXT: The numerical range of samples. Only
>>           relevant for images in a YUV format, ignored when specified for
>>           images in an RGB format. Accepted values are: EGL_YUV_FULL_RANGE_EXT
>>           (0-256) & EGL_YUV_NARROW_RANGE_EXT (16-235).
>>
>>
>>     If eglCreateImageKHR is successful for a EGL_LINUX_DMA_BUF_EXT target, the
>>     EGL takes ownership of the file descriptor and is responsible for closing
>>     it, which it may do at any time while the EGLDisplay is initialized."
>>
>>
>>     Add to the list of error conditions for eglCreateImageKHR:
>>
>>       "* If <target> is EGL_LINUX_DMA_BUF_EXT and <buffer> is not NULL, the
>>          error EGL_BAD_PARAMETER is generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT, and the list of attributes is
>>          incomplete, EGL_BAD_PARAMETER is generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT
>>          attribute is set to a format not supported by the EGL, EGL_BAD_MATCH
>>          is generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT, and the EGL_LINUX_DRM_FOURCC_EXT
>>          attribute indicates a single-plane format, EGL_BAD_ATTRIBUTE is
>>          generated if any of the EGL_DMA_BUF_PLANE1_* or EGL_DMA_BUF_PLANE2_*
>>          attributes are specified.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
>>          EGL_YUV_COLOR_SPACE_HINT_EXT is not EGL_ITU_REC601_EXT,
>>          EGL_ITU_REC709_EXT or EGL_ITU_REC2020_EXT, EGL_BAD_ATTRIBUTE is
>>          generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
>>          EGL_SAMPLE_RANGE_HINT_EXT is not EGL_YUV_FULL_RANGE_EXT or
>>          EGL_YUV_NARROW_RANGE_EXT, EGL_BAD_ATTRIBUTE is generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT and the value specified for
>>          EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT or
>>          EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT is not
>>          EGL_YUV_CHROMA_SITING_0_EXT or EGL_YUV_CHROMA_SITING_0_5_EXT,
>>          EGL_BAD_ATTRIBUTE is generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT and one or more of the values
>>          specified for a plane's pitch or offset isn't supported by EGL,
>>          EGL_BAD_ACCESS is generated.
>>
>>        * If <target> is EGL_LINUX_DMA_BUF_EXT and eglCreateImageKHR fails,
>>          EGL does not retain ownership of the file descriptor and it is the
>>          responsibility of the application to close it."
>>
>>
>> Issues
>>
>>     1. Should this be a KHR or EXT extension?
>>
>>     ANSWER: EXT. Khronos EGL working group not keen on this extension as it is
>>     seen as contradicting the EGLStream direction the specification is going in.
>>     The working group recommends creating additional specs to allow an EGLStream
>>     producer/consumer connected to v4l2/DRM or any other Linux interface.
>>
>>     2. Should this be a generic any platform extension, or a Linux-only
>>     extension which explicitly states the handles are dma_buf fds?
>>
>>     ANSWER: There's currently no intention to port this extension to any OS not
>>     based on the Linux kernel. Consequently, this spec can be explicitly written
>>     against Linux and the dma_buf API.
>>
>>     3. Does ownership of the file descriptor pass to the EGL library?
>>
>>     ANSWER: If eglCreateImageKHR is successful, EGL assumes ownership of the
>>     file descriptors and is responsible for closing them.
>>
>>     4. How are the different YUV color spaces handled (BT.709/BT.601)?
>>
>>     ANSWER: The pixel formats defined in drm_fourcc.h only specify how the data
>>     is laid out in memory. It does not define how that data should be
>>     interpreted. Added a new EGL_YUV_COLOR_SPACE_HINT_EXT attribute to allow the
>>     application to specify which color space the data is in to allow the GL to
>>     choose an appropriate set of co-efficients if it needs to convert that data
>>     to RGB for example.
>>
>>     5. What chroma-siting is used for sub-sampled YUV formats?
>>
>>     ANSWER: The chroma siting is not specified by either the v4l2 or DRM APIs.
>>     This is similar to the color-space issue (4) in that the chroma siting
>>     doesn't affect how the data is stored in memory. However, the GL will need
>>     to know the siting in order to filter the image correctly. While the visual
>>     impact of getting the siting wrong is minor, provision should be made to
>>     allow an application to specify the siting if desired. Added additional
>>     EGL_YUV_CHROMA_HORIZONTAL_SITING_HINT_EXT &
>>     EGL_YUV_CHROMA_VERTICAL_SITING_HINT_EXT attributes to allow the siting to
>>     be specified using a set of pre-defined values (0 or 0.5).
>>
>>     6. How can an application query which formats the EGL implementation
>>     supports?
>>
>>     PROPOSAL: Don't provide a query mechanism but instead add an error condition
>>     that EGL_BAD_MATCH is raised if the EGL implementation doesn't support that
>>     particular format.
>>
>>     7. Which image formats should be supported and how is format specified?
>>
>>     Seem to be two options 1) specify a new enum in this specification and
>>     enumerate all possible formats. 2) Use an existing enum already in Linux,
>>     either v4l2_mbus_pixelcode and/or those formats listed in drm_fourcc.h?
>>
>>     ANSWER: Go for option 2) and just use values defined in drm_fourcc.h.
>>
>>     8. How can AYUV images be handled?
>>
>>     ANSWER: At least on fourcc.org and in drm_fourcc.h, there only seems to be
>>     a single AYUV format and that is a packed format, so everything, including
>>     the alpha component would be in the first plane.
>>
>>     9. How can you import interlaced images?
>>
>>     ANSWER: Interlaced frames are usually stored with the top & bottom fields
>>     interleaved in a single buffer. As the fields would need to be displayed as
>>     at different times, the application would create two EGLImages from the same
>>     buffer, one for the top field and another for the bottom. Both EGLImages
>>     would set the pitch to 2x the buffer width and the second EGLImage would use
>>     a suitable offset to indicate it started on the second line of the buffer.
>>     This should work regardless of whether the data is packed in a single plane,
>>     semi-planar or multi-planar.
>>
>>     If each interlaced field is stored in a separate buffer then it should be
>>     trivial to create two EGLImages, one for each field's buffer.
>>
>>     10. How are semi-planar/planar formats handled that have a different
>>     width/height for Y' and CbCr such as YUV420?
>>
>>     ANSWER: The spec says EGL_WIDTH & EGL_HEIGHT specify the *logical* width and
>>     height of the buffer in pixels. For pixel formats with sub-sampled Chroma
>>     values, it should be trivial for the EGL implementation to calculate the
>>     width/height of the Chroma sample buffers using the logical width & height
>>     and by inspecting the pixel format passed as the EGL_LINUX_DRM_FOURCC_EXT
>>     attribute. I.e. If the pixel format says it's YUV420, the Chroma buffer's
>>     width = EGL_WIDTH/2 & height =EGL_HEIGHT/2.
>>
>>     11. How are Bayer formats handled?
>>
>>     ANSWER: As of Linux 2.6.34, drm_fourcc.h does not include any Bayer formats.
>>     However, future kernel versions may add such formats in which case they
>>     would be handled in the same way as any other format.
>>
>>     12. Should the spec support buffers which have samples in a "narrow range"?
>>
>>     Content sampled from older analogue sources typically don't use the full
>>     (0-256) range of the data type storing the sample and instead use a narrow
>>     (16-235) range to allow some headroom & toeroom in the signals to avoid
>>     clipping signals which overshoot slightly during processing. This is
>>     sometimes known as signals using "studio swing".
>>
>>     ANSWER: Add a new attribute to define if the samples use a narrow 16-235
>>     range or the full 0-256 range.
>>
>>     13. Specifying the color space and range seems cumbersome, why not just
>>     allow the application to specify the full YUV->RGB color conversion matrix?
>>
>>     ANSWER: Some hardware may not be able to use an arbitrary conversion matrix
>>     and needs to select an appropriate pre-defined matrix based on the color
>>     space and the sample range.
>>
>>     14. How do you handle EGL implementations which have restrictions on pitch
>>     and/or offset?
>>
>>     ANSWER: Buffers being imported using dma_buf pretty much have to be
>>     allocated by a kernel-space driver. As such, it is expected that a system
>>     integrator would make sure all devices which allocate buffers suitable for
>>     exporting make sure they use a pitch supported by all possible importers.
>>     However, it is still possible eglCreateImageKHR can fail due to an
>>     unsupported pitch. Added a new error to the list indicating this.
>>
>>     15. Should this specification also describe how to export an existing
>>     EGLImage as a dma_buf file descriptor?
>>
>>     ANSWER: No. Importing and exporting buffers are two separate operations and
>>     importing an existing dma_buf fd into an EGLImage is useful functionality in
>>     itself. Agree that exporting an EGLImage as a dma_buf fd is useful, E.g. it
>>     could be used by an OpenMAX IL implementation's OMX_UseEGLImage function to
>>     give access to the buffer backing an EGLImage to video hardware. However,
>>     exporting can be split into a separate extension specification.
>>
>>
>> Revision History
>>
>> #4 (Tom Cooksey, October 04, 2012)
>>    - Fixed issue numbering!
>>    - Added issues 8 - 15.
>>    - Promoted proposal for Issue 3 to be the answer.
>>    - Added an additional attribute to allow an application to specify the color
>>      space as a hint which should address issue 4.
>>    - Added an additional attribute to allow an application to specify the chroma
>>      siting as a hint which should address issue 5.
>>    - Added an additional attribute to allow an application to specify the sample
>>      range as a hint which should address the new issue 12.
>>    - Added language to end of error section clarifying who owns the fd passed
>>      to eglCreateImageKHR if an error is generated.
>>
>> #3 (Tom Cooksey, August 16, 2012)
>>    - Changed name from EGL_EXT_image_external and re-written language to
>>      explicitly state this for use with Linux & dma_buf.
>>    - Added a list of issues, including some still open ones.
>>
>> #2 (Jesse Barker, May 30, 2012)
>>    - Revision to split eglCreateImageKHR functionality from export
>>      Functionality.
>>    - Update definition of EGLNativeBufferType to be a struct containing a list
>>      of handles to support multi-buffer/multi-planar formats.
>>
>> #1 (Jesse Barker, March 20, 2012)
>>    - Initial draft.
>>
>>
>>
>>
>> _______________________________________________
>> mesa-dev mailing list
>> mesa-dev@xxxxxxxxxxxxxxxxxxxxx
>> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
>
>
>
> _______________________________________________
> mesa-dev mailing list
> mesa-dev@xxxxxxxxxxxxxxxxxxxxx
> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]
  Powered by Linux