Hi everybody,
I need to implement support for a YUV frame buffer in an fbdev driver. As the
fbdev API doesn't support this out of the box, I've spent a couple of days
reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
support to the API. I'd like to share my findings and thoughts, and hopefully
receive some comments back.
The terms 'format', 'pixel format', 'frame buffer format' and 'data format'
will be used interchangeably in this e-mail. They all refer to the way pixels
are stored in memory, including both the representation of a pixel as integer
values and the layout of those integer values in memory.
History and current situation
-----------------------------
The fbdev API predates YUV frame buffers. In those old days developers only
cared (and probably thought) about RGB frame buffers, and they developed an
API that could express all kind of weird RGB layout in memory (such as R-
GGGGGGGGGGG-BBBB for instance, I can't imagine hardware implementing that).
This resulted in individual bit field description for the red, green, blue and
alpha components:
struct fb_bitfield {
__u32 offset; /* beginning of bitfield */
__u32 length; /* length of bitfield */
__u32 msb_right; /* != 0 : Most significant bit is */
/* right */
};
Grayscale formats were pretty common, so a grayscale field tells color formats
(grayscale == 0) from grayscale formats (grayscale != 0).
People already realized that hardware developers were crazily inventive (the
word to remember here is crazily), and that non-standard formats would be
needed at some point. The fb_var_screeninfo ended up containing the following
format-related fields.
struct fb_var_screeninfo {
...
__u32 bits_per_pixel; /* guess what */
__u32 grayscale; /* != 0 Graylevels instead of colors */
struct fb_bitfield red; /* bitfield in fb mem if true color, */
struct fb_bitfield green; /* else only length is significant */
struct fb_bitfield blue;
struct fb_bitfield transp; /* transparency */
__u32 nonstd; /* != 0 Non standard pixel format */
...
};
Two bits have been specified for the nonstd field:
#define FB_NONSTD_HAM 1 /* Hold-And-Modify (HAM) */
#define FB_NONSTD_REV_PIX_IN_B 2 /* order of pixels in each byte is reversed
*/
The FB_NONSTD_HAM bit is used by the video/amifb.c driver only to enable HAM
mode[1]. I very much doubt that any new hardware will implement HAM mode (and
I certainly hope none will).
The FB_NONSTD_REV_PIX_IN_B is used in video/fb_draw.h by the generic bitblit,
fillrect and copy area implementations, not directly by drivers.
A couple of drivers hardcode the nonstd field to 1 for some reason. Those are
video/arcfb.c (1bpp gray display), video/hecubafb.c (1bpp gray display) and
video/metronomefb.c (8bpp gray display).
The following drivers use nonstd for various other (and sometimes weird)
purposes:
video/arkfb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/fsl-diu-fb.c
Set var->nonstd bits into var->sync (why?)
video/pxafb.c
Encode frame buffer xpos and ypos in the nonstd field
video/s3fb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/vga16fb.c
When panning in non-8bpp, non-text mode, decrement xoffset
Do some other weird stuff when not 0
video/i810/i810_main.c
Select direct color mode when set to 1 (truecolor otherwise)
Fast forward a couple of years, hardware provides support for YUV frame
buffers. Several drivers had to provide YUV format selection to applications,
without any standard API to do so. All of them used the nonstd field for that
purpose:
media/video/ivtv/
Enable YUV mode when set to 1
video/pxafb.c
Encode pixel format in the nonstd field
video/sh_mobile_lcdfb.c
If bpp == 12 and nonstd != 0, enable NV12 mode
If bpp == 16 or bpp == 24, ?
video/omap/omapfb_main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422, YUV420 or YUY422)
video/omap2/omapfb/omapfb-main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422 or YUY422)
Those drivers use the nonstd field in different, incompatible ways.
Other related APIs
------------------
Beside the fbdev API, two other kernel/userspace APIs deal with video-related
modes and formats.
- Kernel Mode Setting (KMS)
The KMS API is similar in purpose to XRandR. Its main purpose is to provide a
kernel API to configure output video modes. As such, it doesn't care about
frame buffer formats, as they are irrelevant at the CRTC output.
In addition to handling video modes, the KMS API also provides support for
creating (and managing) frame buffer devices, as well as dumb scan-out
buffers. In-memory data format is relevant there, but KMS only handles width,
height, pitch, depth and bit-per-pixel information. The API doesn't care
whether the frame buffer or the dumb scan-out buffer contains RGB or YUV data.
An RFC[2] has recently been posted to the dri-devel mailing list to "add
overlays as first class KMS objects". The proposal includes explicit overlay
format support, and discussions have so far pushed towards reusing V4L format
codes for the KMS API.
- Video 4 Linux (V4L)
The V4L API version 2 (usually called V4L2) has since the beginning included
explicit support for data format, referred to as pixel formats.
Pixel formats are identified by a 32-bit number in the form of a four
characters code (4CC or FCC[3]). The list of FCCs defined by V4L2 is available
in [4].
A pixel format uniquely defines the layout of pixel data in memory, including
the format type (RGB, YUV, ...), number of bits per components, components
order and subsampling. It doesn't define the range of acceptable values for
pixel components (such as full-range YUV vs. BT.601[5]), which is carried
through a separate colorspace identifier[6].
YUV support in the fbdev API
----------------------------
Experience with the V4L2 API, both for desktop and embedded devices, and the
KMS API, suggests that recent hardware tend to standardize on a relatively
small number of pixel formats that don't require bitfield-level descriptions.
A pixel format definition based on identifiers should thus fullfill the
hardware needs for the foreseeable future.
Given the overlap between the KMS, V4L2 and fbdev APIs, the need to share data
and buffers between those subsystems, and the planned use of V4L2 FCCs in the
KMS overlay API, I believe using V4L2 FCCs to identify fbdev formats would be
a wise decision.
To select a frame buffer YUV format, the fb_var_screeninfo structure will need
to be extended with a format field. The fbdev API and ABI must not be broken,
which prevents us from changing the current structure layout and replacing the
existing format selection mechanism (through the red, green, blue and alpha
bitfields) completely.
Several solutions exist to introduce a format field in the fb_var_screeninfo
structure.
- Use the nonstd field as a format identifier. Existing drivers that use the
nonstd field for other purposes wouldn't be able to implement the new API
while keeping their existing API. This isn't a show stopper for drivers using
the FB_NONSTD_HAM and FB_NONSTD_REV_PIX_IN_B bits, as they don't need to
support any non-RGB format.
Existing drivers that support YUV formats through the nonstd field would have
to select between the current and the new API, without being able to implement
both.
- Use one of the reserved fields as a format identifier. Applications are
supposed to zero the fb_var_screeninfo structure before passing it to the
kernel, but experience showed that many applications forget to do so. Drivers
would then be unable to find out whether applications request a particular
format, or just forgot to initialize the field.
- Add a new FB_NONSTD_FORMAT bit to the nonstd field, and pass the format
through a separate field. If an application sets the FB_NONSTD_FORMAT bit in
the nonstd field, drivers will ignore the red, green, blue, transp and
grayscale fields, and use the format identifier to configure the frame buffer
format. The grayscale field would then be unneeded and could be reused as a
format identifier. Alternatively, one of the reserved fields could be used for
that purpose.
Existing drivers that support YUV formats through the nonstd field don't use
the field's most significant bits. They could implement both their current API
and the new API if the FB_NONSTD_FORMAT bit is stored in one of the nonstd
MSBs.
- Other solutions are possible, such as adding new ioctls. Those solutions are
more intrusive, and require larger changes to both userspace and kernelspace
code.
The third solution has my preference. Comments and feedback will be
appreciated. I will then work on a proof of concept and submit patches.
[1] http://en.wikipedia.org/wiki/Hold_And_Modify
[2] http://lwn.net/Articles/440192/
[3] http://www.fourcc.org/
[4] http://lxr.linux.no/linux+v2.6.38/include/linux/videodev2.h#L271
[5] http://en.wikipedia.org/wiki/Rec._601
[6] http://lxr.linux.no/linux+v2.6.38/include/linux/videodev2.h#L175
--
Regards,
Laurent Pinchart
_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
http://lists.freedesktop.org/mailman/listinfo/dri-devel
