Hi Niklas,
Thank you for the update.
On Fri, May 22, 2020 at 01:52:01AM +0200, Niklas Söderlund wrote:
> Bayer formats are used with cameras and contain green, red and blue
> components, with alternating lines of red and green, and blue and green
> pixels in different orders. For each block of 2x2 pixels there is one
> pixel with a red filter, two with a green filter, and one with a blue
> filter. The filters can be arranged in different patterns.
>
> Add DRM fourcc formats to describe the most common Bayer formats. Also
> add a modifiers to describe the custom packing layouts used by the Intel
> IPU3 and in the MIPI (Mobile Industry Processor Interface) CSI-2
> specification.
>
> Signed-off-by: Niklas Söderlund <niklas.soderlund@xxxxxxxxxxxx>
> ---
> * Changes since v1
> - Rename the defines from DRM_FORMAT_SRGGB8 to DRM_FORMAT_BAYER_RGGB8.
> - Update the fourcc codes passed to fourcc_code() to avoid a conflict.
> - Add diagrams for all Bayer formats memory layout.
> - Update documentation.
> ---
> include/uapi/drm/drm_fourcc.h | 205 ++++++++++++++++++++++++++++++++++
> 1 file changed, 205 insertions(+)
>
> diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
> index 8bc0b31597d80737..d07dd24b49bde6c1 100644
> --- a/include/uapi/drm/drm_fourcc.h
> +++ b/include/uapi/drm/drm_fourcc.h
> @@ -285,6 +285,73 @@ extern "C" {
> #define DRM_FORMAT_YUV444 fourcc_code('Y', 'U', '2', '4') /* non-subsampled Cb (1) and Cr (2) planes */
> #define DRM_FORMAT_YVU444 fourcc_code('Y', 'V', '2', '4') /* non-subsampled Cr (1) and Cb (2) planes */
>
> +/*
> + * Bayer formats
> + *
> + * Bayer formats contain green, red and blue components, with alternating lines
> + * of red and green, and blue and green pixels in different orders. For each
> + * block of 2x2 pixels there is one pixel with a red filter, two with a green
> + * filter, and one with a blue filter. The filters can be arranged in different
> + * patterns.
> + *
> + * For example, RGGB:
> + * row0: RGRGRGRG...
> + * row1: GBGBGBGB...
> + * row2: RGRGRGRG...
> + * row3: GBGBGBGB...
> + * ...
> + *
> + * Vendors have different methods to pack the pixel samples. For this reason the
> + * fourcc only describes pixel sample size and the filter pattern for each block
> + * of 2x2 pixels. A modifier is needed to describe the memory layout.
> + *
> + * In addition to vendor modifiers for memory layout DRM_FORMAT_MOD_LINEAR may
> + * be used to describe a layout where all samples are placed consecutively in
> + * memory. If the sample does not fit inside a single byte each sample is stored
> + * in the minimum number of bytes required. Any unused bits in each sample are
> + * defined as padding bits and set to zero.
> + *
> + * For example, DRM_FORMAT_BAYER_RGGB10 with DRM_FORMAT_MOD_LINEAR:
> + *
> + * 0 row 0 (RGRG) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r0 r0 r0 r0 r0 r0 r0 r0 | 0 0 0 0 0 0 r0 r0 | g0 g0 g0 g0 g0 g0 g0 g0 | 0 0 0 0 0 0 g0 g0 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r1 r1 r1 r1 r1 r1 r1 r1 | 0 0 0 0 0 0 r1 r1 | g1 g1 g1 g1 g1 g1 g1 g1 | 0 0 0 0 0 0 g1 g1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + *
> + * 0 row 1 (GBGB) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g0 g0 g0 g0 g0 g0 g0 g0 | 0 0 0 0 0 0 g0 g0 | b0 b0 b0 b0 b0 b0 b0 b0 | 0 0 0 0 0 0 b0 b0 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g1 g1 g1 g1 g1 g1 g1 g1 | 0 0 0 0 0 0 g1 g1 | b1 b1 b1 b1 b1 b1 b1 b1 | 0 0 0 0 0 0 b1 b1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + */
> +
> +/* 8-bits Bayer formats */
> +#define DRM_FORMAT_BAYER_RGGB8 fourcc_code('R', 'G', 'G', 'B')
> +#define DRM_FORMAT_BAYER_GRBG8 fourcc_code('G', 'R', 'B', 'G')
> +#define DRM_FORMAT_BAYER_GBRG8 fourcc_code('G', 'B', 'R', 'G')
> +#define DRM_FORMAT_BAYER_BGGR8 fourcc_code('B', 'G', 'G', 'R')
> +
> +/* 10-bit Bayer formats */
> +#define DRM_FORMAT_BAYER_RGGB10 fourcc_code('R', 'G', '1', '0')
> +#define DRM_FORMAT_BAYER_GRBG10 fourcc_code('G', 'R', '1', '0')
> +#define DRM_FORMAT_BAYER_GBRG10 fourcc_code('G', 'B', '1', '0')
> +#define DRM_FORMAT_BAYER_BGGR10 fourcc_code('B', 'G', '1', '0')
> +
> +/* 12-bit Bayer formats */
> +#define DRM_FORMAT_BAYER_RGGB12 fourcc_code('R', 'G', '1', '2')
> +#define DRM_FORMAT_BAYER_GRBG12 fourcc_code('G', 'R', '1', '2')
> +#define DRM_FORMAT_BAYER_GBRG12 fourcc_code('G', 'B', '1', '2')
> +#define DRM_FORMAT_BAYER_BGGR12 fourcc_code('B', 'G', '1', '2')
> +
> +/* 14-bit Bayer formats */
> +#define DRM_FORMAT_BAYER_RGGB14 fourcc_code('R', 'G', '1', '4')
> +#define DRM_FORMAT_BAYER_GRBG14 fourcc_code('G', 'R', '1', '4')
> +#define DRM_FORMAT_BAYER_GBRG14 fourcc_code('G', 'B', '1', '4')
> +#define DRM_FORMAT_BAYER_BGGR14 fourcc_code('B', 'G', '1', '4')
> +
>
> /*
> * Format Modifiers:
> @@ -309,6 +376,7 @@ extern "C" {
> #define DRM_FORMAT_MOD_VENDOR_BROADCOM 0x07
> #define DRM_FORMAT_MOD_VENDOR_ARM 0x08
> #define DRM_FORMAT_MOD_VENDOR_ALLWINNER 0x09
> +#define DRM_FORMAT_MOD_VENDOR_MIPI 0x0a
This is an interesting one. I don't think these formats have originated
from MIPI. The colour pattern itself is from Eastman Kodak apparently
(named after Bryce Bayer), but the memory format is not, apart from the
MIPI CSI-2 packed variant.
The formats are probably unlike the formats used on GPUs as they are
relatively similar across a number of vendors and devices.
There are more raw formats than just the Bayer formats, see e.g.
<URL:https://en.wikipedia.org/wiki/Raw_image_format>
At the same time, it'd be good to keep the CSI-2 packed variant as a format
modifier. I wonder if we could have something like
DRM_FORMAT_MOD_VENDOR_RAW, albeit a raw format is not a vendor.
Cc Daniel and Laurent.
>
> /* add more to the end as needed */
>
> @@ -434,6 +502,56 @@ extern "C" {
> */
> #define I915_FORMAT_MOD_Y_TILED_GEN12_MC_CCS fourcc_mod_code(INTEL, 7)
>
> +
> +/*
> + * IPU3 Bayer packing layout
> + *
> + * The IPU3 raw Bayer formats use a custom packing layout where there are no
> + * gaps between each 10-bit sample. It packs 25 pixels into 32 bytes leaving
> + * the 6 most significant bits in the last byte unused zero padding bits.
> + *
> + * For example, DRM_FORMAT_BAYER_BGGR10 with IPU3_FORMAT_MOD_PACKED:
> + *
> + * 0 row 0 (BGBGBGBGBGBGBGBGBGBGBGBGB) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b0 b0 b0 b0 b0 b0 b0 b0 | g0 g0 g0 g0 g0 g0 b0 b0 | b1 b1 b1 b1 g0 g0 g0 g0 | g1 g1 b1 b1 b1 b1 b1 b1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g1 g1 g1 g1 g1 g1 g1 g1 | b2 b2 b2 b2 b2 b2 b2 b2 | g2 g2 g2 g2 g2 g2 b2 b2 | b3 b3 b3 b3 g2 g2 g2 g2 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g3 g3 b3 b3 b3 b3 b3 b3 | g3 g3 g3 g3 g3 g3 g3 g3 | b4 b4 b4 b4 b4 b4 b4 b4 | g4 g4 g4 g4 g4 g4 b4 b4 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b5 b5 b5 b5 g4 g4 g4 g4 | g5 g5 b5 b5 b5 b5 b5 b5 | g5 g5 g5 g5 g5 g5 g5 g5 | b6 b6 b6 b6 b6 b6 b6 b6 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g6 g6 g6 g6 g6 g6 b6 b6 | b7 b7 b7 b7 g6 g6 g6 g6 | g7 g7 b7 b7 b7 b7 b7 b7 | g7 g7 g7 g7 g7 g7 g7 g7 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b8 b8 b8 b8 b8 b8 b8 b8 | g8 g8 g8 g8 g8 g8 b8 b8 | b9 b9 b9 b9 g8 g8 g8 g8 | g9 g9 b9 b9 b9 b9 b9 b9 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g9 g9 g9 g9 g9 g9 g9 g9 | bA bA bA bA bA bA bA bA | gA gA gA gA gA gA bA bA | bB bB bB bB gA gA gA gA |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | gB gB bB bB bB bB bB bB | gB gB gB gB gB gB gB gB | bC bC bC bC bC bC bC bC | 0 0 0 0 0 0 bC bC |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + *
> + * 0 row 1 (GRGRGRGRGRGRGRGRGRGRGRGRG) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g0 g0 g0 g0 g0 g0 g0 g0 | r0 r0 r0 r0 r0 r0 g0 g0 | g1 g1 g1 g1 r0 r0 r0 r0 | r1 r1 g1 g1 g1 g1 g1 g1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r1 r1 r1 r1 r1 r1 r1 r1 | g2 g2 g2 g2 g2 g2 g2 g2 | r2 r2 r2 r2 r2 r2 g2 g2 | g3 g3 g3 g3 r2 r2 r2 r2 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r3 r3 g3 g3 g3 g3 g3 g3 | r3 r3 r3 r3 r3 r3 r3 r3 | g4 g4 g4 g4 g4 g4 g4 g4 | r4 r4 r4 r4 r4 r4 g4 g4 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g5 g5 g5 g5 r4 r4 r4 r4 | r5 r5 g5 g5 g5 g5 g5 g5 | r5 r5 r5 r5 r5 r5 r5 r5 | g6 g6 g6 g6 g6 g6 g6 g6 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r6 r6 r6 r6 r6 r6 g6 g6 | g7 g7 g7 g7 r6 r6 r6 r6 | r7 r7 g7 g7 g7 g7 g7 g7 | r7 r7 r7 r7 r7 r7 r7 r7 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g8 g8 g8 g8 g8 g8 g8 g8 | r8 r8 r8 r8 r8 r8 g8 g8 | g9 g9 g9 g9 r8 r8 r8 r8 | r9 r9 g9 g9 g9 g9 g9 g9 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r9 r9 r9 r9 r9 r9 r9 r9 | gA gA gA gA gA gA gA gA | rA rA rA rA rA rA gA gA | gB gB gB gB rA rA rA rA |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | rB rB gB gB gB gB gB gB | rB rB rB rB rB rB rB rB | gC gC gC gC gC gC gC gC | 0 0 0 0 0 0 gC gC |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + */
> +#define IPU3_FORMAT_MOD_PACKED fourcc_mod_code(INTEL, 8)
> +
> /*
> * Tiled, NV12MT, grouped in 64 (pixels) x 32 (lines) -sized macroblocks
> *
> @@ -804,6 +922,93 @@ extern "C" {
> */
> #define DRM_FORMAT_MOD_ALLWINNER_TILED fourcc_mod_code(ALLWINNER, 1)
>
> +/* Mobile Industry Processor Interface (MIPI) modifiers */
> +
> +/*
> + * MIPI CSI-2 packing layout
> + *
> + * The CSI-2 RAW formats (for example Bayer) use a different packing layout
> + * depending on the sample size.
> + *
> + * - 10-bits per sample
> + * Every four consecutive samples are packed into 5 bytes. Each of the first 4
> + * bytes contain the 8 high order bits of the pixels, and the 5th byte
> + * contains the 2 least-significant bits of each pixel, in the same order.
> + *
> + * For example, DRM_FORMAT_BAYER_BGGR10 with MIPI_FORMAT_MOD_CSI2_PACKED:
> + *
> + * 0 row 0 (BGBGBGBG) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b0 b0 b0 b0 b0 b0 b0 b0 | g0 g0 g0 g0 g0 g0 g0 g0 | b1 b1 b1 b1 b1 b1 b1 b1 | g1 g1 g1 g1 g1 g1 g1 g1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g1 g1 b1 b1 g0 g0 b0 b0 | b2 b2 b2 b2 b2 b2 b2 b2 | g2 g2 g2 g2 g2 g2 g2 g2 | b3 b3 b3 b3 b3 b3 b3 b3 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g3 g3 g3 g3 g3 g3 g3 g3 | g3 g3 b3 b3 g2 g2 b2 b2 |
> + * + - - - - - - - - + - - - - - - - - +
> + *
> + * 0 row 1 (GRGRGRGR) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g0 g0 g0 g0 g0 g0 g0 g0 | r0 r0 r0 r0 r0 r0 r0 r0 | g1 g1 g1 g1 g1 g1 g1 g1 | r1 r1 r1 r1 r1 r1 r1 r1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r1 r1 g1 g1 r0 r0 g0 g0 | g2 g2 g2 g2 g2 g2 g2 g2 | r2 r2 r2 r2 r2 r2 r2 r2 | g3 g3 g3 g3 g3 g3 g3 g3 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r3 r3 r3 r3 r3 r3 r3 r3 | r3 r3 g3 g3 r2 r2 g2 g2 |
> + * + - - - - - - - - + - - - - - - - - +
> + *
> + * - 12-bits per sample
> + * Every two consecutive samples are packed into three bytes. Each of the
> + * first two bytes contain the 8 high order bits of the pixels, and the third
> + * byte contains the four least-significant bits of each pixel, in the same
> + * order.
> + *
> + * For example, DRM_FORMAT_BAYER_GRBG12 with MIPI_FORMAT_MOD_CSI2_PACKED:
> + *
> + * 0 row 0 (GRGR) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g0 g0 g0 g0 g0 g0 g0 g0 | r0 r0 r0 r0 r0 r0 r0 r0 | r0 r0 r0 r0 g0 g0 g0 g0 | g1 g1 g1 g1 g1 g1 g1 g1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r1 r1 r1 r1 r1 r1 r1 r1 | r1 r1 r1 r1 g1 g1 g1 g1 |
> + * + - - - - - - - - + - - - - - - - - +
> + *
> + * 0 row 1 (BGBG) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b0 b0 b0 b0 b0 b0 b0 b0 | g0 g0 g0 g0 g0 g0 g0 g0 | g0 g0 g0 g0 b0 b0 b0 b0 | b1 b1 b1 b1 b1 b1 b1 b1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g1 g1 g1 g1 g1 g1 g1 g1 | g1 g1 g1 g1 b1 b1 b1 b1 |
> + * + - - - - - - - - + - - - - - - - - +
> + *
> + * - 14-bits per sample
> + * Every four consecutive samples are packed into seven bytes. Each of the
> + * first four bytes contain the eight high order bits of the pixels, and the
> + * three following bytes contains the six least-significant bits of each
> + * pixel, in the same order.
> + *
> + * For example, DRM_FORMAT_BAYER_GBRG14 with MIPI_FORMAT_MOD_CSI2_PACKED:
> + *
> + * 0 row 0 (GBGBGBGB) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g0 g0 g0 g0 g0 g0 g0 g0 | b0 b0 b0 b0 b0 b0 b0 b0 | g1 g1 g1 g1 g1 g1 g1 g1 | b1 b1 b1 b1 b1 b1 b1 b1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b0 b0 g0 g0 g0 g0 g0 g0 | g1 g1 g1 g1 b0 b0 b0 b0 | b1 b1 b1 b1 b1 b1 g1 g1 | g2 g2 g2 g2 g2 g2 g2 g2 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | b2 b2 b2 b2 b2 b2 b2 b2 | g3 g3 g3 g3 g3 g3 g3 g3 | b3 b3 b3 b3 b3 b3 b3 b3 | b2 b2 g2 g2 g2 g2 g2 g2 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g3 g3 g3 g3 b2 b2 b2 b2 | b3 b3 b3 b3 b3 b3 g3 g3 |
> + * + - - - - - - - - + - - - - - - - - +
> + *
> + * 0 row 1 (RGRGRGRG) 31
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r0 r0 r0 r0 r0 r0 r0 r0 | g0 g0 g0 g0 g0 g0 g0 g0 | r1 r1 r1 r1 r1 r1 r1 r1 | g1 g1 g1 g1 g1 g1 g1 g1 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g0 g0 r0 r0 r0 r0 r0 r0 | r1 r1 r1 r1 g0 g0 g0 g0 | g1 g1 g1 g1 g1 g1 r1 r1 | r2 r2 r2 r2 r2 r2 r2 r2 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | g2 g2 g2 g2 g2 g2 g2 g2 | r3 r3 r3 r3 r3 r3 r3 r3 | g3 g3 g3 g3 g3 g3 g3 g3 | g2 g2 r2 r2 r2 r2 r2 r2 |
> + * + - - - - - - - - + - - - - - - - - + - - - - - - - - + - - - - - - - - +
> + * | r3 r3 r3 r3 g2 g2 g2 g2 | g3 g3 g3 g3 g3 g3 r3 r3 |
> + * + - - - - - - - - + - - - - - - - - +
> + */
> +#define MIPI_FORMAT_MOD_CSI2_PACKED fourcc_mod_code(MIPI, 1)
> +
> #if defined(__cplusplus)
> }
> #endif
--
Kind regards,
Sakari Ailus
_______________________________________________
dri-devel mailing list
dri-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/dri-devel
