-----Original Message-----
From: Harry Wentland <harry.wentland@xxxxxxx>
Sent: Friday, October 20, 2023 2:51 AM
To: dri-devel@xxxxxxxxxxxxxxxxxxxxx
Cc: wayland-devel@xxxxxxxxxxxxxxxxxxxxx; Harry Wentland
<harry.wentland@xxxxxxx>; Ville Syrjala <ville.syrjala@xxxxxxxxxxxxxxx>; Pekka
Paalanen <pekka.paalanen@xxxxxxxxxxxxx>; Simon Ser <contact@xxxxxxxxxxx>;
Melissa Wen <mwen@xxxxxxxxxx>; Jonas Ådahl <jadahl@xxxxxxxxxx>; Sebastian
Wick <sebastian.wick@xxxxxxxxxx>; Shashank Sharma
<shashank.sharma@xxxxxxx>; Alexander Goins <agoins@xxxxxxxxxx>; Joshua
Ashton <joshua@xxxxxxxxx>; Michel Dänzer <mdaenzer@xxxxxxxxxx>; Aleix Pol
<aleixpol@xxxxxxx>; Xaver Hugl <xaver.hugl@xxxxxxxxx>; Victoria Brekenfeld
<victoria@xxxxxxxxxxxx>; Sima <daniel@xxxxxxxx>; Shankar, Uma
<uma.shankar@xxxxxxxxx>; Naseer Ahmed <quic_naseer@xxxxxxxxxxx>;
Christopher Braga <quic_cbraga@xxxxxxxxxxx>; Abhinav Kumar
<quic_abhinavk@xxxxxxxxxxx>; Arthur Grillo <arthurgrillo@xxxxxxxxxx>; Hector
Martin <marcan@xxxxxxxxx>; Liviu Dudau <Liviu.Dudau@xxxxxxx>; Sasha
McIntosh <sashamcintosh@xxxxxxxxxx>
Subject: [RFC PATCH v2 06/17] drm/doc/rfc: Describe why prescriptive color
pipeline is needed
v2:
- Update colorop visualizations to match reality (Sebastian, Alex Hung)
- Updated wording (Pekka)
- Change BYPASS wording to make it non-mandatory (Sebastian)
- Drop cover-letter-like paragraph from COLOR_PIPELINE Plane Property
section (Pekka)
- Use PQ EOTF instead of its inverse in Pipeline Programming example (Melissa)
- Add "Driver Implementer's Guide" section (Pekka)
- Add "Driver Forward/Backward Compatibility" section (Sebastian, Pekka)
Signed-off-by: Harry Wentland <harry.wentland@xxxxxxx>
Cc: Ville Syrjala <ville.syrjala@xxxxxxxxxxxxxxx>
Cc: Pekka Paalanen <pekka.paalanen@xxxxxxxxxxxxx>
Cc: Simon Ser <contact@xxxxxxxxxxx>
Cc: Harry Wentland <harry.wentland@xxxxxxx>
Cc: Melissa Wen <mwen@xxxxxxxxxx>
Cc: Jonas Ådahl <jadahl@xxxxxxxxxx>
Cc: Sebastian Wick <sebastian.wick@xxxxxxxxxx>
Cc: Shashank Sharma <shashank.sharma@xxxxxxx>
Cc: Alexander Goins <agoins@xxxxxxxxxx>
Cc: Joshua Ashton <joshua@xxxxxxxxx>
Cc: Michel Dänzer <mdaenzer@xxxxxxxxxx>
Cc: Aleix Pol <aleixpol@xxxxxxx>
Cc: Xaver Hugl <xaver.hugl@xxxxxxxxx>
Cc: Victoria Brekenfeld <victoria@xxxxxxxxxxxx>
Cc: Sima <daniel@xxxxxxxx>
Cc: Uma Shankar <uma.shankar@xxxxxxxxx>
Cc: Naseer Ahmed <quic_naseer@xxxxxxxxxxx>
Cc: Christopher Braga <quic_cbraga@xxxxxxxxxxx>
Cc: Abhinav Kumar <quic_abhinavk@xxxxxxxxxxx>
Cc: Arthur Grillo <arthurgrillo@xxxxxxxxxx>
Cc: Hector Martin <marcan@xxxxxxxxx>
Cc: Liviu Dudau <Liviu.Dudau@xxxxxxx>
Cc: Sasha McIntosh <sashamcintosh@xxxxxxxxxx>
---
Documentation/gpu/rfc/color_pipeline.rst | 347 +++++++++++++++++++++++
1 file changed, 347 insertions(+)
create mode 100644 Documentation/gpu/rfc/color_pipeline.rst
diff --git a/Documentation/gpu/rfc/color_pipeline.rst
b/Documentation/gpu/rfc/color_pipeline.rst
new file mode 100644
index 000000000000..af5f2ea29116
--- /dev/null
+++ b/Documentation/gpu/rfc/color_pipeline.rst
@@ -0,0 +1,347 @@
+========================
+Linux Color Pipeline API
+========================
+
+What problem are we solving?
+============================
+
+We would like to support pre-, and post-blending complex color
+transformations in display controller hardware in order to allow for
+HW-supported HDR use-cases, as well as to provide support to
+color-managed applications, such as video or image editors.
+
+It is possible to support an HDR output on HW supporting the Colorspace
+and HDR Metadata drm_connector properties, but that requires the
+compositor or application to render and compose the content into one
+final buffer intended for display. Doing so is costly.
+
+Most modern display HW offers various 1D LUTs, 3D LUTs, matrices, and
+other operations to support color transformations. These operations are
+often implemented in fixed-function HW and therefore much more power
+efficient than performing similar operations via shaders or CPU.
+
+We would like to make use of this HW functionality to support complex
+color transformations with no, or minimal CPU or shader load.
+
+
+How are other OSes solving this problem?
+========================================
+
+The most widely supported use-cases regard HDR content, whether video
+or gaming.
+
+Most OSes will specify the source content format (color gamut, encoding
+transfer function, and other metadata, such as max and average light levels) to a
driver.
+Drivers will then program their fixed-function HW accordingly to map
+from a source content buffer's space to a display's space.
+
+When fixed-function HW is not available the compositor will assemble a
+shader to ask the GPU to perform the transformation from the source
+content format to the display's format.
+
+A compositor's mapping function and a driver's mapping function are
+usually entirely separate concepts. On OSes where a HW vendor has no
+insight into closed-source compositor code such a vendor will tune
+their color management code to visually match the compositor's. On
+other OSes, where both mapping functions are open to an implementer they will
ensure both mappings match.
+
+This results in mapping algorithm lock-in, meaning that no-one alone
+can experiment with or introduce new mapping algorithms and achieve
+consistent results regardless of which implementation path is taken.
+
+Why is Linux different?
+=======================
+
+Unlike other OSes, where there is one compositor for one or more
+drivers, on Linux we have a many-to-many relationship. Many compositors;
many drivers.
+In addition each compositor vendor or community has their own view of
+how color management should be done. This is what makes Linux so beautiful.
+
+This means that a HW vendor can now no longer tune their driver to one
+compositor, as tuning it to one could make it look fairly different
+from another compositor's color mapping.
+
+We need a better solution.
+
+
+Descriptive API
+===============
+
+An API that describes the source and destination colorspaces is a
+descriptive API. It describes the input and output color spaces but
+does not describe how precisely they should be mapped. Such a mapping
+includes many minute design decision that can greatly affect the look of the final
result.
+
+It is not feasible to describe such mapping with enough detail to
+ensure the same result from each implementation. In fact, these
+mappings are a very active research area.
+
+
+Prescriptive API
+================
+
+A prescriptive API describes not the source and destination
+colorspaces. It instead prescribes a recipe for how to manipulate pixel
+values to arrive at the desired outcome.
+
+This recipe is generally an ordered list of straight-forward
+operations, with clear mathematical definitions, such as 1D LUTs, 3D
+LUTs, matrices, or other operations that can be described in a precise manner.
+
+
+The Color Pipeline API
+======================
+
+HW color management pipelines can significantly differ between HW
+vendors in terms of availability, ordering, and capabilities of HW
+blocks. This makes a common definition of color management blocks and
+their ordering nigh impossible. Instead we are defining an API that
+allows user space to discover the HW capabilities in a generic manner,
+agnostic of specific drivers and hardware.
+
+
+drm_colorop Object & IOCTLs
+===========================
+
+To support the definition of color pipelines we define the DRM core
+object type drm_colorop. Individual drm_colorop objects will be chained
+via the NEXT property of a drm_colorop to constitute a color pipeline.
+Each drm_colorop object is unique, i.e., even if multiple color
+pipelines have the same operation they won't share the same drm_colorop
+object to describe that operation.
+
+Note that drivers are not expected to map drm_colorop objects
+statically to specific HW blocks. The mapping of drm_colorop objects is
+entirely a driver-internal detail and can be as dynamic or static as a
+driver needs it to be. See more in the Driver Implementation Guide section
below.
+
+Just like other DRM objects the drm_colorop objects are discovered via
+IOCTLs:
+
+DRM_IOCTL_MODE_GETCOLOROPRESOURCES: This IOCTL is used to retrieve
the
+number of all drm_colorop objects.
+
+DRM_IOCTL_MODE_GETCOLOROP: This IOCTL is used to read one drm_colorop.
+It includes the ID for the colorop object, as well as the plane_id of
+the associated plane. All other values should be registered as
+properties.
+
+Each drm_colorop has three core properties:
+
+TYPE: The type of transformation, such as
+* enumerated curve
+* custom (uniform) 1D LUT
+* 3x3 matrix
+* 3x4 matrix
+* 3D LUT
+* etc.
+
+Depending on the type of transformation other properties will describe
+more details.
+
+BYPASS: A boolean property that can be used to easily put a block into
+bypass mode. While setting other properties might fail atomic check,
+setting the BYPASS property to true should never fail. The BYPASS
+property is not mandatory for a colorop, as long as the entire pipeline
+can get bypassed by setting the COLOR_PIPELINE on a plane to '0'.
+
+NEXT: The ID of the next drm_colorop in a color pipeline, or 0 if this
+drm_colorop is the last in the chain.
+
+An example of a drm_colorop object might look like one of these::
+
+ /* 1D enumerated curve */
+ Color operation 42
+ ├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3 matrix, 3x4
matrix, 3D LUT, etc.} = 1D enumerated curve
+ ├─ "BYPASS": bool {true, false}
+ ├─ "CURVE_1D_TYPE": enum {sRGB EOTF, sRGB inverse EOTF, PQ EOTF, PQ
inverse EOTF, …}
