On Thu, 04 May 2023 15:22:59 +0000 Simon Ser <contact@xxxxxxxxxxx> wrote: > Hi all, > > The goal of this RFC is to expose a generic KMS uAPI to configure the color > pipeline before blending, ie. after a pixel is tapped from a plane's > framebuffer and before it's blended with other planes. With this new uAPI we > aim to reduce the battery life impact of color management and HDR on mobile > devices, to improve performance and to decrease latency by skipping > composition on the 3D engine. This proposal is the result of discussions at > the Red Hat HDR hackfest [1] which took place a few days ago. Engineers > familiar with the AMD, Intel and NVIDIA hardware have participated in the > discussion. > > This proposal takes a prescriptive approach instead of a descriptive approach. > Drivers describe the available hardware blocks in terms of low-level > mathematical operations, then user-space configures each block. We decided > against a descriptive approach where user-space would provide a high-level > description of the colorspace and other parameters: we want to give more > control and flexibility to user-space, e.g. to be able to replicate exactly the > color pipeline with shaders and switch between shaders and KMS pipelines > seamlessly, and to avoid forcing user-space into a particular color management > policy. > > We've decided against mirroring the existing CRTC properties > DEGAMMA_LUT/CTM/GAMMA_LUT onto KMS planes. Indeed, the color management > pipeline can significantly differ between vendors and this approach cannot > accurately abstract all hardware. In particular, the availability, ordering and > capabilities of hardware blocks is different on each display engine. So, we've > decided to go for a highly detailed hardware capability discovery. > > This new uAPI should not be in conflict with existing standard KMS properties, > since there are none which control the pre-blending color pipeline at the > moment. It does conflict with any vendor-specific properties like > NV_INPUT_COLORSPACE or the patches on the mailing list adding AMD-specific > properties. Drivers will need to either reject atomic commits configuring both > uAPIs, or alternatively we could add a DRM client cap which hides the vendor > properties and shows the new generic properties when enabled. Hi, I have some further ideas which do conflict with some existing KMS properties. This dives into the color encoding specific side of the UAPI. The main idea is to make the color pipeline not specific to RGB. We might actually be having YCbCr, XYZ, ICtCp and whatnot instead, at least in the middle of a pipeline. The aim is to avoid the confusion from statements like "my red channel is actually luma and not red". So it's purely syntactic. ISTR some people being against saying "R is just a channel name, it's not necessarily a red component." Therefore I propose to address the color channels with indices instead: ch 0, ch 1, ch 2, ch 3 Then we define the mapping between pixel and wire formats and the indices: R = ch 0 G = ch 1 B = ch 2 A = ch 3 Y = ch 0 U = ch 1 V = ch 2 If necessary, the following can also be defined: Z = ch 1 X = ch 2 L = ch 0 M = ch 1 S = ch 2 The Y from YUV and Y from XYZ share the designation for the name's sake although they are not the same quantity. If YUV is not a well-defined designation wrt. YCbCr, ICtCp and everything else in the same category, we can assign Cb, Cr, I, Ct, Cp etc. instead. That might be more clear anyway even if there is a popular convention. We can also choose differently, to e.g. match the H.273 mapping where channels are assigned such that Y=G, Cr=R, Cb=B. H.273 gives mappings between almost all of these, so if using those make more sense, then let's use those. In the end it shouldn't matter too much, since one does not arbitrarily mix channels from different formats. Special care needs to be taken when defining COLOROP elements that do not handle all channels interchangeably. (E.g. a curve set element is mostly channel-agnostic when it applies the same curve to channels 0-2, but ch 3 is pass-through.) Then, we define COLOROP elements in terms of channel indices. This removes any implied connection to any specific color coding. Elements that just do not make sense for arbitrary channel ordering, e.g. special-case elements or enumerated matrix elements with a specific purpose, will document the intended usage and the expected channel mapping. The main reason to do all this is to ultimately allow e.g. limited range YCbCr scanout with a fully pass-through pipeline with no implied conversion to or from RGB. This is where some existing KMS properties will conflict: those that affect how current implicit YUV-RGB conversions are done. These properties shall be replaced with COLOROP elements in pipelines, so that they can be controlled explicitly and we can know where they reside wrt. e.g. sampling operations. Chroma channel reconstruction from sub-sampled chroma planes could potentially be explicitly represented, and so it could also be controlled (chroma siting). "Nominal" or "normalized" color value encoding at the input and output of each COLOROP element needs to be defined as well. Some elements, like matrices, can theoretically handle arbitrary values, but some elements like LUTs are inherently limited in their input values. Regardless of how a LUT is defined, it almost always assumes an input range [0.0, 1.0] of nominal color value encoding. This is particularly important for cases where out-of-unit-range values are expected: - scRGB - maybe some HDR encodings - YCbCr and anything else with chroma which is normally [-0.5, 0.5] - xvYCC Most likely we need to find some way to map everything to the nominal range [0.0, 1.0] if at all possible. We also need to know how elements handle input values outside of that range. The KMS property to control the scaling filter would be replaced by the scaling COLOROP element with changeable filter property. All this applies to pre-blending pipelines already. For the post-blending pipeline we have the added complication of drivers automatically choosing the wire format, but maybe that can be hand-waved with a special-purpose COLOROP element that does whatever the driver chooses, until it can be fully controlled from userspace. Thanks, pq
Attachment:
pgpmNT9CMbytP.pgp
Description: OpenPGP digital signature
