On Wed, 10 May 2023 at 00:31, Harry Wentland <harry.wentland@xxxxxxx> wrote: > > > > On 5/7/23 19:14, Dave Airlie wrote: > > On Sat, 6 May 2023 at 08:21, Sebastian Wick <sebastian.wick@xxxxxxxxxx> wrote: > >> > >> On Fri, May 5, 2023 at 10:40 PM Dave Airlie <airlied@xxxxxxxxx> wrote: > >>> > >>> On Fri, 5 May 2023 at 01:23, Simon Ser <contact@xxxxxxxxxxx> wrote: > >>>> > >>>> Hi all, > >>>> > >>>> The goal of this RFC is to expose a generic KMS uAPI to configure the color > >>>> pipeline before blending, ie. after a pixel is tapped from a plane's > >>>> framebuffer and before it's blended with other planes. With this new uAPI we > >>>> aim to reduce the battery life impact of color management and HDR on mobile > >>>> devices, to improve performance and to decrease latency by skipping > >>>> composition on the 3D engine. This proposal is the result of discussions at > >>>> the Red Hat HDR hackfest [1] which took place a few days ago. Engineers > >>>> familiar with the AMD, Intel and NVIDIA hardware have participated in the > >>>> discussion. > >>>> > >>>> This proposal takes a prescriptive approach instead of a descriptive approach. > >>>> Drivers describe the available hardware blocks in terms of low-level > >>>> mathematical operations, then user-space configures each block. We decided > >>>> against a descriptive approach where user-space would provide a high-level > >>>> description of the colorspace and other parameters: we want to give more > >>>> control and flexibility to user-space, e.g. to be able to replicate exactly the > >>>> color pipeline with shaders and switch between shaders and KMS pipelines > >>>> seamlessly, and to avoid forcing user-space into a particular color management > >>>> policy. > >>> > >>> I'm not 100% sold on the prescriptive here, let's see if someone can > >>> get me over the line with some questions later. > >>> > >>> My feeling is color pipeline hw is not a done deal, and that hw > >>> vendors will be revising/evolving/churning the hw blocks for a while > >>> longer, as there is no real standards in the area to aim for, all the > >>> vendors are mostly just doing whatever gets Windows over the line and > >>> keeps hw engineers happy. So I have some concerns here around forwards > >>> compatibility and hence the API design. > >>> > >>> I guess my main concern is if you expose a bunch of hw blocks and > >>> someone comes up with a novel new thing, will all existing userspace > >>> work, without falling back to shaders? > >>> Do we have minimum guarantees on what hardware blocks have to be > >>> exposed to build a useable pipeline? > >>> If a hardware block goes away in a new silicon revision, do I have to > >>> rewrite my compositor? or will it be expected that the kernel will > >>> emulate the old pipelines on top of whatever new fancy thing exists. > >> > >> I think there are two answers to those questions. > > > > These aren't selling me much better :-) > >> > >> The first one is that right now KMS already doesn't guarantee that > >> every property is supported on all hardware. The guarantee we have is > >> that properties that are supported on a piece of hardware on a > >> specific kernel will be supported on the same hardware on later > >> kernels. The color pipeline is no different here. For a specific piece > >> of hardware a newer kernel might only change the pipelines in a > >> backwards compatible way and add new pipelines. > >> > >> So to answer your question: if some hardware with a novel pipeline > >> will show up it might not be supported and that's fine. We already > >> have cases where some hardware does not support the gamma lut property > >> but only the CSC property and that breaks night light because we never > >> bothered to write a shader fallback. KMS provides ways to offload work > >> but a generic user space always has to provide a fallback and this > >> doesn't change. Hardware specific user space on the other hand will > >> keep working with the forward compatibility guarantees we want to > >> provide. > > > > In my mind we've screwed up already, isn't a case to be made for > > continue down the same path. > > > > The kernel is meant to be a hardware abstraction layer, not just a > > hardware exposure layer. The kernel shouldn't set policy and there are > > cases where it can't act as an abstraction layer (like where you need > > a compiler), but I'm not sold that this case is one of those yet. I'm > > open to being educated here on why it would be. > > > > Thanks for raising these points. When I started out looking at color > management I favored the descriptive model. Most other HW vendors > I've talked to also tell me that they think about descriptive APIs > since that allows HW vendors to map that to whatever their HW supports. > > Sebastian, Pekka, and others managed to change my mind about this > but I still keep having difficult questions within AMD. > > Sebastian, Pekka, and Jonas have already done a good job to describe > our reasoning behind the prescriptive model. It might be helpful to > see how different the results of different tone-mapping operators > can look: > > http://helgeseetzen.com/wp-content/uploads/2017/06/HS1.pdf > > According to my understanding all other platforms that have HDR now > have a single compositor. At least that's true for Windows. This allows > driver developers to tune their tone-mapping algorithm to match the > algorithm used by the compositor when offloading plane composition. > > This is not true on Linux, where we have a myriad of compositors for > good reasons, many of which have a different view of how they want color > management to look like. Even if we would come up with an API that lets > compositors define their input, output, scaling, and blending space in > detail it would still not be feasible to describe the minutia of > the tone-mapping algorithms, hence leading to differences in output > when KMS color management is used. > > I am debating whether we need to be serious about a userspace library > (or maybe a user-mode driver) to provide an abstraction from the > descriptive to the prescriptive model. HW vendors need a way to provide > timely support for new HW generations without requiring updates to a > large number of compositors. There are also other vendor side effects to having this in userspace. Will the library have a loader? Will it allow proprietary plugins? Will it allow proprietary reimplementations? What will happen when a vendor wants distros to ship *their* proprietary fork of said library? How would NVIDIA integrate this with their proprietary stack? Dave.
