On 2023-11-08 11:19, Pekka Paalanen wrote: > On Wed, 8 Nov 2023 09:31:17 -0500 > Harry Wentland <harry.wentland@xxxxxxx> wrote: > >> On 2023-11-08 06:40, Sebastian Wick wrote: >>> On Wed, Nov 8, 2023 at 11:16 AM Pekka Paalanen <ppaalanen@xxxxxxxxx> wrote: >>>> >>>> On Tue, 7 Nov 2023 11:58:26 -0500 >>>> Harry Wentland <harry.wentland@xxxxxxx> wrote: >>>> >>>>> On 2023-11-07 04:55, Pekka Paalanen wrote: >>>>>> On Mon, 6 Nov 2023 11:19:27 -0500 >>>>>> Harry Wentland <harry.wentland@xxxxxxx> wrote: >>>>>> >>>>>>> On 2023-10-20 06:36, Pekka Paalanen wrote: >>>>>>>> On Thu, 19 Oct 2023 10:56:40 -0400 >>>>>>>> Harry Wentland <harry.wentland@xxxxxxx> wrote: >>>>>>>> >>>>>>>>> On 2023-10-10 12:13, Melissa Wen wrote: >>>>>>>>>> O 09/08, Harry Wentland wrote: >>>>>>>>>>> Signed-off-by: Harry Wentland <harry.wentland@xxxxxxx> >>>>>>>> >>>>>>>> ... >>>>>>>> >>>>>>>>>> Also, with this new plane API in place, I understand that we will >>>>>>>>>> already need think on how to deal with the mixing between old drm color >>>>>>>>>> properties (color encoding and color range) and these new way of setting >>>>>>>>>> plane color properties. IIUC, Pekka asked a related question about it >>>>>>>>>> when talking about CRTC automatic RGB->YUV (?) >>>>>>>>>> >>>>>>>>> >>>>>>>>> We'll still need to confirm whether we'll want to deprecate these >>>>>>>>> existing properties. If we do that we'd want a client prop. Things >>>>>>>>> should still work without deprecating them, if drivers just pick up >>>>>>>>> after the initial encoding and range CSC. >>>>>>>>> >>>>>>>>> But realistically it might be better to deprecate them and turn them >>>>>>>>> into explicit colorops. >>>>>>>> >>>>>>>> The existing properties would need to be explicitly reflected in the >>>>>>>> new pipelines anyway, otherwise there would always be doubt at which >>>>>>>> point of a pipeline the old properties apply, and they might even >>>>>>>> need to change positions between pipelines. >>>>>>>> >>>>>>>> I think it is simply easier to just hide all old color related >>>>>>>> properties when userspace sets the client-cap to enable pipelines. The >>>>>>>> problem is to make sure to hide all old properties on all drivers that >>>>>>>> support the client-cap. >>>>>>>> >>>>>>>> As a pipeline must be complete (describe everything that happens to >>>>>>>> pixel values), it's going to be a flag day per driver. >>>>>>>> >>>>>>>> Btw. the plane FB YUV->RGB conversion needs a colorop in every pipeline >>>>>>>> as well. Maybe it's purely informative and non-configurable, keyed by >>>>>>>> FB pixel format, but still. >>>>>>>> >>>>>>>> We also need a colorop to represent sample filtering, e.g. bilinear, >>>>>>>> like I think Sebastian may have mentioned in the past. Everything >>>>>>>> before the sample filter happens "per tap" as Joshua Ashton put it, and >>>>>>>> everything after it happens on the sample that was computed as a >>>>>>>> weighted average of the filter tap inputs (texels). >>>>>>>> >>>>>>>> There could be colorops other than sample filtering that operate on >>>>>>>> more than one sample at a time, like blur or sharpness. There could >>>>>>>> even be colorops that change the image size like adding padding that >>>>>>>> the following colorop hardware requires, and then yet another colorop >>>>>>>> that clips that padding away. For an example, see >>>>>>>> https://lists.freedesktop.org/archives/dri-devel/2023-October/427015.html >>>>>>>> >>>>>>>> If that padding and its color can affect the pipeline results of the >>>>>>>> pixels near the padding (e.g. some convolution is applied with them, >>>>>>>> which may be the reason why padding is necessary to begin with), then >>>>>>>> it would be best to model it. >>>>>>>> >>>>>>> >>>>>>> I hear you but I'm also somewhat shying away from defining this at this point. >>>>>> >>>>>> Would you define them before the new UAPI is released though? >>>>>> >>>>>> I agree there is no need to have them in this patch series, but I think >>>>>> we'd hit the below problems if the UAPI is released without them. >>>>>> >>>>>>> There are already too many things that need to happen and I will focus on the >>>>>>> actual color blocks (LUTs, matrices) first. We'll always be able to add a new >>>>>>> (read-only) colorop type to define scaling and tap behavior at any point and >>>>>>> a client is free to ignore a color pipeline if it doesn't find any tap/scale >>>>>>> info in it. >>>>>> >>>>>> How would userspace know to look for tap/scale info, if there is no >>>>>> upstream definition even on paper? >>>>>> >>>>> >>>>> So far OSes did not care about this. Whether that's good or bad is >>>>> something everyone can answer for themselves. >>>>> >>>>> If you write a compositor and really need this you can just ignore >>>>> color pipelines that don't have this, i.e., you'll probably want >>>>> to wait with implementing color pipeline support until you have what >>>>> you need from DRM/KMS. >>>>> >>>>> This is not to say I don't want to have support for Weston. But I'm >>>>> wondering if we place too much importance on getting every little >>>>> thing figured out whereas we could be making forward progress and >>>>> address more aspects of a color pipeline in the future. There is a >>>>> reason gamescope has made a huge difference in driving the color >>>>> management work forward. >>>>> >>>>>> And the opposite case, if someone writes userspace without tap/scale >>>>>> colorops, and then drivers add those, and there is no pipeline without >>>>>> them, because they always exist. Would that userspace disregard all >>>>>> those pipelines because it does not understand tap/scale colorops, >>>>>> leaving no usable pipelines? Would that not be kernel regressing >>>>>> userspace? >>>>>> >>>>> >>>>> The simple solution is to leave previously advertised pipelines >>>>> untouched and add a new version that does include scaling information. >>>>> >>>>>> If the kernel keeps on exposing pipelines without the colorops, it >>>>>> fails the basic promise of the whole design: that all pixel value >>>>>> affecting operations are at least listed if not controllable. >>>>>> >>>>>> How will we avoid painting ourselves in a corner? >>>>>> >>>>>> Maybe we need a colorop for "here be dragons" documented as having >>>>>> unknown and unreliable effects, until driver authors are sure that >>>>>> everything has been modelled in the pipeline and there are no unknowns? >>>>>> Or a flag on the pipelines, if we can have that. Then we can at least >>>>>> tell when the pipeline does not fulfil the basic promise. >>>>>> >>>>> >>>>> The will always be dragons at some level. >>>> >>>> Do I understand right that the goal of fully understood color pipelines >>>> is a lost cause? >>>> >>>> That every pipeline might always have something unknown and there is no >>>> way for userspace to know if it does? Maybe because driver developers >>>> don't know either? >>>> >>>> By something unknown I refer to anything outside of basic precision >>>> issues. Doing interpolation or mixing of inputs on the wrong side of a >>>> known non-linear colorop, for example. >>> >>> I don't think that's the case. Hardware vendors should understand the >>> hardware and exposing everything that affects the values is the goal >>> here. There will be a transitional period where the pipelines might >>> not expose every detail yet but that's fine. It's better than what we >>> have now and should become even better with time. It would maybe be >>> helpful in the future to have a cap, or property, or whatever, to >>> indicate that the pipelines are "complete" descriptions of what >>> happens to the values but we can discuss it when it becomes relevant. >>> >> >> I agree, for the most part. But how do you then define "complete" if >> you exclude precision issues? > > If someone can develop some kind of precision indication in the UAPI, > we might be able to answer that question then. > >>>> An incremental UAPI development approach is fine by me, meaning that >>>> pipelines might not be complete at first, but I believe that requires >>>> telling userspace whether the driver developers consider the pipeline >>>> complete (no undescribed operations that would significantly change >>>> results from the expected results given the UAPI exposed pipeline). >>>> >>>> The prime example of what I would like to know is that if a FB >>>> contains PQ-encoded image and I use a color pipeline to scale that >>>> image up, will the interpolation happen before or after the non-linear >>>> colorop that decodes PQ. That is a significant difference as pointed >>>> out by Joshua. >>>> >> >> That's fair and I want to give that to you. My concern stems from >> the sentiment that I hear that any pipeline that doesn't explicitly >> advertise this is useless. I don't agree there. Let's not let perfect >> be the enemy of good. > > It's up to the use case. The policy of what is sufficient should reside > in userspace. > > What about matching compositor shader composition with KMS? > > Can we use that as a rough precision threshold? If userspace implements > the exact same color pipeline as the KMS UAPI describes, then that and > the KMS composited result should be indistinguishable in side-by-side > or alternating visual inspection on any monitor in isolation. > > Did this whole effort not start from wanting to off-load things to > display hardware but still maintain visual equivalence to software/GPU > composition? > I agree with you and I want all that as well. All I'm saying is that every userspace won't have the same policy of what is sufficient. Just because Weston has a very high threshold doesn't mean we can't move forward with a workable solution for other userspace, as long as we don't do something that prevents us from doing the perfect solution eventually. And yes, I do want a solution that works for Weston and hear your comments on what that requires. Harry > > Thanks, > pq
