On Tue, 09 May 2023 20:22:30 +0000 Simon Ser <contact@xxxxxxxxxxx> wrote: > On Tuesday, May 9th, 2023 at 21:53, Dave Airlie <airlied@xxxxxxxxx> wrote: > > > There are also other vendor side effects to having this in userspace. > > > > Will the library have a loader? > > Will it allow proprietary plugins? > > Will it allow proprietary reimplementations? > > What will happen when a vendor wants distros to ship their > > proprietary fork of said library? > > > > How would NVIDIA integrate this with their proprietary stack? > > Since all color operations exposed by KMS are standard, the library > would just be a simple one: no loader, no plugin, no proprietary pieces, > etc. Hi, that's certainly the long term goal, and *if* Linux software can in any way guide hardware design, then I believe it is an achievable goal. I understand "standard" as something that is widely implemented in various hardware rather than only "well-defined and documented and free to implement in any hardware if its vendor cared". However, like I mentioned in my other reply to Steven, I expect there will be a time period when each hardware has custom processing blocks no other hardware (same or different vendor) has. I might not call them outright proprietary though, because in order have them exposed via UAPI, the mathematical model of the processing block must be documented with its UAPI. This means there cannot be secrets on what the hardware does, which means there cannot be a requirement for secret sauce in userspace either. I wonder if we can also require new COLOROP elements to be freely implementable by anyone anywhere in any way one wants? Or do kernel maintainers just need to NAK proposals for elements that might not be that free? Anything that is driver-chosen or automatic can also be proprietary, because today's KMS UAPI rules do not require documenting how automatic features work, e.g. the existing YUV-to-RGB conversion. Hardware could have whatever wild skin tone improvement algorithms hidden in there for example. In this new proposal, there cannot be undocumented behaviour. Dave, if we went with a descriptive UAPI model, everything behind it could be proprietary and secret. That's not open in the least. On Wed, 10 May 2023 at 00:31, Harry Wentland <harry.wentland@xxxxxxx> wrote: > > I am debating whether we need to be serious about a userspace library > (or maybe a user-mode driver) to provide an abstraction from the > descriptive to the prescriptive model. HW vendors need a way to provide > timely support for new HW generations without requiring updates to a > large number of compositors. Drivers can always map old COLOROP elements to new style hardware blocks if they can achieve the same mathematical operation up to whatever precision was promised before. I think that should be the main form of supporting hardware evolution. Then also add new alternative COLOROP elements that can better utilize the hardware block. Naturally that means that COLOROP elements must be designed to be somewhat generic to have a reasonable life time. They cannot be extremely tightly married to the hardware implementation that might cease to exist in the very next hardware revision. Let's say some vendor has a hardware block that does a series of operations in an optimized fashion, perhaps with hardwired constants. This is exposed as a custom COLOROP element. The next hardware revision no longer has this block, but it has a bunch of new blocks that can produce the exact same result. The driver for this hardware can expose two different pipelines: one using the old COLOROP element, and another using a bunch of other COLOROP elements which exposes the new flexibility of the hardware design better. If userspace chooses the former pipeline, the driver just programs the bunch of blocks to behave accordingly. Hopefully the other COLOROP elements will be more standard than the old element. Over time, I hope this causes an evolution where hardware implements only the most standard COLOROP elements, and special-case compound elements will eventually fall out of use over the decades. Thanks, pq
Attachment:
pgpaNG7nwdE5h.pgp
Description: OpenPGP digital signature
