On Wed, Jun 05, 2024 at 10:59:11AM -0300, Jason Gunthorpe wrote: > On Tue, Jun 04, 2024 at 04:56:57PM -0700, Dan Williams wrote: > > * Introspection / validation: Subsystem community needs to be able to > > audit behavior after the fact. > > > > To me this means even if the kernel is letting a command through based > > on the stated Command Effect of "Configuration Change after Cold Reset" > > upstream community has a need to be able to read the vendor > > specification for that command. I.e. commands might be vendor-specific, > > but never vendor-private. I see this as similar to the requirement for > > open source userspace for sophisticated accelerators. > > I'm less hard on this. As long as reasonable open userspace exists I > think it is fine to let other stuff through too. I can appreciate the > DRM stance on this, but IMHO, there is meaningfully more value for open > source in trying get an open Vulkan implementation vs blocking users > from reading their vendor'd diagnostic SI values. > > I don't think we should get into some kind of extremism and insist > that every single bit must be documented/standardized or Linux won't > support it. I figured it might be useful to paint what we do in DRM with a bit more nuance. In the principles, we're indeed fairly radical in what we require, but in practice we aim for a much more pragmatic approach in what we merge. There's two major axis here: 1. One is ecosystem maturity. One end is 3d, with vulkan as the clear industry standard, and an upstream full-featured userspace driver in mesa3d is the only technically reasonable choice. And all gpu vendors agree and by this year even nvidia started hiring an upstream team. But this didn't happen magically overnight, it took 1-2 decades of background discussions and tactical push&pulling to get there. The other end is currently AI accelerators. It's a complete mess, where across the platform (client, edge, cloud), customer and vendor dimension every point has a different stack. And the problem is so obvious that everyone is working to fix this, which means currently https://xkcd.com/927/ is happening in parallel. Just to get things going we're accepting pretty much anything that's a notch above total garbage for userspace and for merging into the kernel. 2. The other part is how much it impacts applications. If you can't run the same application across different vendors, the case for an upstream stack becomes a lot weaker. At the other end is infrastructure enabling like device configuration, error handling and recovery, hw debugging and reliablity/health reporting. That's a lot more vendor specific in nature and needs to be customized anyway per deployement. And only much higher in the stack, maybe in k8s, can a technically reasonable unification even happen. So again we're much more lenient about infrastructure enabling and uapi than stuff applications will use directly. Currently that's enough of a mess in drm that I feel like enforcing something like fwctl is still too much. But maybe once fwctl is established with other subsystems/devices we can start the conversations with vendors to get this going a few years down the road. Both together mean we land a lot of code that's questionable at best, clear garbage at worst. But since we've been in the merging garbage business just to get things going for decades, we've become pretty good at dealing with the kernel internal and uapi fallout, some say too good. But personally I don't think there's a path to where we are with 3d/vulkan that doesn't go through years of this kind of suck, and very much merged into upstream kind of suck. For all the concerns about trusting vendors/devices to not abuse very broad uapi interfaces: Modern accelerator command submission boils down to "run this context at this $addr", and the kernel never ever directly sees anything more fly by. That's the same interface you need for a no-op job as a full blown AI workload, so in theory maximal abuse potential. In practice, it doesn't seem to be an issue, at least not beyond the intentionally pragmatic choices where we merge kernel code with known sub-par/incomplete userspace. I'm not sure why, but to my knowledge all attempts to break the spirit of our userspace rules while following the letter die in vendor-internal discussions, at least for all the established upstream driver teams. And for new ones it takes years of private chats to get them going and fully established in upstream anyway. Maybe one reason we have a bit an extremist reputation is that all the public takes are the radical principled requirements, while the actual pragmatic discussions mostly happen in private. tldr; fwctl as I understand it feels like a bridge to far for drm today, but I'd very much like someone else to make this happen so we could eventually push towards adoption too. Cheers, Sima -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
