Hi Ricardo, On Thu, Sep 08, 2022 at 08:13:57PM +0200, Ricardo Ribalda wrote: > On Thu, 8 Sept 2022 at 20:13, Ricardo Ribalda wrote: > > On Thu, 8 Sept 2022 at 17:34, Maxime Ripard wrote: > > > On Thu, Sep 08, 2022 at 06:16:40PM +0300, Laurent Pinchart wrote: > > > > On Thu, Sep 08, 2022 at 04:59:05PM +0200, Maxime Ripard wrote: > > > > > On Thu, Sep 08, 2022 at 05:14:41PM +0300, Laurent Pinchart wrote: > > > > > > On Thu, Sep 08, 2022 at 10:08:46AM +0200, Maxime Ripard wrote: > > > > > > > Hi Ricardo, > > > > > > > > > > > > > > On Thu, Sep 08, 2022 at 09:11:11AM +0200, Ricardo Ribalda wrote: > > > > > > > > > - Still on slide 16, V4L2 as an API is usable without disclosing vendor > > > > > > > > > IP. What is not possible is upstreaming a driver. I don't see this as > > > > > > > > > significantly different between V4L2 and the new API proposal. I > > > > > > > > > expect this to be discussed on Monday. > > > > > > > > > > > > > > > > I am only considering upstream drivers. There is not much to discuss > > > > > > > > for downstream or closed drivers :) > > > > > > > > > > > > > > Are we really discussing upstream *drivers*? If anything, it looks like > > > > > > > the Kcam proposal moves most of the drivers out of upstream. > > > > > > > > > > > > Given that the API proposal sets at a significant lower level than V4L2 > > > > > > in the stack, the concept of "userspace driver" (I meant it in the sense > > > > > > of GPU support in mesa) plays a bigger role. It would be good to clarify > > > > > > what is meant by "driver" and maybe use the term "kernel driver" when > > > > > > only the kernel part is covered, to avoid misunderstandings. > > > > > > > > > > I think there's a bit of a misunderstanding about what exactly is in a > > > > > DRM driver, and what is in Mesa. > > > > > > > > > > Mesa doesn't program the hardware at all, it's merely a glorified > > > > > compiler. It's not more of a driver than GCC is an OS. Most importantly > > > > > for our discussion, Mesa doesn't perform any kind of register access (or > > > > > register access request), only the (kernel) driver does that. > > > > > > > > Mesa compiles shaders, but also more generally produces command streams > > > > that are passed as blobs to the DRM driver, which then forwards them to > > > > the device with as little processing and validation as possible (when > > > > the device is designed with multi-clients in mind, that processing and > > > > validation can be reduced a lot). > > > > > > That's true, but at no point in time is the CPU ever touches that > > > command stream blob in the case of DRM... > > > > As Laurent says, the latest hardware is very similar to GPUs, you pass > > a set of commands to a firmware that does the actual R/W to the > > hardware. *Some* of the latest hardware. There are new SoCs getting to the market today with GPUs that are fully programmed from the kernel, and even more that are fully programmable from the kernel even if the stack provided by the SoC vendor has a firmware that takes care of programming the ISP. One thing that isn't clear in your proposal is where the line is drawn. Passing a blob to the ISP firmware involves some kind of communication mechanism, which ultimately deals with hardware registers somewhere. It's not clear if those registers are part of the blob that userspace passes to the kernel. I'd assume not, but clarifying where the line is would be useful. > > For hardware that is a register set, the vendor should have a good > > idea about what kind of validation should be needed: raw access (deny > > list) or more abstracted (allow list). > > > > The most critical part is the DMA, and that will always be abstracted. > > Also I doubt that we will have new hardware without an IOMMU, so we > > have the same layers of security as today. I know of SoCs in the making that have ISPs and no IOMMU. > > > > Recent ISPs have a similar architecture, with a set of registers used > > > > to communicate with the ISP firmware, and then most of the hardware > > > > registers for the actual image processing blocks being programmed > > > > based from the command stream. "Command stream" may not be a very good > > > > term for ISPs as it's not really a stream of commands, but > > > > conceptually, we're dealing with a blob that is computed by userspace. > > > > > > ... while in Kcam, the CPU knows and will interpret that command stream. > > > Maybe not in all cases, but it's still a significant difference. > > > > > > If we had to draw a parallel with something else in the kernel, it looks > > > way more like eBPF or the discussion we had on where to parse the > > > bitstreams for stateless codecs. > > > > > > The first one has been severely constrained to avoid the issues we've > > > raised, and we all know how the second one went. > > > > In eBPF, you are moving some user code to the kernel, with an unstable API. > > > > In KCAM, (and in DRM), you let the user build a set of operations, > > that you pass to the kernel via a stable API, then it is validated and > > scheduled by the kernel. > > > > X11 was much more bizarre, the GPIO iomem was remapped into userspace. > > s/GPIO/GPU/ ;) -- Regards, Laurent Pinchart
