2024年7月31日(水) 22:16 Daniel Vetter <daniel.vetter@xxxxxxxx>: > > On Wed, 31 Jul 2024 at 13:55, Sakari Ailus <sakari.ailus@xxxxxx> wrote: > > This is also very different from GPUs or accel devices that are built to be > > user-programmable. If I'd compare ISPs to different devices, then the > > closest match would probably be video codecs -- which also use V4L2. > > Really just aside, but I figured I should correct this. DRM supports > plenty of video codecs. They're all tied to gpus, but the real reason > really is that the hw has decent command submission support so that > running the entire codec in userspace except the basic memory and > batch execution and synchronization handling in the kernel is a > feasible design. FWIW, V4L2 also has an interface for video decoders that require bitstream processing in software, it's called the V4L2 Stateless Decoder interface [1]. It defines low level data structures that map directly to the particular codec specification, so the kernel interface is generic and the userspace doesn't need to have hardware-specific components. Hardware that consumes command buffers can be supported simply by having the kernel driver fill the command buffers as needed (as opposed to writing the registers directly). On the other hand, DRM also has the fixed function (i.e. V4L2-alike) KMS interface for display controllers, rather than a command buffer passthrough, even though some display controllers actually are driven by command buffers. So arguably it's possible and practical to do both command buffer-based and fixed interfaces for both display controllers and video codecs. Do you happen to know some background behind why one or the other was chosen for each of them in DRM? For how it applies to ISPs, there are both types of ISPs out in the wild, some support command buffers, while some are programmed directly via registers. For the former, I can see some loss of flexibility if the command buffers are hidden behind a fixed function API, because the userspace would only be able to do what the kernel driver supports internally, which could make some use case-specific optimizations very challenging if not impossible. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-stateless-decoder.html > And actually good, because your kernel wont ever blow > up trying to parse complex media formats because it just doesn't. > -Sima > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch >
