Le jeudi 18 avril 2019 à 10:18 +0200, Daniel Vetter a écrit : > > It would be cool if both could be used concurrently and not just return > > -EBUSY when the device is used with the other subsystem. > > We live in this world already :-) I think there's even patches (or merged > already) to add fences to v4l, for Android. This work is currently suspended. It will require some feature on DRM display to really make this useful, but there is also a lot of challanges in V4L2. In GFX space, most of the use case are about rendering as soon as possible. Though, in multimedia we have two problems, we need to synchronize the frame rendering with the audio, and output buffers may comes out of order due to how video CODECs are made. In the first, we'd need a mechanism where we can schedule a render at a specific time or vblank. We can of course already implement this in software, but with fences, the scheduling would need to be done in the driver. Then if the fence is signalled earlier, the driver should hold on until the delay is met. If the fence got signalled late, we also need to think of a workflow. As we can't schedule more then one render in DRM at one time, I don't really see yet how to make that work. For the second, it's complicated on V4L2 side. Currently we signal buffers when they are ready in the display order. With fences, we receive early pairs buffer and fence (in decoding order). There exist cases where reordering is done by the driver (stateful CODEC). We cannot schedule these immediately we would need a new mechanism to know which one come next. If we just reuse current mechnism, it would void the fence usage since the fence will always be signalled by the time it reaches DRM or other v4l2 component. There also other issues, for video capture pipeline, if you are not rendering ASAP, you need the HW timestamp in order to schedule. Again, we'd get the fence early, but the actual timestamp will be signalled at the very last minutes, so we also risk of turning the fence into pure overhead. Note that as we speak, I have colleagues who are experimenting with frame timestamp prediction that slaves to the effective timestamp (catching up over time). But we still have issues when the capture driver skipped a frame (missed a capture window). I hope this is useful reflection data, Nicolas
