Hi Christian, going to reply in more detail when I have some more time, so just some quick thoughts for now. Am Mittwoch, dem 02.11.2022 um 12:18 +0100 schrieb Christian König: > Am 01.11.22 um 22:09 schrieb Nicolas Dufresne: > > [SNIP] > > > > But the client is just a video player. It doesn't understand how to > > > > allocate BOs for Panfrost or AMD or etnaviv. So without a universal > > > > allocator (again ...), 'just allocate on the GPU' isn't a useful > > > > response to the client. > > > Well exactly that's the point I'm raising: The client *must* understand > > > that! > > > > > > See we need to be able to handle all restrictions here, coherency of the > > > data is just one of them. > > > > > > For example the much more important question is the location of the data > > > and for this allocating from the V4L2 device is in most cases just not > > > going to fly. > > It feels like this is a generic statement and there is no reason it could not be > > the other way around. > > And exactly that's my point. You always need to look at both ways to > share the buffer and can't assume that one will always work. > > As far as I can see it you guys just allocate a buffer from a V4L2 > device, fill it with data and send it to Wayland for displaying. > > To be honest I'm really surprised that the Wayland guys hasn't pushed > back on this practice already. > > This only works because the Wayland as well as X display pipeline is > smart enough to insert an extra copy when it find that an imported > buffer can't be used as a framebuffer directly. > With bracketed access you could even make this case work, as the dGPU would be able to slurp a copy of the dma-buf into LMEM for scanout. > > I have colleague who integrated PCIe CODEC (Blaize Xplorer > > X1600P PCIe Accelerator) hosting their own RAM. There was large amount of ways > > to use it. Of course, in current state of DMABuf, you have to be an exporter to > > do anything fancy, but it did not have to be like this, its a design choice. I'm > > not sure in the end what was the final method used, the driver isn't yet > > upstream, so maybe that is not even final. What I know is that there is various > > condition you may use the CODEC for which the optimal location will vary. As an > > example, using the post processor or not, see my next comment for more details. > > Yeah, and stuff like this was already discussed multiple times. Local > memory of devices can only be made available by the exporter, not the > importer. > > So in the case of separated camera and encoder you run into exactly the > same limitation that some device needs the allocation to happen on the > camera while others need it on the encoder. > > > > The more common case is that you need to allocate from the GPU and then > > > import that into the V4L2 device. The background is that all dGPUs I > > > know of need the data inside local memory (VRAM) to be able to scan out > > > from it. > > The reality is that what is common to you, might not be to others. In my work, > > most ARM SoC have display that just handle direct scannout from cameras and > > codecs. > > > The only case the commonly fails is whenever we try to display UVC > > created dmabuf, > > Well, exactly that's not correct! The whole x86 use cases of direct > display for dGPUs are broken because media players think they can do the > simple thing and offload all the problematic cases to the display server. > > This is absolutely *not* the common use case you describe here, but > rather something completely special to ARM. It the normal case for a lot of ARM SoCs. That world is certainly not any less big than the x86 dGPU world. A huge number of devices are ARM based set-top boxes and other video players. Just because it is a special case for you doesn't mean it's a global special case. > > > which have dirty CPU write cache and this is the type of thing > > we'd like to see solved. I think this series was addressing it in principle, but > > failing the import and the raised point is that this wasn't the optimal way. > > > > There is a community project called LibreELEC, if you aren't aware, they run > > Khodi with direct scanout of video stream on a wide variety of SoC and they use > > the CODEC as exporter all the time. They simply don't have cases were the > > opposite is needed (or any kind of remote RAM to deal with). In fact, FFMPEG > > does not really offer you any API to reverse the allocation. > > Ok, let me try to explain it once more. It sounds like I wasn't able to > get my point through. > > That we haven't heard anybody screaming that x86 doesn't work is just > because we handle the case that a buffer isn't directly displayable in > X/Wayland anyway, but this is absolutely not the optimal solution. > > The argument that you want to keep the allocation on the codec side is > completely false as far as I can see. > > We already had numerous projects where we reported this practice as bugs > to the GStreamer and FFMPEG project because it won't work on x86 with dGPUs. > And on a lot of ARM SoCs it's exactly the right thing to do. Many codecs need contiguous memory there, so importing a scatter-gather buffer from the GPU via dma-buf will simply not work. > This is just a software solution which works because of coincident and > not because of engineering. By mandating a software fallback for the cases where you would need bracketed access to the dma-buf, you simply shift the problem into userspace. Userspace then creates the bracket by falling back to some other import option that mostly do a copy and then the appropriate cache maintenance. While I understand your sentiment about the DMA-API design being inconvenient when things are just coherent by system design, the DMA- API design wasn't done this way due to bad engineering, but due to the fact that performant DMA access on some systems just require this kind of bracketing. Regards, Lucas
