On 11/07/2023 20:18, Nicolas Dufresne wrote: > Le mardi 11 juillet 2023 à 19:12 +0200, Paul Kocialkowski a écrit : >> Hi everyone! >> >> After various discussions following Andrzej's talk at EOSS, feedback from the >> Media Summit (which I could not attend unfortunately) and various direct >> discussions, I have compiled some thoughts and ideas about stateless encoders >> support with various proposals. This is the result of a few years of interest >> in the topic, after working on a PoC for the Hantro H1 using the hantro driver, >> which turned out to have numerous design issues. >> >> I am now working on a H.264 encoder driver for Allwinner platforms (currently >> focusing on the V3/V3s), which already provides some usable bitstream and will >> be published soon. >> >> This is a very long email where I've tried to split things into distinct topics >> and explain a few concepts to make sure everyone is on the same page. >> >> # Bitstream Headers >> >> Stateless encoders typically do not generate all the bitstream headers and >> sometimes no header at all (e.g. Allwinner encoder does not even produce slice >> headers). There's often some hardware block that makes bit-level writing to the >> destination buffer easier (deals with alignment, etc). >> >> The values of the bitstream headers must be in line with how the compressed >> data bitstream is generated and generally follow the codec specification. >> Some encoders might allow configuring all the fields found in the headers, >> others may only allow configuring a few or have specific constraints regarding >> which values are allowed. >> >> As a result, we cannot expect that any given encoder is able to produce frames >> for any set of headers. Reporting related constraints and limitations (beyond >> profile/level) seems quite difficult and error-prone. >> >> So it seems that keeping header generation in-kernel only (close to where the >> hardware is actually configured) is the safest approach. > > This seems to match with what happened with the Hantro VP8 proof of concept. The > encoder does not produce the frame header, but also, it produces 2 encoded > buffers which cannot be made contiguous at the hardware level. This notion of > plane in coded data wasn't something that blended well with the rest of the API > and we didn't want to copy in the kernel while the userspace would also be > forced to copy to align the headers. Our conclusion was that it was best to > generate the headers and copy both segment before delivering to userspace. I > suspect this type of situation will be quite common. > >> >> # Codec Features >> >> Codecs have many variable features that can be enabled or not and specific >> configuration fields that can take various values. There is usually some >> top-level indication of profile/level that restricts what can be used. >> >> This is a very similar situation to stateful encoding, where codec-specific >> controls are used to report and set profile/level and configure these aspects. >> A particularly nice thing about it is that we can reuse these existing controls >> and add new ones in the future for features that are not yet covered. >> >> This approach feels more flexible than designing new structures with a selected >> set of parameters (that could match the existing controls) for each codec. > > Though, reading more into this emails, we still have a fair amount of controls > to design and add, probably some compound controls too ? I expect that for stateless encoders support for read-only requests will be needed: https://patchwork.linuxtv.org/project/linux-media/list/?series=5647 I worked on that in the past together with dynamic control arrays. The dynamic array part was merged, but the read-only request part wasn't (there was never a driver that actually needed it). I don't know if that series still applies, but if there is a need for it then I can rebase it and post an RFCv3. Regards, Hans
