Hi everyone! Just to let you know. I have just pushed a Branch that includes some first steps to make the h264-stateless encoder working in Gstreamer. The work is based on the VP8 Stateless Encoder patches Benjamin Gaignard created. https://gitlab.freedesktop.org/mgrzeschik/gstreamer/-/commits/1.22/topic/h264-stateless-encoder The codec this is used with, is the rkvenc that can be found on rockchip rk3568. I will send an RFC driver for that in the next weeks after my vacation. On Tue, Jul 11, 2023 at 07:12:41PM +0200, Paul Kocialkowski wrote:
After various discussions following Andrzej's talk at EOSS, feedback from the Media Summit (which I could not attend unfortunately) and various direct discussions, I have compiled some thoughts and ideas about stateless encoders support with various proposals. This is the result of a few years of interest in the topic, after working on a PoC for the Hantro H1 using the hantro driver, which turned out to have numerous design issues. I am now working on a H.264 encoder driver for Allwinner platforms (currently focusing on the V3/V3s), which already provides some usable bitstream and will be published soon. This is a very long email where I've tried to split things into distinct topics and explain a few concepts to make sure everyone is on the same page. # Bitstream Headers Stateless encoders typically do not generate all the bitstream headers and sometimes no header at all (e.g. Allwinner encoder does not even produce slice headers). There's often some hardware block that makes bit-level writing to the destination buffer easier (deals with alignment, etc). The values of the bitstream headers must be in line with how the compressed data bitstream is generated and generally follow the codec specification. Some encoders might allow configuring all the fields found in the headers, others may only allow configuring a few or have specific constraints regarding which values are allowed. As a result, we cannot expect that any given encoder is able to produce frames for any set of headers. Reporting related constraints and limitations (beyond profile/level) seems quite difficult and error-prone. So it seems that keeping header generation in-kernel only (close to where the hardware is actually configured) is the safest approach.
For the case with the rkvenc, the headers are also not created by the kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions that are part of the codecparsers module.
# Codec Features Codecs have many variable features that can be enabled or not and specific configuration fields that can take various values. There is usually some top-level indication of profile/level that restricts what can be used. This is a very similar situation to stateful encoding, where codec-specific controls are used to report and set profile/level and configure these aspects. A particularly nice thing about it is that we can reuse these existing controls and add new ones in the future for features that are not yet covered. This approach feels more flexible than designing new structures with a selected set of parameters (that could match the existing controls) for each codec.
I back the Idea of generic profiles instead of explicit configuration from the usersapace point of view. The parameterization works like this: Read the sane default parameter set from the driver. Modify the parameters based on the userspace decisions. - (currently hardcoded and not based on any user input) Write the updated parameters back to the driver.
# Reference and Reconstruction Management
<snip>
# Frame Types
<snip>
# Rate Control
<snip>
# Regions of Interest
<snip> Since the first shot of the rkvenc is I-Frame only code, these other topics are currently undefined and unimplemented in the Gstreamer stack. Regards, Michael -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Attachment:
signature.asc
Description: PGP signature
