Re: Stateless Encoding uAPI Discussion and Proposal

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi everyone!

Just to let you know. I have just pushed a Branch that includes some first
steps to make the h264-stateless encoder working in Gstreamer. The work is
based on the VP8 Stateless Encoder patches Benjamin Gaignard created.

https://gitlab.freedesktop.org/mgrzeschik/gstreamer/-/commits/1.22/topic/h264-stateless-encoder

The codec this is used with, is the rkvenc that can be found on rockchip
rk3568. I will send an RFC driver for that in the next weeks after my vacation.

On Tue, Jul 11, 2023 at 07:12:41PM +0200, Paul Kocialkowski wrote:
After various discussions following Andrzej's talk at EOSS, feedback from the
Media Summit (which I could not attend unfortunately) and various direct
discussions, I have compiled some thoughts and ideas about stateless encoders
support with various proposals. This is the result of a few years of interest
in the topic, after working on a PoC for the Hantro H1 using the hantro driver,
which turned out to have numerous design issues.

I am now working on a H.264 encoder driver for Allwinner platforms (currently
focusing on the V3/V3s), which already provides some usable bitstream and will
be published soon.

This is a very long email where I've tried to split things into distinct topics
and explain a few concepts to make sure everyone is on the same page.

# Bitstream Headers

Stateless encoders typically do not generate all the bitstream headers and
sometimes no header at all (e.g. Allwinner encoder does not even produce slice
headers). There's often some hardware block that makes bit-level writing to the
destination buffer easier (deals with alignment, etc).

The values of the bitstream headers must be in line with how the compressed
data bitstream is generated and generally follow the codec specification.
Some encoders might allow configuring all the fields found in the headers,
others may only allow configuring a few or have specific constraints regarding
which values are allowed.

As a result, we cannot expect that any given encoder is able to produce frames
for any set of headers. Reporting related constraints and limitations (beyond
profile/level) seems quite difficult and error-prone.

So it seems that keeping header generation in-kernel only (close to where the
hardware is actually configured) is the safest approach.

For the case with the rkvenc, the headers are also not created by the
kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions
that are part of the codecparsers module.

# Codec Features

Codecs have many variable features that can be enabled or not and specific
configuration fields that can take various values. There is usually some
top-level indication of profile/level that restricts what can be used.

This is a very similar situation to stateful encoding, where codec-specific
controls are used to report and set profile/level and configure these aspects.
A particularly nice thing about it is that we can reuse these existing controls
and add new ones in the future for features that are not yet covered.

This approach feels more flexible than designing new structures with a selected
set of parameters (that could match the existing controls) for each codec.

I back the Idea of generic profiles instead of explicit configuration
from the usersapace point of view.

The parameterization works like this:

Read the sane default parameter set from the driver.
Modify the parameters based on the userspace decisions.
- (currently hardcoded and not based on any user input)
Write the updated parameters back to the driver.

# Reference and Reconstruction Management
<snip>

# Frame Types
<snip>

# Rate Control
<snip>

# Regions of Interest
<snip>

Since the first shot of the rkvenc is I-Frame only code, these other topics are
currently undefined and unimplemented in the Gstreamer stack.


Regards,
Michael

--
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Attachment: signature.asc
Description: PGP signature


[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]

  Powered by Linux