Hi Nicolas, On Mon 24 Jul 23, 10:03, Nicolas Dufresne wrote: > Le vendredi 21 juillet 2023 à 20:19 +0200, Michael Grzeschik a écrit : > > > As a result, we cannot expect that any given encoder is able to produce frames > > > for any set of headers. Reporting related constraints and limitations (beyond > > > profile/level) seems quite difficult and error-prone. > > > > > > So it seems that keeping header generation in-kernel only (close to where the > > > hardware is actually configured) is the safest approach. > > > > For the case with the rkvenc, the headers are also not created by the > > kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions > > that are part of the codecparsers module. > > One level of granularity we can add is split headers (like SPS/PPS) and > slice/frame headers. Do you mean asking the driver to return a buffer with only SPS/PPS and then return another buffer with the slice/frame header? Looks like there's already a control for it: V4L2_CID_MPEG_VIDEO_HEADER_MODE which takes either - V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE: looks like what you're suggesting - V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME: usual case So that could certainly be supported to easily allow userspace to stuff extra NALUs in-between. > It remains that in some cases, like HEVC, when the slice > header is byte aligned, it can be nice to be able to handle it at application > side in order to avoid limiting SVC support (and other creative features) by our > API/abstraction limitations. Do you see something in the headers that we expect the kernel to generate that would need specific changes to support features like SVC? From what I can see there's a svc_extension_flag that's only set for specific NALUs (prefix_nal_unit/lice_layer_extension) so these could be inserted by userspace. Also I'm not very knowledgeable about SVC so it's not very clear to me if it's possible to take an encoder that doesn't support SVC and turn the resulting stream into something SVC-ready by adding extra NAL units or if the encoder should be a lot more involved. Also do you know if we have stateful codecs supporting SVC? > I think a certain level of "per CODEC" reasoning is > also needed. Just like, I would not want to have to ask the kernel to generate > user data SEI and other in-band data. Yeah it looks like there is definitely a need for adding extra NALUs from userspace without passing that data to the kernel. Cheers, Paul -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com
Attachment:
signature.asc
Description: PGP signature
