Hi Sakari, Thank you for the patch. On Wed, Dec 07, 2022 at 12:58:09PM +0200, Sakari Ailus wrote: > Document how setting up routes interacts with formats and selections. > > Signed-off-by: Sakari Ailus <sakari.ailus@xxxxxxxxxxxxxxx> > --- > since v1: > > - Rewording changes based on Tomi's comments. In particular stricter > separation between "stream" and "stream ID". > > .../userspace-api/media/v4l/dev-subdev.rst | 50 ++++++++++++++----- > 1 file changed, 37 insertions(+), 13 deletions(-) > > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst > index 5075b1828b32d..92710bb9a0caa 100644 > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst > @@ -406,6 +406,8 @@ pixel array is not rectangular but cross-shaped or round. The maximum > size may also be smaller than the BOUNDS rectangle. > > > +.. _format-propagation: > + > Order of configuration and format propagation > --------------------------------------------- > > @@ -507,12 +509,12 @@ source pads. > Streams, multiplexed media pads and internal routing > ---------------------------------------------------- > > -Commonly V4L2 subdevices support only separate video streams, that is, only a > -single stream can pass through a media link and a media pad. Thus each pad > -contains a format configuration for that single stream. In some cases a subdev > -can do stream processing and split a stream into two or compose two streams > -into one, but the inputs and outputs for the subdev are still a single stream > -per pad. > +Simple V4L2 subdevices do not support multiple, unrelated video streams, > +and only a single stream can pass through a media link and a media pad. > +Thus each pad contains a format and selection configuration for that > +single stream. A subdev can do stream processing and split a stream into > +two or compose two streams into one, but the inputs and outputs for the > +subdev are still a single stream per pad. > > Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, multiple > data streams are transmitted on the same bus, which is represented by a media > @@ -539,14 +541,33 @@ streams from one end of the link to the other, and subdevices have routing > tables which describe how the incoming streams from sink pads are routed to the > source pads. > > -A stream ID (often just "stream") is a media link-local identifier for a stream. > +A stream ID is a media link-local identifier for a stream. > In other words, a particular stream ID must exist on both sides of a media > link, but another stream ID can be used for the same stream at the other side > -of the subdevice. > +of the subdevice. The same stream ID is used to refer to the stream on > +both pads of the link on all ioctls operating on pads. > + > +A stream at a specific point in the media pipeline is identified by the > +sub-devdev and a pad ID-stream ID pair. For subdevices that do not support > +multiplexed streams the 'stream' field is always 0. > + > +Interaction between routes, formats and selections s/selections/selection rectangles/ > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +The addition of routes to the V4L2 sub-device interface moves the > +sub-device formats and selections from pads to pad ID-stream ID pairs. This reads as "pad" "ID-stream" "ID" "pairs". Even knowing how the API behaves, I had trouble understanding this. > +Besides the usual pad ID, also the stream ID needs to be provided for > +setting formats and selections. > + > +Instead of the sub-device wide merging of streams from all source pads > +towards all sink pads, data flows for each route are separate from each > +other. Any number of routes from streams on sink pads towards streams on > +source pads is allowed, to the extent supported by drivers. For every > +stream on a sink pad, however, only a single route is allowed. This explains routes, not format and selection rectangles. > -A stream at a specific point in the media pipeline is identified with the > -subdev and a (pad, stream) pair. For subdevices that do not support > -multiplexed streams the 'stream' is always 0. > +Stream IDs, part of routes, are configured by using the > +:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl. This will > +also revert the format and selection configuration to device defaults. I don't expect someone who doesn't have detailed knowledge of the implementation to understand what you mean here :-( > Configuring streams > ^^^^^^^^^^^^^^^^^^^ > @@ -565,8 +586,11 @@ Controller API <media_controller>` > setting the routing table will reset all the stream configurations in a media > entity. > > -3) Configure streams. Each route endpoint must be configured > -with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`. > +3) Configure formats and selections. Each route is configured separately We don't configure routes but streams, don't we ? Mentioning routes here seems confusing to me. > +as documented plain subdevices in :ref:`<format-propagation>`. The stream Did you mean "as documented for plain subdevices" ? This is the part that bothers me. I don't think we can consider the configuration of routes to be independent in the general case. How the formats and selection rectangle of different streams interact with each other is what I'd like to see documented. As this is an area that has been entirely unexplored until now as far as I can tell, I would be fine if we wanted to merge streams support without support for selection rectangles to start with, and experiment with the selection API later, once we have good use cases that can be used for design and development. > +ID is set to the same stream ID associated with either sink or source pads > +of routes configured using the :ref:`VIDIOC_SUBDEV_S_ROUTING > +<VIDIOC_SUBDEV_G_ROUTING>` ioctl. > > Multiplexed streams setup example > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- Regards, Laurent Pinchart
