Document internal sink pads, pads that have both SINK and INTERNAL flags
set. Use the IMX219 camera sensor as an example.
Signed-off-by: Sakari Ailus <sakari.ailus@xxxxxxxxxxxxxxx>
Reviewed-by Julien Massot <julien.massot@xxxxxxxxxxxxx>
---
.../userspace-api/media/v4l/dev-subdev.rst | 145 ++++++++++++++++++
1 file changed, 145 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index b76e02e54512..d30dcb9e2537 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
sub-device and a (pad, stream) pair. For sub-devices that do not support
multiplexed streams the 'stream' field is always 0.
+.. _v4l2-subdev-internal-source-pads:
+
+Internal sink pads and routing
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Cases where a single sub-device source pad is traversed by multiple streams, one
+or more of which originate from within the sub-device itself, are special as
+there is no external sink pad for such routes. In those cases, the sources of
+the internally generated streams are represented by internal sink pads, which
+are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
+pad flag set.
+
+Internal pads have all the properties of an external pad, including formats and
+selections. The format in this case is the source format of the stream. An
+internal pad always has a single stream only (0).
+
+Routes from an internal sink pad to an external source pad are typically not
+modifiable but they can be activated and deactivated using the
+:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
+on driver capabilities.
+
Interaction between routes, streams, formats and selections
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -668,3 +689,127 @@ To configure this pipeline, the userspace must take the following steps:
the configurations along the stream towards the receiver, using
:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
stream endpoint in each sub-device.
+
+Internal pads setup example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- An IMX219 camera sensor source sub-device, with one source pad (0), one
+ internal sink pad (1) as the source of the image data and an internal sink
+ pad (2) as the source of the embedded data. There are two routes, one from the
+ internal sink pad 1 to the source 0 (image data) and another from the internal
+ sink pad 2 to the source pad 0 (embedded data). Both streams are always
+ active, i.e. there is no need to separately enable the embedded data
+ stream. The sensor uses the CSI-2 interface.
+
+- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
+ connected to the sensor, and two source pads (pads 1 and 2), to the DMA
+ engine. The receiver demultiplexes the incoming streams to the source pads.
+
+- DMA engines in the SoC, one for each stream. Each DMA engine is connected to a
+ single source pad of the receiver.
+
+The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
+userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
+V4L2 devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: enable the links from the sensor to the
+ receiver and from the receiver to the DMA engines. This step does not differ
+ from normal non-multiplexed media controller setup.
+
+2) Configure routing
+
+.. flat-table:: Camera sensor. There are no configurable routes.
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 1/0
+ - 0/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from the sink pad
+ * - 2/0
+ - 0/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from the internal sink pad
+
+.. flat-table:: Receiver routing table. Typically both routes need to be
+ explicitly set.
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 1/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from camera sensor
+ * - 0/1
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from camera sensor
+
+The options available in sensor's routing configuration are dictated by hardware
+capabilities: typically camera sensors always produce an image data stream while
+it may be possible to enable and disable the embedded data stream.
+
+3) Configure formats and selections
+
+ This example assumes that the formats are propagated from sink pad to the
+ source pad as-is. The tables contain fields of both struct v4l2_subdev_format
+ and struct v4l2_mbus_framefmt.
+
+.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
+ static or propagated. The order is aligned with configured
+ routes.
+ :header-rows: 1
+ :fill-cells:
+
+ * - Sub-device
+ - Pad/Stream
+ - Width
+ - Height
+ - Code
+ * - :rspan:`3` IMX219
+ - 1/0
+ - 3296
+ - 2480
+ - MEDIA_BUS_FMT_SRGGB10
+ * - 0/0
+ - **3296**
+ - **2480**
+ - **MEDIA_BUS_FMT_SRGGB10**
+ * - 2/0
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_IMX219_EMBEDDED
+ * - 1/1
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+ * - :rspan:`3` CSI-2 receiver
+ - 0/0
+ - **3296**
+ - **2480**
+ - **MEDIA_BUS_FMT_SRGGB10**
+ * - 1/0
+ - 3296
+ - 2480
+ - MEDIA_BUS_FMT_SRGGB10
+ * - 0/1
+ - **3296**
+ - **2**
+ - **MEDIA_BUS_FMT_META_10**
+ * - 2/0
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+
+The embedded data format does not need to be configured as the format is
+dictated by the pixel data format in this case.
--
2.39.2
