Add documentation explaining the ability to capture statistics from
the mali-c55 driver's new V4L2 device, as well as the various tap
points from which those statistics can be drawn in the ISP's
processing flow. Additionally add a page detailing the new V4L2
meta format for the mali-c55 statistics.
Reviewed-by: Laurent Pinchart <laurent.pinchart@xxxxxxxxxxxxxxxx>
Acked-by: Nayden Kanchev <nayden.kanchev@xxxxxxx>
Co-developed-by: Jacopo Mondi <jacopo.mondi@xxxxxxxxxxxxxxxx>
Signed-off-by: Jacopo Mondi <jacopo.mondi@xxxxxxxxxxxxxxxx>
Signed-off-by: Daniel Scally <dan.scally@xxxxxxxxxxxxxxxx>
---
Changes in v8:
- None
Changes in v7:
- None
Changes in v6:
- None
Changes in v5:
- New patch
Documentation/admin-guide/media/mali-c55.rst | 60 ++++++++++++++++++-
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/metafmt-arm-mali-c55.rst | 29 +++++++++
MAINTAINERS | 1 +
4 files changed, 90 insertions(+), 1 deletion(-)
create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst
index 72cdded507b3..7eaeac63ddf7 100644
--- a/Documentation/admin-guide/media/mali-c55.rst
+++ b/Documentation/admin-guide/media/mali-c55.rst
@@ -67,10 +67,11 @@ The driver has 4 V4L2 subdevices:
- `mali_c55 resizer fr`: The Full-Resolution pipe resizer
- `mali_c55 resizer ds`: The Downscale pipe resizer
-The driver has 2 V4L2 video devices:
+The driver has 3 V4L2 video devices:
- `mali-c55 fr`: The full-resolution pipe's capture device
- `mali-c55 ds`: The downscale pipe's capture device
+- `mali-c55 3a stats`: The 3A statistics capture device
Frame sequences are synchronised across to two capture devices, meaning if one
pipe is started later than the other the sequence numbers returned in its
@@ -333,6 +334,63 @@ configured, followed by formats in the appropriate places:
# Set format on the video device and stream
yavta -f RGB565 -s 1920x1080 -c10 /dev/video0
+.. _mali-c55-3a-stats:
+
+Capturing ISP Statistics
+========================
+
+The ISP is capable of producing statistics for consumption by image processing
+algorithms running in userspace. These statistics can be captured by queueing
+buffers to the `mali-c55 3a stats` V4L2 Device whilst the ISP is streaming. Only
+the :ref:`V4L2_META_FMT_MALI_C55_STATS <v4l2-meta-fmt-mali-c55-3a-stats>`
+format is supported, so no format-setting need be done:
+
+.. code-block:: none
+
+ # We assume the media graph has been configured to support RGB565 capture
+ # from the mali-c55 fr V4L2 Device, which is at /dev/video0. The statistics
+ # V4L2 device is at /dev/video3
+
+ yavta -f RGB565 -s 1920x1080 -c32 /dev/video0 && \
+ yavta -c10 -F /dev/video3
+
+The layout of the buffer is described by :c:type:`mali_c55_stats_buffer`,
+but broadly statistics are generated to support three image processing
+algorithms; AEXP (Auto-Exposure), AWB (Auto-White Balance) and AF (Auto-Focus).
+These stats can be drawn from various places in the Mali C55 ISP pipeline, known
+as "tap points". This high-level block diagram is intended to explain where in
+the processing flow the statistics can be drawn from::
+
+ +--> AEXP-2 +----> AEXP-1 +--> AF-0
+ | +----> AF-1 |
+ | | |
+ +---------+ | +--------------+ | +--------------+ |
+ | Input +-+-->+ Digital Gain +---+-->+ Black Level +---+---+
+ +---------+ +--------------+ +--------------+ |
+ +-----------------------------------------------------------------+
+ |
+ | +--------------+ +---------+ +----------------+
+ +-->| Sinter Noise +-+ White +--+--->| Lens Shading +--+---------------+
+ | Reduction | | Balance | | | | | |
+ +--------------+ +---------+ | +----------------+ | |
+ +---> AEXP-0 (A) +--> AEXP-0 (B) |
+ +--------------------------------------------------------------------------+
+ |
+ | +----------------+ +--------------+ +----------------+
+ +-->| Tone mapping +-+--->| Demosaicing +->+ Purple Fringe +-+-----------+
+ | | | +--------------+ | Correction | | |
+ +----------------+ +-> AEXP-IRIDIX +----------------+ +---> AWB-0 |
+ +----------------------------------------------------------------------------+
+ | +-------------+ +-------------+
+ +------------------->| Colour +---+--->| Output |
+ | Correction | | | Pipelines |
+ +-------------+ | +-------------+
+ +--> AWB-1
+
+At present all statistics are drawn from the 0th tap point for each algorithm;
+I.E. AEXP statistics from AEXP-0 (A), AWB statistics from AWB-0 and AF
+statistics from AF-0. In the future this will be configurable.
+
References
==========
.. [1] https://git.linuxtv.org/v4l-utils.git/
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index 86ffb3bc8ade..148b4165cb3c 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree::
:maxdepth: 1
+ metafmt-arm-mali-c55
metafmt-d4xx
metafmt-generic
metafmt-intel-ipu3
diff --git a/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
new file mode 100644
index 000000000000..186e0deb9ece
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
@@ -0,0 +1,29 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _v4l2-meta-fmt-mali-c55-3a-stats:
+
+*************************************
+V4L2_META_FMT_MALI_C55_STATS ('C55S')
+*************************************
+
+3A Statistics
+=============
+
+The ISP device collects different statistics over an input bayer frame. Those
+statistics can be obtained by userspace from the
+:ref:`mali-c55 3a stats <mali-c55-3a-stats>` metadata capture video node, using
+the :c:type:`v4l2_meta_format` interface. The buffer contains a single instance
+of the C structure :c:type:`mali_c55_stats_buffer` defined in
+``mali-c55-config.h``, so the structure can be obtained from the buffer by:
+
+.. code-block:: C
+
+ struct mali_c55_stats_buffer *stats =
+ (struct mali_c55_stats_buffer *)buf;
+
+For details of the statistics see :c:type:`mali_c55_stats_buffer`.
+
+Arm Mali-C55 uAPI data types
+============================
+
+.. kernel-doc:: include/uapi/linux/media/arm/mali-c55-config.h
diff --git a/MAINTAINERS b/MAINTAINERS
index b985881b3d4f..7672963e9f24 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1863,6 +1863,7 @@ T: git git://linuxtv.org/media_tree.git
F: Documentation/admin-guide/media/mali-c55-graph.dot
F: Documentation/admin-guide/media/mali-c55.rst
F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml
+F: Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
F: drivers/media/platform/arm/mali-c55/
F: include/uapi/linux/media/arm/mali-c55-config.h
--
2.34.1
