On 05/10/2021 11:57, Tomi Valkeinen wrote:
Hi,
This is v9 of the multiplexed streams series. v8 can be found from:
https://lore.kernel.org/all/20210830110116.488338-1-tomi.valkeinen@xxxxxxxxxxxxxxxx/
Here is a diff between v8 and v9, which may help in reviews.
diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst
index b28607c3fc79..410e03a29f2a 100644
--- a/Documentation/driver-api/media/v4l2-subdev.rst
+++ b/Documentation/driver-api/media/v4l2-subdev.rst
@@ -523,7 +523,7 @@ Centrally managed subdev active state
Traditionally V4L2 subdev drivers maintained internal state for the active
configuration for the subdev. This is often implemented as an array of struct
-v4l2_mbus_framefmt, on entry for each pad.
+v4l2_mbus_framefmt, one entry for each pad.
In addition to the active configuration, each subdev filehandle has contained
an array of struct v4l2_subdev_pad_config, managed by V4L2 core, which
@@ -531,17 +531,28 @@ contains the TRY configuration.
To simplify the subdev drivers the V4L2 subdev API now optionally supports a
centrally managed active configuration. A subdev driver must use
-v4l2_subdev_alloc_state() to initialize the active state between calls to
-media_entity_pads_init() and v4l2_*_register_subdev().
+v4l2_subdev_init_finalize() to initialize the active state between calls to
+media_entity_pads_init() and v4l2_*_register_subdev(), and must call
+v4l2_subdev_cleanup() to free the state.
-The active state must be locked before access, and can be done with e.g.
+The active state must be locked before access, and can be done with
v4l2_subdev_lock_state() or v4l2_subdev_lock_active_state().
The V4L2 core will pass either the TRY or ACTIVE state to various subdev ops.
-Unfortunately all the subdev drivers do not comply with this yet, and may
-pass NULL for the ACTIVE case, so the called subdev ops must also handle the
-NULL case. This can be easily managed by the helpers
-v4l2_subdev_validate_state() or v4l2_subdev_validate_and_lock_state().
+Unfortunately all the subdev drivers do not comply with this yet, and may pass
+NULL for the ACTIVE case. This is only a problem for subdev drivers which use
+the cetrally managed active state and are used in media pipelines with older
+subdev drivers. In these cases the called subdev ops must also handle the NULL
+case. This can be easily managed by the use of
+v4l2_subdev_validate_and_lock_state() helper.
+
+Streams, multiplexed media pads and internal routing
+----------------------------------------------------
+
+A subdevice driver can implement support for multiplexed streams by setting
+the V4L2_SUBDEV_FL_MULTIPLEXED subdev flag and implementing support for
+centrally managed subdev active state, routing and stream based
+configuration.
V4L2 sub-device functions and data structures
---------------------------------------------
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index a67c2749089a..ea3efa97bb08 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -503,3 +503,167 @@ source pads.
:maxdepth: 1
subdev-formats
+
+Streams, multiplexed media pads and internal routing
+----------------------------------------------------
+
+Commonly V4L2 subdevices support only separate video streams, that is, each
+link in the media grap and each pad in a subdevice passes through a single
+video stream. Thus each pad contains a format configuration for that single
+stream. In some cases a subdev can do stream processing and split a stream
+into two or compose two streams into one, but the inputs and outputs for the
+subdev are still a single stream per pad.
+
+Some hardware, e.g. MIPI CSI-2, support multiplexed streams, that is, a single
+bus carries multiple streams. Thus a camera could output two streams, a pixel
+stream and a metadata stream, and a bridge subdev could route the streams
+from multiple input pads into a single output pad.
+
+Subdevice drivers that support multiplexed streams are compatible with
+non-multiplexed subdev drivers, but, of course, requires such a routing
+configuration where the link between those two types of drivers contain only
+a single stream.
+
+Understanding streams
+^^^^^^^^^^^^^^^^^^^^^
+
+A stream is a stream of content (e.g. pixel data or metadata) flowing through
+the media pipeline from a source (e.g. a sensor) towards the final sink
+(e.g. a receiver in a SoC). Each media link carries all the streams from
+one end of the link to the other, whereas subdevices have routing tables
+which describe how the incoming streams from sink pads are routed to the
+source pads.
+
+A stream ID (often just "stream") is a media link-local identifier for a
+stream. In other words, configuration for a particular stream ID must exist
+on both sides of a media link, but another stream ID can be used for the same
+stream at the other side of the subdevice.
+
+A stream at a specific point in the media pipeline is identified with the
+subdev and a (pad, stream) pair. For subdevices that do not support
+multiplexed streams the 'stream' is always 0.
+
+Configuring streams
+^^^^^^^^^^^^^^^^^^^
+
+The configuration of the streams is done individually for each subdevice and
+the validity of the streams between subdevices is validated when the pipeline
+is started.
+
+There are three steps in configuring the streams:
+
+1) Set up links. Connect the pads between subdevices using the :ref:`Media
+Controller API <media_controller>`
+
+2) Routing. The routing table for the subdevice must be set with
+:ref:`VIDIOC_SUBDEV_S_ROUTING <VIDIOC_SUBDEV_G_ROUTING>` ioctl.
+
+3) Configure streams. Each route endpoint must be configured
+with :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>`.
+
+Multiplexed streams setup example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- Two identical sensors (Sensor A and Sensor B). Each sensor has a single
+ source pad (pad 0), and outputs two streams, pixel data and metadata.
+
+- Multiplexer bridge (Bridge). The bridge has two sink pads, connected to the
+ sensors (pads 0, 1), and one source pad (pad 2), which outputs all 4
+ streams.
+
+- Receiver in the SoC (Receiver). The receiver has a single sink pad (pad 0),
+ connected to the bridge, and four source pads (pads 1-4), going to the DMA
+ engine. The receiver demultiplexes the incoming streams to the four source
+ pads.
+
+- Four DMA Engines in the SoC (DMA Engine). Each DMA engine is connected to a
+ single source pad in the receiver.
+
+The sensors, the bridge and the receiver are modeled as V4L2 subdevices,
+exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
+modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: connect the sensors to the bridge,
+bridge to the receiver, and the receiver to the DMA engines. This step does
+not differ from normal non-multiplexed media controller setup.
+
+2) Configure routing.
+
+.. flat-table:: Sensor routing table (identical on both sensors)
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0 (unused)
+ - 0/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
+ - Pixel data stream. Source route, i.e. the sink fields are unused.
+ * - 0/0 (unused)
+ - 0/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_SOURCE
+ - Metadata stream. Source route, i.e. the sink fields are unused.
+
+.. flat-table:: Bridge routing table
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor A
+ * - 0/1
+ - 2/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor A
+ * - 1/0
+ - 2/2
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor B
+ * - 1/1
+ - 2/3
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor B
+
+.. flat-table:: Receiver routing table
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 1/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor A
+ * - 0/1
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor A
+ * - 0/2
+ - 3/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from Sensor B
+ * - 0/3
+ - 4/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from Sensor B
+
+3) Configure streams
+
+After configuring the routing table, the next step is configuring the streams.
+This step is similar to configuring the pads in a non-multiplexed streams
+setup, with the difference that we need to configure each (pad, stream) pair
+(i.e. route endpoint), instead of just a pad.
+
+Presuming there are no format conversions in the pipeline, the userspace needs
+to configure all the route endpoints using four formats (two pixel formats
+and two metadata formats) with VIDIOC_SUBDEV_S_FMT.
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index 1de30d5b437f..e6bd94d63e4f 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -244,6 +244,7 @@ static int rvin_try_format(struct rvin_dev *vin, u32 which,
{
struct v4l2_subdev *sd = vin_to_source(vin);
struct v4l2_subdev_state *sd_state;
+ static struct lock_class_key key;
struct v4l2_subdev_format format = {
.which = which,
.pad = vin->parallel.source_pad,
@@ -252,7 +253,7 @@ static int rvin_try_format(struct rvin_dev *vin, u32 which,
u32 width, height;
int ret;
- sd_state = v4l2_alloc_subdev_state(sd, V4L2_SUBDEV_FORMAT_ACTIVE);
+ sd_state = __v4l2_subdev_state_alloc(sd, "rvin:state->lock", &key);
if (IS_ERR(sd_state))
return PTR_ERR(sd_state);
@@ -288,7 +289,7 @@ static int rvin_try_format(struct rvin_dev *vin, u32 which,
rvin_format_align(vin, pix);
done:
- v4l2_free_subdev_state(sd_state);
+ __v4l2_subdev_state_free(sd_state);
return ret;
}
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 63ea5e472c33..e607c3ae2520 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -613,6 +613,7 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
const char *name, unsigned int num_pads,
const struct v4l2_subdev_ops *ops, u32 function)
{
+ static struct lock_class_key key;
struct v4l2_subdev *subdev;
unsigned int i;
int ret;
@@ -675,7 +676,8 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
* Allocate the pad configuration to store formats and selection
* rectangles.
*/
- entity->config = v4l2_alloc_subdev_state(&entity->subdev, V4L2_SUBDEV_FORMAT_ACTIVE);
+ entity->config = __v4l2_subdev_state_alloc(&entity->subdev,
+ "vsp1:config->lock", &key);
if (IS_ERR(entity->config)) {
media_entity_cleanup(&entity->subdev.entity);
return PTR_ERR(entity->config);
@@ -690,6 +692,6 @@ void vsp1_entity_destroy(struct vsp1_entity *entity)
entity->ops->destroy(entity);
if (entity->subdev.ctrl_handler)
v4l2_ctrl_handler_free(entity->subdev.ctrl_handler);
- v4l2_free_subdev_state(entity->config);
+ __v4l2_subdev_state_free(entity->config);
media_entity_cleanup(&entity->subdev.entity);
}
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index e3a7793157db..2a64ff003e4b 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -27,11 +27,10 @@
#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
{
- static struct lock_class_key __key;
struct v4l2_subdev_state *state;
+ static struct lock_class_key key;
- state = __v4l2_alloc_subdev_state(sd, V4L2_SUBDEV_FORMAT_TRY,
- "v4l2_subdev_fh->state", &__key);
+ state = __v4l2_subdev_state_alloc(sd, "fh->state->lock", &key);
if (IS_ERR(state))
return PTR_ERR(state);
@@ -42,7 +41,7 @@ static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
static void subdev_fh_free(struct v4l2_subdev_fh *fh)
{
- v4l2_free_subdev_state(fh->state);
+ __v4l2_subdev_state_free(fh->state);
fh->state = NULL;
}
@@ -815,11 +814,11 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
return -EINVAL;
}
- krouting.which = routing->which;
krouting.num_routes = routing->num_routes;
krouting.routes = routes;
- return v4l2_subdev_call(sd, pad, set_routing, state, &krouting);
+ return v4l2_subdev_call(sd, pad, set_routing, state,
+ routing->which, &krouting);
}
default:
@@ -925,10 +924,11 @@ v4l2_init_stream_configs(struct v4l2_subdev_stream_configs *stream_configs,
continue;
/*
- * Each route needs a format on both ends of the route, except for
- * source streams which only need one format.
+ * Each route needs a format on both ends of the route, except
+ * for source streams which only need one format.
*/
- num_configs += (route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE) ? 1 : 2;
+ num_configs +=
+ (route->flags & V4L2_SUBDEV_ROUTE_FL_SOURCE) ? 1 : 2;
}
if (num_configs) {
@@ -942,7 +942,10 @@ v4l2_init_stream_configs(struct v4l2_subdev_stream_configs *stream_configs,
stream_configs->num_configs = num_configs;
}
- /* Fill in the 'pad' and stream' value for each item in the array from the routing table */
+ /*
+ * Fill in the 'pad' and stream' value for each item in the array from
+ * the routing table
+ */
for (i = 0; i < routing->num_routes; ++i) {
struct v4l2_subdev_route *route = &routing->routes[i];
u32 idx;
@@ -1277,7 +1280,8 @@ int v4l2_subdev_link_validate(struct media_link *link)
}
EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
-bool v4l2_subdev_has_route(struct media_entity *entity, unsigned int pad0, unsigned int pad1)
+bool v4l2_subdev_has_route(struct media_entity *entity, unsigned int pad0,
+ unsigned int pad1)
{
struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
struct v4l2_subdev_krouting *routing;
@@ -1308,9 +1312,7 @@ bool v4l2_subdev_has_route(struct media_entity *entity, unsigned int pad0, unsig
EXPORT_SYMBOL_GPL(v4l2_subdev_has_route);
struct v4l2_subdev_state *
-__v4l2_alloc_subdev_state(struct v4l2_subdev *sd,
- enum v4l2_subdev_format_whence which,
- const char *lock_name,
+__v4l2_subdev_state_alloc(struct v4l2_subdev *sd, const char *lock_name,
struct lock_class_key *lock_key)
{
struct v4l2_subdev_state *state;
@@ -1322,8 +1324,6 @@ __v4l2_alloc_subdev_state(struct v4l2_subdev *sd,
__mutex_init(&state->lock, lock_name, lock_key);
- state->which = which;
-
/* Drivers that support streams do not need the legacy pad config */
if (!(sd->flags & V4L2_SUBDEV_FL_MULTIPLEXED) && sd->entity.num_pads) {
state->pads = kvmalloc_array(sd->entity.num_pads,
@@ -1349,9 +1349,9 @@ __v4l2_alloc_subdev_state(struct v4l2_subdev *sd,
return ERR_PTR(ret);
}
-EXPORT_SYMBOL_GPL(__v4l2_alloc_subdev_state);
+EXPORT_SYMBOL_GPL(__v4l2_subdev_state_alloc);
-void v4l2_free_subdev_state(struct v4l2_subdev_state *state)
+void __v4l2_subdev_state_free(struct v4l2_subdev_state *state)
{
if (!state)
return;
@@ -1363,7 +1363,7 @@ void v4l2_free_subdev_state(struct v4l2_subdev_state *state)
kvfree(state->pads);
kfree(state);
}
-EXPORT_SYMBOL_GPL(v4l2_free_subdev_state);
+EXPORT_SYMBOL_GPL(__v4l2_subdev_state_free);
#endif /* CONFIG_MEDIA_CONTROLLER */
@@ -1394,12 +1394,12 @@ void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
}
EXPORT_SYMBOL_GPL(v4l2_subdev_notify_event);
-int __v4l2_subdev_alloc_state(struct v4l2_subdev *sd, const char *name,
- struct lock_class_key *key)
+int __v4l2_subdev_init_finalize(struct v4l2_subdev *sd, const char *name,
+ struct lock_class_key *key)
{
struct v4l2_subdev_state *state;
- state = __v4l2_alloc_subdev_state(sd, V4L2_SUBDEV_FORMAT_ACTIVE, name, key);
+ state = __v4l2_subdev_state_alloc(sd, name, key);
if (IS_ERR(state))
return PTR_ERR(state);
@@ -1407,14 +1407,14 @@ int __v4l2_subdev_alloc_state(struct v4l2_subdev *sd, const char *name,
return 0;
}
-EXPORT_SYMBOL_GPL(__v4l2_subdev_alloc_state);
+EXPORT_SYMBOL_GPL(__v4l2_subdev_init_finalize);
-void v4l2_subdev_free_state(struct v4l2_subdev *sd)
+void v4l2_subdev_cleanup(struct v4l2_subdev *sd)
{
- v4l2_free_subdev_state(sd->state);
+ __v4l2_subdev_state_free(sd->state);
sd->state = NULL;
}
-EXPORT_SYMBOL_GPL(v4l2_subdev_free_state);
+EXPORT_SYMBOL_GPL(v4l2_subdev_cleanup);
struct v4l2_subdev_state *v4l2_subdev_lock_active_state(struct v4l2_subdev *sd)
{
@@ -1449,18 +1449,15 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
dst->routes = NULL;
dst->num_routes = 0;
- if (src->num_routes == 0) {
- dst->which = src->which;
- } else {
- dst->routes = kvmalloc_array(src->num_routes, sizeof(*src->routes),
- GFP_KERNEL);
+ if (src->num_routes > 0) {
+ dst->routes = kvmalloc_array(src->num_routes,
+ sizeof(*src->routes), GFP_KERNEL);
if (!dst->routes)
return -ENOMEM;
memcpy(dst->routes, src->routes,
src->num_routes * sizeof(*src->routes));
dst->num_routes = src->num_routes;
- dst->which = src->which;
}
return v4l2_init_stream_configs(&state->stream_configs, dst);
diff --git a/drivers/staging/media/tegra-video/vi.c b/drivers/staging/media/tegra-video/vi.c
index 691f5e04b0a1..3516735f0cad 100644
--- a/drivers/staging/media/tegra-video/vi.c
+++ b/drivers/staging/media/tegra-video/vi.c
@@ -491,6 +491,7 @@ static int __tegra_channel_try_format(struct tegra_vi_channel *chan,
struct v4l2_pix_format *pix)
{
const struct tegra_video_format *fmtinfo;
+ static struct lock_class_key key;
struct v4l2_subdev *subdev;
struct v4l2_subdev_format fmt;
struct v4l2_subdev_state *sd_state;
@@ -507,7 +508,8 @@ static int __tegra_channel_try_format(struct tegra_vi_channel *chan,
if (!subdev)
return -ENODEV;
- sd_state = v4l2_alloc_subdev_state(subdev, V4L2_SUBDEV_FORMAT_ACTIVE);
+ sd_state = __v4l2_subdev_state_alloc(subdev, "tegra:state->lock",
+ &key);
if (IS_ERR(sd_state))
return PTR_ERR(sd_state);
/*
@@ -558,7 +560,7 @@ static int __tegra_channel_try_format(struct tegra_vi_channel *chan,
v4l2_fill_pix_format(pix, &fmt.format);
tegra_channel_fmt_align(chan, pix, fmtinfo->bpp);
- v4l2_free_subdev_state(sd_state);
+ __v4l2_subdev_state_free(sd_state);
return 0;
}
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index cd1750e495df..f90cc6417d30 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -933,7 +933,8 @@ struct media_pad *__media_entity_next_routed_pad(struct media_pad *root,
struct media_pad *iter);
/**
- * media_entity_for_each_routed_pad - Iterate over entity pads connected by routes
+ * media_entity_for_each_routed_pad - Iterate over entity pads connected by
+ * routes
*
* @root: The root pad to which the iterated pads have a route
* @iter: The iterator pad
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 476d797ecc68..5e50f2ded653 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -728,24 +728,20 @@ struct v4l2_subdev_stream_configs {
/**
* struct v4l2_subdev_krouting - subdev routing table
*
- * @which: format type (from enum v4l2_subdev_format_whence)
- * @routes: &struct v4l2_subdev_route
* @num_routes: number of routes
+ * @routes: &struct v4l2_subdev_route
*
- * This structure is used to translate arguments received from
- * VIDIOC_SUBDEV_G/S_ROUTING() ioctl to subdev device drivers operations.
+ * This structure contains the routing table for a subdev.
*/
struct v4l2_subdev_krouting {
- u32 which;
- struct v4l2_subdev_route *routes;
unsigned int num_routes;
+ struct v4l2_subdev_route *routes;
};
/**
* struct v4l2_subdev_state - Used for storing subdev state information.
*
* @lock: mutex for the state
- * @which: state type (from enum v4l2_subdev_format_whence)
* @pads: &struct v4l2_subdev_pad_config array
* @routing: routing table for the subdev
* @stream_configs: stream configurations (only for V4L2_SUBDEV_FL_MULTIPLEXED)
@@ -756,7 +752,6 @@ struct v4l2_subdev_krouting {
*/
struct v4l2_subdev_state {
struct mutex lock;
- u32 which;
struct v4l2_subdev_pad_config *pads;
struct v4l2_subdev_krouting routing;
struct v4l2_subdev_stream_configs stream_configs;
@@ -872,6 +867,7 @@ struct v4l2_subdev_pad_ops {
struct v4l2_mbus_config *config);
int (*set_routing)(struct v4l2_subdev *sd,
struct v4l2_subdev_state *state,
+ enum v4l2_subdev_format_whence which,
struct v4l2_subdev_krouting *route);
};
@@ -950,7 +946,12 @@ struct v4l2_subdev_internal_ops {
/*
* Set this flag if this subdev supports multiplexed streams. This means
* that the driver supports routing and handles the stream parameter in its
- * v4l2_subdev_pad_ops handlers.
+ * v4l2_subdev_pad_ops handlers. More specifically, this means:
+ *
+ * - Centrally managed active state is enabled
+ * - Legacy pad config is _not_ supported (state->pads)
+ * - Routing ioctls are available
+ * - Multiple streams per pad are supported
*/
#define V4L2_SUBDEV_FL_MULTIPLEXED (1U << 4)
@@ -1008,7 +1009,7 @@ struct v4l2_subdev_platform_data {
* device using v4l2_async_register_subdev_sensor().
* @pdata: common part of subdevice platform data
* @state: active state for the subdev (NULL for subdevs tracking the state
- * internally)
+ * internally)
*
* Each instance of a subdev driver should create this struct, either
* stand-alone or embedded in a larger struct.
@@ -1040,6 +1041,18 @@ struct v4l2_subdev {
struct v4l2_async_notifier *notifier;
struct v4l2_async_notifier *subdev_notifier;
struct v4l2_subdev_platform_data *pdata;
+
+ /*
+ * The fields below are private, and should only be accessed via
+ * appropriate functions.
+ */
+
+ /*
+ * TODO: state should most likely be changed from a pointer to an
+ * embedded field. For the time being it's kept as a pointer to more
+ * easily catch uses of state in the cases where the driver doesn't
+ * support it.
+ */
struct v4l2_subdev_state *state;
};
@@ -1263,32 +1276,24 @@ bool v4l2_subdev_has_route(struct media_entity *entity, unsigned int pad0,
unsigned int pad1);
/**
- * v4l2_alloc_subdev_state - allocate v4l2_subdev_state
+ * __v4l2_subdev_state_alloc - allocate v4l2_subdev_state
*
* @sd: pointer to &struct v4l2_subdev for which the state is being allocated.
- * @which: configuration type for the state (from enum v4l2_subdev_format_whence)
+ * @lock_name: name of the state lock
+ * @key: lock_class_key for the lock
*
- * Must call v4l2_free_subdev_state() when state is no longer needed.
+ * Must call __v4l2_subdev_state_free() when state is no longer needed.
*/
-#define v4l2_alloc_subdev_state(sd, which) \
- ({ \
- static struct lock_class_key __key; \
- const char *name = KBUILD_BASENAME \
- ":" __stringify(__LINE__) ":sd->state->lock"; \
- __v4l2_alloc_subdev_state(sd, which, name, &__key); \
- })
-
-struct v4l2_subdev_state *
-__v4l2_alloc_subdev_state(struct v4l2_subdev *sd,
- enum v4l2_subdev_format_whence which,
- const char *lock_name, struct lock_class_key *key);
+struct v4l2_subdev_state *__v4l2_subdev_state_alloc(struct v4l2_subdev *sd,
+ const char *lock_name,
+ struct lock_class_key *key);
/**
- * v4l2_free_subdev_state - free a v4l2_subdev_state
+ * __v4l2_subdev_state_free - free a v4l2_subdev_state
*
* @state: v4l2_subdev_state to be freed.
*/
-void v4l2_free_subdev_state(struct v4l2_subdev_state *state);
+void __v4l2_subdev_state_free(struct v4l2_subdev_state *state);
#endif /* CONFIG_MEDIA_CONTROLLER */
@@ -1358,33 +1363,36 @@ void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
const struct v4l2_event *ev);
/**
- * v4l2_subdev_alloc_state() - Allocate active subdev state for subdevice
- * @sd: The subdev for which the state is allocated
+ * v4l2_subdev_init_finalize() - Finalize the initialization of the subdevice
+ * @sd: The subdev
*
- * This will allocate a subdev state and store it to
- * &struct v4l2_subdev->state.
+ * This finalizes the initialization of the subdev, including allocation of
+ * the active state for the subdev.
*
- * Must call v4l2_subdev_free_state() when the state is no longer needed.
+ * This must be called by the subdev drivers that use the centralized active
+ * state, after the subdev struct has been initialized and
+ * media_entity_pads_init() has been called.
+ *
+ * Must call v4l2_subdev_cleanup() when the subdev is being removed.
*/
-#define v4l2_subdev_alloc_state(sd) \
+#define v4l2_subdev_init_finalize(sd) \
({ \
static struct lock_class_key __key; \
const char *name = KBUILD_BASENAME \
- ":" __stringify(__LINE__) ":sd->state->lock"; \
- __v4l2_subdev_alloc_state(sd, name, &__key); \
+ ":" __stringify(__LINE__) ":subdev->state->lock"; \
+ __v4l2_subdev_init_finalize(sd, name, &__key); \
})
-int __v4l2_subdev_alloc_state(struct v4l2_subdev *sd, const char *name,
- struct lock_class_key *key);
+int __v4l2_subdev_init_finalize(struct v4l2_subdev *sd, const char *name,
+ struct lock_class_key *key);
/**
- * v4l2_subdev_free_state() - Free the active subdev state for subdevice
+ * v4l2_subdev_cleanup() - Release the resources needed by the subdevice
* @sd: The subdevice
*
- * This will free the subdev's state and set
- * &struct v4l2_subdev->state to NULL.
+ * This will release the resources allocated in v4l2_subdev_init_finalize.
*/
-void v4l2_subdev_free_state(struct v4l2_subdev *sd);
+void v4l2_subdev_cleanup(struct v4l2_subdev *sd);
/**
* v4l2_subdev_get_active_state() - Return the active subdev state for subdevice
@@ -1400,7 +1408,8 @@ v4l2_subdev_get_active_state(struct v4l2_subdev *sd)
}
/**
- * v4l2_subdev_lock_active_state() - Lock and return the active subdev state for subdevice
+ * v4l2_subdev_lock_active_state() - Lock and return the active subdev state for
+ * subdevice
* @sd: The subdevice
*
* Return the locked active state for the subdevice, or NULL if the subdev
@@ -1429,34 +1438,26 @@ void v4l2_subdev_lock_state(struct v4l2_subdev_state *state);
void v4l2_subdev_unlock_state(struct v4l2_subdev_state *state);
/**
- * v4l2_subdev_validate_state() - Gets the TRY or ACTIVE subdev state
+ * v4l2_subdev_validate_and_lock_state() - Gets locked TRY or ACTIVE subdev
+ * state
* @sd: subdevice
* @state: subdevice state as passed to the subdev op
*
- * Subdev ops used to be sometimes called with NULL as the state for ACTIVE
- * case. Even if the v4l2 core now passes proper state for both TRY and
- * ACTIVE cases, a subdev driver may call an op in another subdev driver,
- * passing NULL.
+ * Due to legacy reasons, when subdev drivers call ops in other subdevs they use
+ * NULL as the state parameter, as subdevs always used to have their active
+ * state stored privately.
*
- * This function can be used as a helper to get the state also for the ACTIVE
- * case. The subdev driver that supports ACTIVE state can use this function
- * as the first thing in its ops, ensuring that the state variable contains
- * either the TRY or ACTIVE state.
- */
-static inline struct v4l2_subdev_state *
-v4l2_subdev_validate_state(struct v4l2_subdev *sd,
- struct v4l2_subdev_state *state)
-{
- return state ? state : sd->state;
-}
-
-/**
- * v4l2_subdev_validate_and_lock_state() - Gets locked TRY or ACTIVE subdev state
- * @sd: subdevice
- * @state: subdevice state as passed to the subdev op
+ * However, newer state-aware subdev drivers, which store their active state in
+ * a common place, subdev->state, expect to always get a proper state as a
+ * parameter.
+ *
+ * These state-aware drivers can use v4l2_subdev_validate_and_lock_state()
+ * instead of v4l2_subdev_lock_state(). v4l2_subdev_validate_and_lock_state()
+ * solves the issue by using subdev->state in case the passed state is
+ * NULL.
*
- * This is a helper function which does the same as v4l2_subdev_validate_state
- * () except that it also locks the state.
+ * This is a temporary helper function, and should be removed when we can ensure
+ * that all drivers pass proper state when calling other subdevs.
*/
static inline struct v4l2_subdev_state *
v4l2_subdev_validate_and_lock_state(struct v4l2_subdev *sd,
@@ -1486,7 +1487,8 @@ int v4l2_subdev_set_routing(struct v4l2_subdev *sd,
struct v4l2_subdev_krouting *routing);
/**
- * v4l2_subdev_set_routing_with_fmt() - Set given routing and format to subdev state
+ * v4l2_subdev_set_routing_with_fmt() - Set given routing and format to subdev
+ * state
* @sd: The subdevice
* @state: The subdevice state
* @routing: Routing that will be copied to subdev state
@@ -1532,7 +1534,8 @@ int v4l2_state_find_opposite_end(struct v4l2_subdev_krouting *routing, u32 pad,
u32 stream, u32 *other_pad, u32 *other_stream);
/**
- * v4l2_state_get_opposite_stream_format() - Get pointer to opposite stream format
+ * v4l2_state_get_opposite_stream_format() - Get pointer to opposite stream
+ * format
* @state: subdevice state
* @pad: pad id
* @stream: stream id
@@ -1564,7 +1567,7 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
/**
* v4l2_routing_simple_verify() - Verify that all streams are non-overlapping
- * 1-to-1 streams
+ * 1-to-1 streams
* @routing: routing to verify
*
* This verifies that the given routing contains only non-overlapping 1-to-1
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index d00ef85db1c7..f61ace2b5755 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -206,13 +206,13 @@ struct v4l2_subdev_capability {
* Is the route active? An active route will start when streaming is enabled
* on a video node.
*/
-#define V4L2_SUBDEV_ROUTE_FL_ACTIVE BIT(0)
+#define V4L2_SUBDEV_ROUTE_FL_ACTIVE (1 << 0)
/*
* Is the route immutable, i.e. can it be activated and inactivated?
* Set by the driver.
*/
-#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE BIT(1)
+#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE (1 << 1)
/*
* Is the route a source endpoint? A source endpoint route refers to a stream
@@ -221,7 +221,7 @@ struct v4l2_subdev_capability {
* fields are unused.
* Set by the driver.
*/
-#define V4L2_SUBDEV_ROUTE_FL_SOURCE BIT(2)
+#define V4L2_SUBDEV_ROUTE_FL_SOURCE (1 << 2)
/**
* struct v4l2_subdev_route - A route inside a subdev
