Hi, I have sent a few serieses to linux-media recently, which have been aiming at adding multiplexed streams support to v4l2 framework and to TI CAL driver. https://lore.kernel.org/linux-media/20210427124523.990938-1-tomi.valkeinen@xxxxxxxxxxxxxxxx/ https://lore.kernel.org/linux-media/20210420120433.902394-1-tomi.valkeinen@xxxxxxxxxxxxxxxx/ https://lore.kernel.org/linux-media/20210423102952.137638-1-tomi.valkeinen@xxxxxxxxxxxxxxxx/ The approach in those serieses was to have two different kinds of subdev pads: the normal ones that support configuration and pass a single stream, and the new ones supporting multiple streams. These new ones do not support any configuration. The idea is that somewhere along the pipeline there must be a non-multiplexed pad with the format. So, if you have a pad with multiplexed streams, you need to follow that stream to either direction and find a non-multiplexed pad. The format on that pad is also the format on the stream. Some of the problems with that approach are: - Subdevs no longer support self-contained configuration, as the configs come from some other subdev. - A subdev with multiple streams cannot do any config changes (like scaling), but instead has to be split up to multiple subdevs to get non-multiplexed pads. - A sensor that provides multiple streams must be split into a sensor subdev and a mux subdev. - It's not possible to link a subdev with multiplexed-supporting pad and a subdev with non-multiplexed pad. - I think it is just a bit messy, and the behavior when looking from the userspace side is a bit odd. I have a new approach in my work branch, which this series introduces as an RFC. In this approach the subdevs are configured based on streams, not pads. Each pad can contain multiple streams. In other words, previously set_fmt was called for pad X, now set_fmt is called for pad X + stream Y. I think this approach is much better than the previous one. All the problems above go away, and I think the code is simpler and the behavior of the subdevs is easier to understand. This series is not based on an upstream branch, and is not meant for real code review. The intention is to get an ack for the approach, after which I'll send proper serieses for these. This also includes the previously sent RFC for a subdev-wide state struct. This is needed to add TRY support for routing and the stream based configuration. This series is based on most of the work I've sent previously, but the following old patches have been dropped: v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations media: Documentation: Add GS_ROUTING documentation v4l: subdev: routing kernel helper functions v4l: subdev: add v4l2_subdev_get_format_dir() media: uapi: add MEDIA_PAD_FL_MULTIPLEXED flag v4l: subdev: Take routing information into account in link validation media: ti-vpe: cal: add routing support media: ti-vpe: cal: add multiplexed streams support The new patches, along with all the more hacky driver patches I need for testing can be found from: git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git multistream/work Tomi Laurent Pinchart (1): v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Tomi Valkeinen (10): media: v4l2-subdev: add subdev-wide state struct media: v4l2-subdev: driver fixes for subdev-wide state struct v4l: subdev: routing kernel helper functions v4l: subdev: add V4L2_SUBDEV_ROUTE_FL_SOURCE v4l: subdev: add stream based configuration v4l: subdev: add 'stream' to subdev ioctls v4l: subdev: use streams in v4l2_subdev_link_validate() v4l: subdev: add routing & stream config to v4l2_subdev_state v4l: subdev: add V4L2_SUBDEV_FL_MULTIPLEXED media: ti-vpe: cal: add multistream support drivers/media/i2c/ov5640.c | 10 +- drivers/media/platform/ti-vpe/cal-camerarx.c | 322 ++++++++++-- drivers/media/platform/ti-vpe/cal-video.c | 103 +++- drivers/media/platform/ti-vpe/cal.c | 34 +- drivers/media/platform/ti-vpe/cal.h | 12 +- drivers/media/v4l2-core/v4l2-ioctl.c | 25 +- drivers/media/v4l2-core/v4l2-subdev.c | 523 ++++++++++++++++--- include/media/v4l2-subdev.h | 141 +++-- include/uapi/linux/v4l2-subdev.h | 73 ++- 9 files changed, 1049 insertions(+), 194 deletions(-) -- 2.25.1
