Re: [PATCH v7 0/4] media: camss: sm8250: Virtual channels support for SM8250

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 08/01/2023 04:54, Laurent Pinchart wrote:
On Sun, Jan 08, 2023 at 04:51:22AM +0200, Laurent Pinchart wrote:
Hi Milen,

On Fri, Dec 09, 2022 at 11:40:33AM +0200, quic_mmitkov@xxxxxxxxxxx wrote:
From: Milen Mitkov <quic_mmitkov@xxxxxxxxxxx>

For v7:
- Fix an issue with output state for different versions of the IFE
   hardware (for platforms different from QRB5, e.g. QRB3).

For v6:
- Fix for a potential race condition in csid
- More detailed description on how to use/test this feature in
   user-space in the last patch.

For v5:
- Use entity->use_count instead of s_stream subdev call ret code to
   check if another instance of the pipeline is running. Prevents an
   error on 6.1 and up, when stopping one of several simultaneous
   instances.
- flush buffers instead of just returning if the pipeline didn't start.

For v4:
- fixes the warning reported by the kernel test robot
- tiny code change to enable the vc functionality with the partially-applied
   multistream patches on linux-next (tested on tag:next-20221010)

For v3:
- setting the sink pad format on the CSID entity will now propagate the
   format to the source pads to keep the subdev in a valid internal state.
- code syntax improvements

For v2:
- code syntax improvements
- The info print for the enabled VCs was demoted to a dbg print. Can be
   enabled with dynamic debug, e.g.:
echo "file drivers/media/platform/qcom/camss/* +p" > /sys/kernel/debug/dynamic_debug/control

NOTE: These changes depend on the multistream series, that as of yet
is still not merged upstream. However, part of the
multistream patches are merged in linux-next (tested on
tag:next-20221010), including the patch that introduces the
video_device_pipeline_alloc_start() functionality. This allows
applying and using this series on linux-next without applying the
complete multistream set.

The CSID hardware on SM8250 can demux the input data stream into
maximum of 4 multiple streams depending on virtual channel (vc)
or data type (dt) configuration.

Situations in which demuxing is useful:
- HDR sensors that produce a 2-frame HDR output, e.g. a light and a dark frame
   (the setup we used for testing, with the imx412 sensor),
   or a 3-frame HDR output - light, medium-lit, dark frame.
- sensors with additional metadata that is streamed over a different
   virtual channel/datatype.
- sensors that produce frames with multiple resolutions in the same pixel
   data stream

With these changes, the CSID entity has, as it did previously, a single
sink port (0), and always exposes 4 source ports (1, 2,3, 4). The
virtual channel configuration is determined by which of the source ports
are linked to an output VFE line.
Another question, how does this work when demultiplexing different data
types for the same virtual channel ?

For example, the link below will
configure the CSID driver to enable vc 0 and vc 1:

media-ctl -l '"msm_csid0":1->"msm_vfe0_rdi0":0[1]'
media-ctl -l '"msm_csid0":2->"msm_vfe0_rdi1":0[1]'

which will be demuxed and propagated into /dev/video0
and /dev/video1 respectively. With this, the userspace can use
any normal V4L2 client app to start/stop/queue/dequeue from these
video nodes. Tested with the yavta app.
I'd like to get a high-level view of the result. Could you provide the
media graph with this series applied (both -p and --print-dot) ?

The format of each RDI channel of the used VFE(s) (msm_vfe0_rdi0,
msm_vfe0_rdi1,...) must also be configured explicitly.

Note that in order to keep a valid internal subdevice state,
setting the sink pad format of the CSID subdevice will propagate
this format to the source pads. However, since the CSID hardware
can demux the input stream into several streams each of which can
be a different format, in that case each source pad's
format must be set individually, e.g.:

media-ctl -V '"msm_csid0":1[fmt:SRGGB10/3840x2160]'
media-ctl -V '"msm_csid0":2[fmt:SRGGB10/960x540]'

Milen Mitkov (4):
   media: camss: sm8250: Virtual channels for CSID
   media: camss: vfe: Reserve VFE lines on stream start and link to CSID
   media: camss: vfe-480: Multiple outputs support for SM8250
   media: camss: sm8250: Pipeline starting and stopping for multiple
     virtual channels

  .../platform/qcom/camss/camss-csid-gen2.c     | 54 ++++++++++------
  .../media/platform/qcom/camss/camss-csid.c    | 44 +++++++++----
  .../media/platform/qcom/camss/camss-csid.h    | 11 +++-
  .../media/platform/qcom/camss/camss-vfe-170.c |  4 +-
  .../media/platform/qcom/camss/camss-vfe-480.c | 61 ++++++++++++-------
  .../platform/qcom/camss/camss-vfe-gen1.c      |  4 +-
  drivers/media/platform/qcom/camss/camss-vfe.c |  1 +
  .../media/platform/qcom/camss/camss-video.c   | 21 ++++++-
  drivers/media/platform/qcom/camss/camss.c     |  2 +-
  9 files changed, 138 insertions(+), 64 deletions(-)

Hi Laurent,

Happy New Year!

Would be good if you can help us merge this series in the upcoming merge window. I will answer your 3 questions here.

Q: I'd like to get a high-level view of the result. Could you provide the media graph with this series applied (both -p and --print-dot) ?

A: As I am not sure what is the policy with regards to mail attachments on the kernel mailing lists I've base64 encoded a small archive that has 4 different media-ctl dumps: -p before and after media links setup (for 2 virtual channels demuxing on CSID0) and --print-dot before and after media links setup. It can be found here:

https://pastes.io/sp0pkgrgfh

Copy/paste the content to a file and: base64 -d <file> | tar -gz

p.s. if there's another way to share files here please mention it :)


Q: How does this work when demultiplexing different data types for the same virtual channel ?

A: Our open-source customers defined the scope of work and they needed virtual channels demuxing support only. Demuxing via datatype is not supported currently.


Q: I was wondering if Qualcomm had a spare RB5 vision development kit that I could use for testing, both for this series, and for future development (including the libcamera side).

A: We have to check internally as there's a limited supply of RB5 boards. We'll get back to you.


Thanks,

Milen




[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [Linux for Sparc]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux