HI Sakari,
On Tue, Apr 23, 2019 at 04:28:40PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Tue, Apr 23, 2019 at 12:19:32PM +0200, Jacopo Mondi wrote:
> > Hi Sakari,
> >
> > On Wed, Apr 17, 2019 at 01:41:48PM +0300, Sakari Ailus wrote:
> > > Hi Jacopo,
> > >
> > > I promised to write a bit about the routing problem I attempted to write on
> > > #v4l. It's a bit late, but here it is. Let me know if you share my
> > > understanding (and especially do so if you don't :)).
> >
> > Thanks for doing this, and sorry it took me a while to reply!
> >
> > >
> > > A route is identified by the source and sink pads as well as the stream ID
> > > on the source and the sink pads. Its flags allow enabling or disabling a
> > > route.
> > >
> > > Most devices that function as transmitters (such as sensors) have a fixed
> > > number of routes they can support. These's no need to change them; enabling
> > > and disabling them will be enough for the user.
> > >
> >
> > Fine so far.
> >
> > We indeed define routes as:
> > (sink pad - sink stream) -> (source pad - source stream)
> >
> > > For receivers this is different. What needs to be supported on the receiver
> > > side is directing any incoming stream (a 32-bit unsigned integer) to any
> > > source pad.
> > >
> > > This is because pads are not alike --- one may be connected to a block that
> > > further processes the image whereas the others may be connected to a DMA
> > > engine, just writing the data to memory.
> > >
> > > The receivers also may not make assumptions beyond the sub-device API: the
> > > stream is a 32-bit unsigned integer, there is currently no API requirement
> > > to have the stream IDs on a particular low integer range. In principle we
> > > could define that range, but I'd rather try to find better solutions than
> > > that: it's hard to come up with a number as it depends on the hardware.
> > > Some kind of an upper limit could be the number of CSI-2 channels
> > > multiplied by CSI-2 data types. That would be enough for CSI-2. 16 or 32
> > > virtual channels and up to 64 data types would mean up to 2048 routes
> > > between a demultiplexer's sink pad and *each* of its source pads. And this
> > > comes with an assumption that the source pads only support a single
> > > stream...
> >
> > I lost you here.
> >
> > My mental model was far more (too?) simple:
> >
> > - Routing tables inside an entity might have an arbitrary size, as how
> > that configuration depends on the device and the driver
> > implementation. As an example, the adv748x accepts 7 analogue
> > inputs to chose from and route to a CSI-2 source pad.
> > depending on the implementation, those 7 inputs could be modeled as
> > 7 pads with one stream each, or a single pad with 7 channels or
> > whatever else, and I agree the 'stream_id' values range is totally
> > up to the driver implementation -inside an entity-.
> >
> > - The cross-entity (sorry, I lack a better term here) multiplexing
> > happens on physical bus that allows so, and I can only think of
> > CSI-2 at the moment. Sure, you could share the lines of a parallel
> > bus playing with enables/disables of the transmitters, but this
> > is a custom hack that does not play well in this model.
> > Each CSI-2 source pad has up to 4 streams (one per VC) and the
> > content of those streams is retrieved from the transmitter by the
> > receiver through the remote frame_desc operation [*], as receivers
> > might need to be setup using to filter on particular VC/DT combinations
> > to receive the stream.
>
> Note that the concept of a "stream" is different in our proposed API and in
> CSI-2 (virtual channel): on CSI-2 virtual channels can carry multiple data
> types but in our definition they do not. That means there will be more
> streams on SUBDEV_[GS]_ROUTING API than there are on a CSI-2 bus.
>
> We also do not limit the use of the new API to old CSI-2 D-PHYs that
> only support four virtual channels. Therefore we need to think how this
> would work on 16 or 32 virtual channels, and preferrably not posing
> problems with larger numbers either.
>
Indeed, each stream carries one DT and can be sent in one VC, so at least
64*4 possible routes (per source pad) for what is currently specified by
CSI-2 and maybe up to 64*32 to have space for future extensions as you
explained below.
I agree, 2048 routes per source pad is not an easily manageable number
of entries for a routing table.
> >
> > - Each CSI-2 receiver sink pad supports 4 streams (the CSI-2 VCs) and any
> > of those streams can be directed to any of its source pads, to
> > implement what you have described (one pad connected to an ISP-like
> > device, on other to the DMA engine directly etc)
> >
> > - DT negotiation is still a bit vague. The example we have on the
> > series (adv748x and r-car csi2) configure the receiver's accepted DT
> > inspecting the remote frame_desc. Again, as per [*] this migh be
> > limitied to 1 DT for VC, which might not be enough and would require
> > re-thinking the operation used for the negotiation.
> >
> > - Configuring a multiplexed source pad image format is today not
> > possible, the format is always propagated from the sink pad to which
> > a route is enabled to the source pad. Might this be a limitation on
> > how we control which DTs are multiplexed inside a VC and won't allow
> > to model any format conversion that might happen at the source pad
> > output. I was toying myself with the idea of a stream-aware set
> > format operation for multiplexed source pads, not sure it might work
> > though.
> >
> > Can I ask you why:
> > - you mention 16 or 32 VCs ? Each CSI-2 link supports up to 4.
>
> See above.
>
> > - you put DT in the routing mix, and I suspect it is here where our
> > disconnection happens. I always assume DT configuration as a result
> > of a format configuration operation, which currently has limitations
> > as noted here above.
>
> Yes; DT comes from the format and is generally determined by the
> transmitting driver. Nothing else limits the number of DTs on a VC except
> the number of DTs --- which is currently 64.
>
> >
> >
> > [*] I would later like to talk about if this is the most appropriate
> > operation to handle this negotiation, as I'm not sure we can handle DT
> > negotiation properly with that, but that's for later.
> >
> > >
> > > CSI-2 receivers support a number of simultaneous streams, and as the stream
> > > is demultiplexed there, this means there will be as many source pads as
> > > there are supported simultaneous streams. This heavily depe`nds on the
> > > hardware, but the number could be e.g. 1, 4 or 8. Much smaller than 2048 in
> > > any case.
> > >
> >
> > Why 8? Is this related to DT multiplexing again?
>
> Yes and no. It's just a random number between 1 and 2048. :-)
>
> >
> > > Another option could be creating no routes at all at device init time, and
> > > letting the user create them. We could add a new flag for routes telling
> > > that a route is dynamic: disabling a dynamic route would simply delete it.
> > > Likewise, a SUBDEV_G_ROUTING returning no routes (but no error either)
> > > would also tell the user only dynamic routes are supported.
> > >
I see your point, but I cannot see how one would easily map a stream_id to a
DT/VC combination. One way would be:
DT = stream_id % 64; VC = stream_id / 64;
Just to make sure we're on the same page, how would you enable/disable
transmitting, say, embedded data (DT=18) on VC=2? By enabling/disabling
the stream_id 146 ( = 18 + 64 * 2) ?
Thanks
j
> > > We should document this and change the driver implementation accordingly.
> > > No API changes are needed (apart from adding the dynamic flag --- or rather
> > > call it "volatile", to avoid confusing with routes that can be enabled or
> > > disabled while streaming).
> > >
> >
> > I have v5 of the series ready. Do you want me to post it to progress
> > discussion, or should we clarify this first?
>
> Feel free to post it. But to merge the series, I think we need to address
> this.
>
> --
> Regards,
>
> Sakari Ailus
> sakari.ailus@xxxxxxxxxxxxxxx
Attachment:
signature.asc
Description: PGP signature
