HI Sakari, On Fri, May 03, 2019 at 12:32:02AM +0300, Sakari Ailus wrote: > Hi Jacopo, > > On Sat, Apr 27, 2019 at 01:00:42AM +0200, Jacopo Mondi wrote: > > HI Sakari, > > > > On Tue, Apr 23, 2019 at 04:28:40PM +0300, Sakari Ailus wrote: > > > Hi Jacopo, > > > > > > On Tue, Apr 23, 2019 at 12:19:32PM +0200, Jacopo Mondi wrote: > > > > Hi Sakari, > > > > > > > > On Wed, Apr 17, 2019 at 01:41:48PM +0300, Sakari Ailus wrote: > > > > > Hi Jacopo, > > > > > > > > > > I promised to write a bit about the routing problem I attempted to write on > > > > > #v4l. It's a bit late, but here it is. Let me know if you share my > > > > > understanding (and especially do so if you don't :)). > > > > > > > > Thanks for doing this, and sorry it took me a while to reply! > > > > > > > > > > > > > > A route is identified by the source and sink pads as well as the stream ID > > > > > on the source and the sink pads. Its flags allow enabling or disabling a > > > > > route. > > > > > > > > > > Most devices that function as transmitters (such as sensors) have a fixed > > > > > number of routes they can support. These's no need to change them; enabling > > > > > and disabling them will be enough for the user. > > > > > > > > > > > > > Fine so far. > > > > > > > > We indeed define routes as: > > > > (sink pad - sink stream) -> (source pad - source stream) > > > > > > > > > For receivers this is different. What needs to be supported on the receiver > > > > > side is directing any incoming stream (a 32-bit unsigned integer) to any > > > > > source pad. > > > > > > > > > > This is because pads are not alike --- one may be connected to a block that > > > > > further processes the image whereas the others may be connected to a DMA > > > > > engine, just writing the data to memory. > > > > > > > > > > The receivers also may not make assumptions beyond the sub-device API: the > > > > > stream is a 32-bit unsigned integer, there is currently no API requirement > > > > > to have the stream IDs on a particular low integer range. In principle we > > > > > could define that range, but I'd rather try to find better solutions than > > > > > that: it's hard to come up with a number as it depends on the hardware. > > > > > Some kind of an upper limit could be the number of CSI-2 channels > > > > > multiplied by CSI-2 data types. That would be enough for CSI-2. 16 or 32 > > > > > virtual channels and up to 64 data types would mean up to 2048 routes > > > > > between a demultiplexer's sink pad and *each* of its source pads. And this > > > > > comes with an assumption that the source pads only support a single > > > > > stream... > > > > > > > > I lost you here. > > > > > > > > My mental model was far more (too?) simple: > > > > > > > > - Routing tables inside an entity might have an arbitrary size, as how > > > > that configuration depends on the device and the driver > > > > implementation. As an example, the adv748x accepts 7 analogue > > > > inputs to chose from and route to a CSI-2 source pad. > > > > depending on the implementation, those 7 inputs could be modeled as > > > > 7 pads with one stream each, or a single pad with 7 channels or > > > > whatever else, and I agree the 'stream_id' values range is totally > > > > up to the driver implementation -inside an entity-. > > > > > > > > - The cross-entity (sorry, I lack a better term here) multiplexing > > > > happens on physical bus that allows so, and I can only think of > > > > CSI-2 at the moment. Sure, you could share the lines of a parallel > > > > bus playing with enables/disables of the transmitters, but this > > > > is a custom hack that does not play well in this model. > > > > Each CSI-2 source pad has up to 4 streams (one per VC) and the > > > > content of those streams is retrieved from the transmitter by the > > > > receiver through the remote frame_desc operation [*], as receivers > > > > might need to be setup using to filter on particular VC/DT combinations > > > > to receive the stream. > > > > > > Note that the concept of a "stream" is different in our proposed API and in > > > CSI-2 (virtual channel): on CSI-2 virtual channels can carry multiple data > > > types but in our definition they do not. That means there will be more > > > streams on SUBDEV_[GS]_ROUTING API than there are on a CSI-2 bus. > > > > > > We also do not limit the use of the new API to old CSI-2 D-PHYs that > > > only support four virtual channels. Therefore we need to think how this > > > would work on 16 or 32 virtual channels, and preferrably not posing > > > problems with larger numbers either. > > > > > > > Indeed, each stream carries one DT and can be sent in one VC, so at least > > 64*4 possible routes (per source pad) for what is currently specified by > > CSI-2 and maybe up to 64*32 to have space for future extensions as you > > explained below. > > > > I agree, 2048 routes per source pad is not an easily manageable number > > of entries for a routing table. > > > > > > > > > > - Each CSI-2 receiver sink pad supports 4 streams (the CSI-2 VCs) and any > > > > of those streams can be directed to any of its source pads, to > > > > implement what you have described (one pad connected to an ISP-like > > > > device, on other to the DMA engine directly etc) > > > > > > > > - DT negotiation is still a bit vague. The example we have on the > > > > series (adv748x and r-car csi2) configure the receiver's accepted DT > > > > inspecting the remote frame_desc. Again, as per [*] this migh be > > > > limitied to 1 DT for VC, which might not be enough and would require > > > > re-thinking the operation used for the negotiation. > > > > > > > > - Configuring a multiplexed source pad image format is today not > > > > possible, the format is always propagated from the sink pad to which > > > > a route is enabled to the source pad. Might this be a limitation on > > > > how we control which DTs are multiplexed inside a VC and won't allow > > > > to model any format conversion that might happen at the source pad > > > > output. I was toying myself with the idea of a stream-aware set > > > > format operation for multiplexed source pads, not sure it might work > > > > though. > > > > > > > > Can I ask you why: > > > > - you mention 16 or 32 VCs ? Each CSI-2 link supports up to 4. > > > > > > See above. > > > > > > > - you put DT in the routing mix, and I suspect it is here where our > > > > disconnection happens. I always assume DT configuration as a result > > > > of a format configuration operation, which currently has limitations > > > > as noted here above. > > > > > > Yes; DT comes from the format and is generally determined by the > > > transmitting driver. Nothing else limits the number of DTs on a VC except > > > the number of DTs --- which is currently 64. > > > > > > > > > > > > > > > [*] I would later like to talk about if this is the most appropriate > > > > operation to handle this negotiation, as I'm not sure we can handle DT > > > > negotiation properly with that, but that's for later. > > > > > > > > > > > > > > CSI-2 receivers support a number of simultaneous streams, and as the stream > > > > > is demultiplexed there, this means there will be as many source pads as > > > > > there are supported simultaneous streams. This heavily depe`nds on the > > > > > hardware, but the number could be e.g. 1, 4 or 8. Much smaller than 2048 in > > > > > any case. > > > > > > > > > > > > > Why 8? Is this related to DT multiplexing again? > > > > > > Yes and no. It's just a random number between 1 and 2048. :-) > > > > > > > > > > > > Another option could be creating no routes at all at device init time, and > > > > > letting the user create them. We could add a new flag for routes telling > > > > > that a route is dynamic: disabling a dynamic route would simply delete it. > > > > > Likewise, a SUBDEV_G_ROUTING returning no routes (but no error either) > > > > > would also tell the user only dynamic routes are supported. > > > > > > > > > I see your point, but I cannot see how one would easily map a stream_id to a > > DT/VC combination. One way would be: > > DT = stream_id % 64; VC = stream_id / 64; > > > > Just to make sure we're on the same page, how would you enable/disable > > transmitting, say, embedded data (DT=18) on VC=2? By enabling/disabling > > the stream_id 146 ( = 18 + 64 * 2) ? > > The stream ID that is used by the receiver must match that used on the > transmitter. So presumably the transmitter will use a reasonable small > integer number for that stream. The number of streams on the transmitter > side is typically small, even if the maximum possible value is large. All > the user needs to do is to use the same integer on the receiver's routing > setup. The virtual channel or the data type aren't visible in the uAPI > anyway. Sorry, but still I don't get how you would control enablement/disablement of DTs using the routing APIs, and I'm not sure, if that's the idea, it actually scales. > > Does this answer the question? We could try discussing this on #v4l to > converge faster. :-) > I'll be around on irc tomorrow, let's see if our timezones overlaps :) > -- > Kind regards, > > Sakari Ailus > sakari.ailus@xxxxxxxxxxxxxxx
Attachment:
signature.asc
Description: PGP signature
