Hi Mauro, On 07/29/2011 06:02 AM, Mauro Carvalho Chehab wrote: > Em 28-07-2011 19:57, Sylwester Nawrocki escreveu: >> On 07/28/2011 03:20 PM, Mauro Carvalho Chehab wrote: > >>> Accumulating sub-dev controls at the video node is the right thing to do. >>> >>> An MC-aware application will need to handle with that, but that doesn't sound to >>> be hard. All such application would need to do is to first probe the subdev controls, >>> and, when parsing the videodev controls, not register controls with duplicated ID's, >>> or to mark them with some special attribute. >> >> IMHO it's not a big issue in general. Still, both subdev and the host device may >> support same control id. And then even though the control ids are same on the subdev >> and the host they could mean physically different controls (since when registering >> a subdev at the host driver the host's controls take precedence and doubling subdev >> controls are skipped). > > True, but, except for specific usecases, the host control is enough. For some ones those "specific use cases" could be most important (or all) the use cases. :-) > >> Also there might be some preference at user space, at which stage of the pipeline >> to apply some controls. This is where the subdev API helps, and plain video node >> API does not. > > Again, this is for specific usecases. On such cases, what is expected is that the more > generic control will be exported via V4L2 API. IMHO it's not about use cases, just about which hardware can be covered by the API and which can not. When the signal processing systems are becoming distributed and more interactively reconfigurable it's getting impossible to reasonably control them through a centralized interface. > >>> >>>> Thus it's a bit hard to imagine that we could do something like "optionally >>>> not to inherit controls" as the subdev/MC API is optional. :) >>> >>> This was actually implemented. There are some cases at ivtv/cx18 driver where both >>> the bridge and a subdev provides the same control (audio volume, for example). The >>> idea is to allow the bridge driver to touch at the subdev control without exposing >>> it to userspace, since the desire was that the bridge driver itself would expose >>> such control, using a logic that combines changing the subdev and the bridge registers >>> for volume. >> >> This seem like hard coding a policy in the driver;) Then there is no way (it might not >> be worth the effort though) to play with volume level at both devices, e.g. to obtain >> optimal S/N ratio. > > In general, playing with just one control is enough. Andy had a different opinion > when this issue were discussed, and he thinks that playing with both is better. > At the end, this is a developers decision, depending on how much information > (and bug reports) he had. > >> This is a hack...sorry, just joking ;-) Seriously, I think the >> situation with the userspace subdevs is a bit different. Because with one API we >> directly expose some functionality for applications, with other we code it in the >> kernel, to make the devices appear uniform at user space. > > Not sure if I understood you. V4L2 export drivers functionality to userspace in an > uniform way. MC api is for special applications that might need to access some > internal functions on embedded devices. Yes, that's what I meant. > > Of course, there are some cases where it doesn't make sense to export a subdev control > via V4L2 API. > >>>> Also, the sensor subdev can be configured in the video node driver as well as >>>> through the subdev device node. Both APIs can do the same thing but in order >>>> to let the subdev API work as expected the video node driver must be forbidden >>>> to configure the subdev. >>> >>> Why? For the sensor, a V4L2 API call will look just like a bridge driver call. >>> The subdev will need a mutex anyway, as two MC applications may be opening it >>> simultaneously. I can't see why it should forbid changing the control from the >>> bridge driver call. >> >> Please do not forget there might be more than one subdev to configure and that >> the bridge itself is also a subdev (which exposes a scaler interface, for instance). >> A situation pretty much like in Figure 4.4 [1] (after the scaler there is also >> a video node to configure, but we may assume that pixel resolution at the scaler >> pad 1 is same as at the video node). Assuming the format and crop configuration >> flow is from sensor to host scaler direction, if we have tried to configure _all_ >> subdevs when the last stage of the pipeline is configured (i.e. video node) >> the whole scaler and crop/composition configuration we have been destroyed at >> that time. And there is more to configure than VIDIOC_S_FMT can do. > > Think from users perspective: all user wants is to see a video of a given resolution. > S_FMT (and a few other VIDIOC_* calls) have everything that the user wants: the > desired resolution, framerate and format. > > Specialized applications indeed need more, in order to get the best images for > certain types of usages. So, MC is there. > > Such applications will probably need to know exactly what's the sensor, what are > their bugs, how it is connected, what are the DSP blocks in the patch, how the > DSP algorithms are implemented, etc, in order to obtain the the perfect image. I suspect we might need more the specialized, H/W aware libraries rather than applications. It's also about efficient hardware utilisation. It's often important to get most out of the device, i.e. be able to precisely configure all the bits and pieces, depending on the use case. This could be done by having as thin kernel driver as possible and a H/W dedicated library. > Even on embedded devices like smartphones and tablets, I predict that both > types of applications will be developed and used: people may use a generic > application like flash player, and an specialized application provided by > the manufacturer. Users can even develop their own applications generic > apps using V4L2 directly, at the devices that allow that. > > As I said before: both application types are welcome. We just need to warrant > that a pure V4L application will work reasonably well. Is Laurent pointed out, more complex devices could be supported through libv4l2. > >> Allowing the bridge driver to configure subdevs at all times would prevent >> the subdev/MC API to work. > > Well, then we need to think on an alternative for that. It seems an interesting > theme for the media workshop at the Kernel Summit/2011. Sure. But isn't it that simple as letting the driver implement MC API only and abstract the hardware details in dedicated library/libv4l2 plugin ? I just don't see a point in moving the adaptation layer into the kernel. > >>>> There is a conflict there that in order to use >>>> 'optional' API the 'main' API behaviour must be affected.... >>> >>> It is optional from userspace perspective. A V4L2-only application should be able >>> to work with all drivers. However, a MC-aware application will likely be specific >>> for some hardware, as it will need to know some device-specific stuff. >>> >>> Both kinds of applications are welcome, but dropping support for V4L2-only applications >>> is the wrong thing to do. >>> >>>> And I really cant use V4L2 API only as is because it's too limited. >>> >>> Why? >> >> For instance there is really yet no support for scaler and composition onto >> a target buffer in the Video Capture Interface (we also use sensors with >> built in scalers). It's difficult to efficiently manage capture/preview pipelines. >> It is impossible to discover the system topology. > > Scaler were always supported by V4L2: if the resolution specified by S_FMT is not > what the sensor provides, then scale. All non-embedded drivers with sensor or bridge > supports scale does that. This is not really a level of support I have been thinking about. I thought about user space knowing what exactly is scaled and where. > > Composition is not properly supported yet. It could make sense to add it to V4L. How do you > think MC API would help with composite? Yes, the new S_SELECTION ioctl replacing S_CROP is meant to add composition support. With MC you can support composition through cropping on a source pad, however the ioctl naming might yet need to be revisited. > > Managing capture/preview pipelines will require some support at V4L2 level. This is > a problem that needs to be addressed there anyway, as buffers for preview/capture > need to be allocated. There's an RFC about that, but I don't think it covers the > pipelines for it. It requires fairly complex hardware configuration in hardware assisted cases. So it's better done at MC API level. Nevertheless there is much that can be improved at V4L2 level too. > > Discovering the system topology indeed is not part of V4L2 API and will never be. > This is MC API business. There's no overlap with V4L2. True, however I was referring to the V4L2 API (video node) _only_. > >>>> Might be that's why I see more and more often migration to OpenMAX recently. >>> >>> I don't think so. People may be adopting OpenMAX just because of some marketing strategy >>> from the OpenMAX forum. We don't spend money to announce V4L2 ;) >> >> :) >> >>> >>> I think that writing a pure OpenMAX driver is the wrong thing to do, as, at the long >>> term, it will cost _a_lot_ for the vendors to maintain something that will never be >>> merged upstream. >> >> In general it depends on priorities. For some chip vendors it might be more important >> to provide a solution in short time frame. Getting something in mainline isn't effortless >> and spending lot's of time on this for some parties is unacceptable. > > Adding something at a forum like OpenMAX is probably not an easy task ;) It generally > takes a long time to change something on open forum specifications. Also, participating > on those forums require lots of money, with membership and travel expenses. > > Adding a new driver that follows the API's is not a long-term effort. The delay is > basically one kernel cycle, e. g. about 3-4 months. > > Most of the delays on merging drivers for embedded systems, however, take a > way longer than that, unfortunately. From what I see, what is delaying driver > submission is due to: > - the lack of how to work with the Linux community. Some developers take a > long time to get the 'modus operandi'; > - the need of API changes. It is still generally faster to add a new API > addition at the Kernel than on an open forum; > - the discussions inside the various teams (generally from the same company, > or the company and their customers) about the better way to implement some feature. > > All the above also happens when developing an OpenMAX driver: companies need to > learn how to work with the OpenMax forums, API changes may be required, so they > need to participate at the forums, the several internal teams and customers > will be discussing the requirements. > > I bet that there's also one additional step: to submit the driver to some company > that will check the driver compliance with the official API. Such certification > process is generally expensive and takes a long time. > > At the end of the day, they'll also spend a lot of time to have the driver done, > or they'll end by releasing a "not-quite-openmax" driver, and then needing to > rework on that, due to customers complains, or to certification-compliance, > loosing time and money. >From what I can see many of the big players in the graphics devices world are already a _contributor_ members of the Khronos Group. And the membership costs are probably a piece of cake for such a big companies. OMX is different from V4L2 in that there is no need to discuss every trivial detail in the API, specific for particular H/W. Adopters just add proprietary extensions when needed, not every detail is conservatively fixed in the API. This can't really be done in video4linux. But we also shouldn't be forcing users to use an API that limits H/W capabilities, _if_ we are interested in supporting modern complex devices in V4L2. -- Regards, Sylwester -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
