Re: Question: interaction between selection API, ENUM_FRAMESIZES and S_FMT?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Hans

On Mon, 24 Jun 2013, Hans Verkuil wrote:

> Hi all,
> 
> While working on extending v4l2-compliance with cropping/selection test cases
> I decided to add support for that to vivi as well (this would give applications
> a good test driver to work with).
> 
> However, I ran into problems how this should be implemented for V4L2 devices
> (we are not talking about complex media controller devices where the video
> pipelines are setup manually).
> 
> There are two problems, one related to ENUM_FRAMESIZES and one to S_FMT.
> 
> The ENUM_FRAMESIZES issue is simple: if you have a sensor that has several
> possible frame sizes, and that can crop, compose and/or scale, then you need
> to be able to set the frame size. Currently this is decided by S_FMT which
> maps the format size to the closest valid frame size. This however makes
> it impossible to e.g. scale up a frame, or compose the image into a larger
> buffer.
> 
> For video receivers this issue doesn't exist: there the size of the incoming
> video is decided by S_STD or S_DV_TIMINGS, but no equivalent exists for sensors.

Isn't it a part of the current uncertainty of the type "who should scale?" 
Or what output format should be set on the sensor for any specific final 
user-facing output frame? I thought we decided not to try to become much 
smarter here with the classical V4L2 API, instead, those for whom this is 
really important should support subdevice- or even pad-level 
configuration?

> I propose that a new selection target is added: V4L2_SEL_TGT_FRAMESIZE.
> 
> However, this leads to another problem: the current S_FMT behavior is that
> it implicitly sets the framesize. That behavior we will have to keep, otherwise
> applications will start to behave differently.
> 
> I have an idea on how to solve that, but the solution is related to the second
> problem I found:
> 
> When you set a new format size, then the compose rectangle must be set to the
> new format size as well since that has always been the behavior in the past
> that applications have come to expect.
> 
> But this makes certain operations impossible to execute: if a driver can't
> scale, then you can never select a new format size larger than the current
> (possibly cropped) frame size, even though you would want to compose the
> unscaled image into such a larger buffer.
> 
> So what is the behavior that I would expect from drivers?
> 
> 1) After calling S_STD, S_DV_TIMINGS or S_SELECTION(V4L2_SEL_TGT_FRAMESIZE)
> the cropping, composing and format parameters are all adjusted to support the
> new input video size (typically they are all set to the new size).
> 
> 2) After calling S_CROP/S_SELECTION(CROP) the compose and format parameters are all
> adjusted to support the new crop rectangle.
> 
> 3) After calling S_SEL(COMPOSE) the format parameters are adjusted.
> 
> 4) Calling S_FMT validates the format parameters to support the compose
> rectangle.
> 
> However, today step 4 does something different: the compose rectangle will be
> adjusted to the format size (and in the case of a sensor supporting different
> framesizes the whole pipeline will be adjusted).
> 
> The only way I see that would solve this (although it isn't perfect) is to
> change the behavior of S_FMT only if the selection API was used before by the
> filehandle. The core can keep easily keep track of that. When the application
> calls S_FMT and no selection API was used in the past by that filehandle, then
> the core will call first S_SELECTION(V4L2_SEL_TGT_FRAMESIZE). If that returns
> -EINVAL, then it will call S_SELECTION(V4L2_SEL_TGT_COMPOSE). Finally it will
> call S_FMT. Note that a similar sequence is needed for the display case.
> 
> This means that a driver supporting the selection API can implement the logical
> behavior and the core will implement the historically-required illogical part.
> 
> So the fix for this would be to add a new selection target and add compatibility
> code to the v4l2-core.
> 
> With that in place I can easily add crop/compose support to vivi.
> 
> One area of uncertainty is how drivers currently implement S_FMT: do they reset
> any cropping done before? They should keep the crop rectangle according to the
> spec (well, it is implied there). Guennadi, what does soc_camera do?

No, soc-camera core doesn't touch cropping parameters in s_fmt. Similarly 
host drivers aren't expected to do that.

Thanks
Guennadi

> Sylwester, I am also looking at exynos4-is/fimc-lite.c. I do see that setting
> the compose rectangle will adjust it to the format size instead of the other
> way around, but I can't tell if setting the format size will also adjust the
> compose rectangle if that is now out-of-bounds of the new format size.
> 
> Comments? Questions?
> 
> Regards,
> 
> 	Hans
> 

---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]
  Powered by Linux