Hi Hans,
Hans Verkuil wrote:
On Sat September 22 2012 14:38:07 Sakari Ailus wrote:
Hi Hans,
Thanks for the comments.
Hans Verkuil wrote:
On Thu September 20 2012 22:21:22 Sakari Ailus wrote:
...
Capability flag for monotonic timestamps
----------------------------------------
A capability flag can be used to tell whether the timestamp is monotonic.
However, it's not extensible cleanly to provide selectable timestamps. These
are not features that are needed right now, though.
The upside of this option is ease of implementation and use, but it's not
extensible. Also we're left with a flag that's set for all drivers: in the
end it provides no information to the user and is only noise in the spec.
Control for timestamp type
--------------------------
Using a control to tell the type of the timestamp is extensible but not as
easy to implement than the capability flag: each and every device would get
an additional control. The value should likely be also file handle specific,
and we do not have file handle specific controls yet.
Yes, we do. You can make per-file handle controls. M2M devices need that.
Thanks for correcting me.
I'm not sure why this would be filehandle specific, BTW.
Good point. I thought that as other properties of the buffers are
specific to file handles, including format when using CREATE_BUFS, it'd
make sense to make the timestamp source file-handle specific as well.
What do you think?
I don't think it makes sense to have different streams from the same device
use different clocks.
In the meantime the control could be read-only, and later made read-write
when the timestamp type can be made selectable. Much of he work of
timestamping can be done by the framework: drivers can use a single helper
function and need to create one extra standard control.
Should the control also have an effect on the types of the timestamps in
V4L2 events? Likely yes.
You are missing one other option:
Using v4l2_buffer flags to report the clock
-------------------------------------------
By defining flags like this:
V4L2_BUF_FLAG_CLOCK_MASK 0x7000
/* Possible Clocks */
V4L2_BUF_FLAG_CLOCK_UNKNOWN 0x0000 /* system or monotonic, we don't know */
V4L2_BUF_FLAG_CLOCK_MONOTONIC 0x1000
you could tell the application which clock is used.
This does allow for more clocks to be added in the future and clock selection
would then be done by a control or possibly an ioctl. For now there are no
plans to do such things, so this flag should be sufficient. And it can be
implemented very efficiently. It works with existing drivers as well, since
they will report CLOCK_UNKNOWN.
I am very much in favor of this approach.
Thanks for adding this. I knew I was forgetting something but didn't
remember what --- I swear it was unintentional! :-)
If we'd add more clocks without providing an ability to choose the clock
from the user space, how would the clock be selected? It certainly isn't
the driver's job, nor I think it should be system-specific either
(platform data on embedded systems).
IF a driver supports more than one clock (which I really don't see happening
anytime soon), then we either need a control to select the clock or an ioctl.
The timestamps can be taken from any standard clock using a helper
function. Right now drivers call do_gettimeofday() or ktime_get_ts(),
it's the same whether the function was called e.g. v4l2_get_ts(). The
driver doesn't even need to know which timer is being used to make that
timestamp. It used to be the realtime clock and soon it's the monotonic
clock. So I really don't see why the timestamp source should depend on
the driver. It rather depends on what the user wants.
That said, I also don't see use for other clocks _right now_ than the
monotonic one.
And something as well to enumerate the available clocks. I'm leaning towards
ioctls, but I think this should be decided if we ever get an actual use-case
for this.
I think it should be a control --- I see no reason to add new IOCTLs
when controls are equally, or even better, fit for the same job. But
let's decide this only when the functionality is needed.
It's up to the application and its needs. That would suggest we should
always provide monotonic timestamps to applications (besides a potential
driver-specific timestamp), and for that purpose the capability flag ---
I admit I disliked the idea at first --- is enough.
What comes to buffer flags, the application would also have to receive
the first buffer from the device to even know what kind of timestamps
the device uses, or at least call QUERYBUF. And in principle the flag
should be checked on every buffer, unless we also specify the flag is
the same for all buffers. And at certain point this will stop to make
any sense...
It should definitely be the same for all buffers. And since apps will
typically call querybuf anyway I don't see this as a problem. These
clocks are also specific to the streaming I/O API, so reporting this as
part of that API makes sense to me as well.
>
A capability flag is cleaner solution from this perspective, and it can
be amended by a control (or an ioctl) later on: the flag can be
disregarded by applications whenever the control is present.
Yuck.
What's so bad in it then? The user doesn't need to start dealing with
buffers whenever (s)he needs to know what kind of timestamps the device
provides.
Timestamps are also used on the event interface (and also on subdevs),
which currently uses monotonic timestamps (at least it was intended to
do so, it's missing from the documentation). The timestamps on V4L2 and
V4L2 subdev must match to be useful for the application, so we should
not allow using other clock than monotonic before we truly make the
timestamp source selectable by the application.
A flash subdev, for example, could produce events while the driver
producing timestamps for the video buffers doesn't know the type of the
timestamp used by the flash driver. One still must be able to compare
the two in the user space to make use of them.
I'm ok-ish with using video buffer flags to tell the type of the
timestamp, but those flags might not be useful (like the capability
flag) if the timestamp source is made selectable. What I still don't
like is that you have to start dealing with buffers before you know what
kind of timestamps you're getting.
I'd like to have Laurent's and Remi's opinion before proceeding to the
implementation.
...
Device-dependent timestamp
--------------------------
Should we agree on selectable timestamps, the existing timestamp field (or a
union with another field of different type) could be used for the
device-dependent timestamps.
No. Device timestamps should get their own field. You want to be able to relate
device timestamps with the monotonic timestamps, so you need both.
Alternatively we can choose to re-use the
existing timecode field.
At the moment there's no known use case for passing device-dependent
timestamps at the same time with monotonic timestamps.
Well, the use case is there, but there is no driver support. The device
timestamps should be 64 bits to accomodate things like PTS and DTS from
MPEG streams. Since timecode is 128 bits we might want to use two u64 fields
or perhaps 4 u32 fields.
That should be an union for different kinds (or rather types) of
device-dependent timestamps. On uvcvideo I think this is u32, not u64.
We should be also able to tell what kind device dependent timestamp
there is --- should buffer flags be used for that as well?
That's definitely part of the buffer flags. The presence of timecode is already
signalled using that. And not every buffer may have device timestamps (that
depends on the hardware), so you have to signal it through the buffer flags.
An anonymous union might be best with the buffer flags signalling the type of
the union. What I don't know is how to specify the type. Shall we just specify
the type of the union (e.g. 4 u32 fields or 2 u64 fields) and leave the
interpretation of those fields up to the application based on the driver name?
Or shall the type act more like a fourcc in that it also uniquely identifies
the interpretation of the timestamps?
I think it should uniquely identify it, at least if the MPEG timestamps
produced by some devices are relevant for video buffers. In this case it
wouldn't make much sense for the applications having to know what are
the drivers producing such timestamps. And this might not be the only
case like that.
Or should all device timestamps be converted to a timespec by the driver?
Good question.
Answers on a postcard.
Who are you going to send it? ;-)
Cheers,
--
Sakari Ailus
sakari.ailus@xxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
