Hi Laurent, We not only need t(dqbuf) - t(exposure), we are also interested in t(dqbuf) - t(frame_recieved) to know more detail inside the camera for analysis. Thanks, Esker On Thu, Nov 9, 2023 at 7:34 PM Laurent Pinchart <laurent.pinchart@xxxxxxxxxxxxxxxx> wrote: > > Hi Esker, > > On Thu, Nov 09, 2023 at 08:59:13AM +0800, Esker Wong wrote: > > Hi Laurent, > > > > The use case here we want is actually the latency. We want to know > > what is the most accurate delay the user will feel when the camera > > capture the frame to the frame get displayed. So anytime earlier then > > the dqbuf should be an improvement here. It would help us know better > > of a camera quality. > > If it's the latency you're after, wouldn't it be best to use the > timestamp produced by the camera ? That's the most accurate information. > It is expressed relative to the camera internal clock, but UVC transmits > information that enables conversion of the value to a system timestamp. > > > On Thu, Nov 9, 2023 at 8:03 AM Laurent Pinchart wrote: > > > On Wed, Nov 08, 2023 at 11:46:40PM +0100, Ricardo Ribalda wrote: > > > > On Wed, 8 Nov 2023 at 21:32, <nicolas@xxxxxxxxxxxx> wrote: > > > > > > > > > > The fact that you interpret the time from FRAME_SYNC to DQBUF (well the > > > > > READ IO notification) as the actual latency is yours of course. It > > > > > assumes that the camera on the other end does not introduce other > > > > > > > > We want to use this signal to measure how much power is used since we > > > > start receiving the frame until we can use it. > > > > I agree with you that the latency between capture and dqbuf should be > > > > measured using the timestamp. That is not our use case here. > > > > > > > > > source of latency (or that these are negligible). You are also going to > > > > > introduce a lot of jitter, since it relies on when the OS decides to > > > > > wake up your process. > > > > > > > > We have measured a jitter of around 2.5 msec, which is acceptable for our needs. > > > > > > > > > I think my opinion resides in if you can accurately *enough* implement > > > > > what the spec says for FRAME_SYNC then do it, otherwise just don't lie. > > > > > > > > What the specs says is: > > > > ``` > > > > Triggered immediately when the reception of a frame has begun > > > > ``` > > > > In my opinion, that is true for usb devices, we are triggering it as > > > > soon as the transfer has started to the eyes of the driver. We cannot > > > > trigger earlier than that. > > > > > > > > > > > > > I think for ISO, "after the first chunk" i a small lie, but acceptable. > > > > > But for BULK, the way it was explained is that it will be always very > > > > > close to DQBUF time. and it should not emit FRAME_SYNC for this type of > > > > > UVC device. If it fits other events fine of course, I'm just making a > > > > > judgment on if its fits V4L2_EVENT_FRAME_SYNC or not. > > > > > > > > nit: I believe that you have swapped iso and bulk on this description > > > > > > I've confused the USB packet size and the UVC payload size. The latter > > > is typically much bigger for bulk devices than isoc devices, but the > > > former will be in similar order of magnitudes in a large number of > > > cases, but not all cases. > > > > > > The URB size is the result of the USB packet size and number of packets > > > per URB. The uvcvideo driver currently sets the number of packets per > > > URB to 32 at most (and lowers it if the frame size is small, or if not > > > enough memory can be allocated). This could be increased or made dynamic > > > in the future, as higher speeds typically benefit from larger URB sizes. > > > The packet size differs between bulk and isoc endpoints. > > > > > > For bulk, the packet size can be up to 512 bytes for USB 2.0 and 1024 > > > bytes for USB 3.0, and the device can select a smaller size. The largest > > > URB size (again based on the current implementation of the uvcvideo > > > driver) is thus 32 KiB. > > > > > > For isochronous the situation is more complicated. The term "packet" as > > > used in the uvcvideo driver actually means all the data transferred in > > > one service interval, thus made of multiple isoc packets. It is heavily > > > dependent on the USB speed, and the device can advertise different > > > supported sizes (which translate directly to the reserved bandwidth for > > > the transfer), with the driver picking the smallest bandwidth large > > > enough for the data rate required by the resolution and frame rate. The > > > theoretical worst case is 1024 bytes per isoc packet * 16 isoc packets > > > per burst * 6 burst per interval * 32 "packets" per URB, equal to 3 MiB. > > > > > > Even with the largest URB size you have witnessed of ~1 MiB, we will end > > > up lying quite a bit if we consider the URB completion callback for the > > > first URB of the frame as indicating the start of reception. > > > > > > > > In term of accuracy, if timestamp was passed with the FRAME_SYNC event, > > > > > it would not matter how fast your process the event anymore and greatly > > > > > improve accuracy. > > > > > > > > +1 to that. If we could easily change the uAPI for FRAME_SYNC that > > > > should definitely be implemented. > > > > > > > > > > Not to mention that the UVC timestamping requires a bit of love. > > > > > > > > > > > > @Laurent Pinchart, @Kieran Bingham any progress reviewing :P : > > > > > > https://patchwork.linuxtv.org/project/linux-media/list/?series=10083 > > > > > > > > > > Thanks for working on this by the way, hope someone will find the time > > > > > to review this. The timestamps should in theory provide a jitter free > > > > > > > > It already has a couple of Reviewed-by stamped in.... ;) > > > > > > > > > measurement of the delay Esker is trying to measure, and if it wasn't > > > > > of bugs (and crazy complexity) it would in the worst case match the > > > > > transfer time. > > > > > > > > Sorry to repeat myself, but just to avoid the confusion: Esker needs > > > > to know how much power is used since we start receiving a frame until > > > > it is available for dqbuf, not de frame latency. > > > > > > As I think everybody is aware, the earliest notification you get on the > > > CPU side is the *end* of reception of the first URB, which can possibly > > > be significantly later than the start of reception of the frame. > > > > > > Based on what I understand, the goal is to measure the CPU power > > > consumption related to CPU processing of the frame. If that's the case, > > > there's good and bad news. The good news is that the CPU doesn't process > > > the frame at all until the URB has been received (if you were to measure > > > the power consumption of the USB host controller too, it would be a > > > different story), so the delay shouldn't be a problem. The bad news is > > > that I don't see how the information you're trying to get will help you, > > > as there's plenty of other things unrelated to the uvcvideo driver that > > > can take CPU time while a frame is being received. That may not be any > > > of my business, but from the point of view of the uvcvideo driver, I'm > > > less inclined to accept a possibly significant V4L2_EVENT_FRAME_SYNC lie > > > if the use case ends up making little sense :-) > > -- > Regards, > > Laurent Pinchart
