On Tue, Apr 18, 2023, Avichal Rakesh wrote: > On Tue, Apr 18, 2023 at 6:07 PM Alan Stern <stern@xxxxxxxxxxxxxxxxxxx> wrote: > > > > On Tue, Apr 18, 2023 at 03:45:53PM -0700, Avichal Rakesh wrote: > > > I see, and I think I understand Greg's previous comment better as > > > well: The UVC driver isn't falling behind on the video stream, it is > > > falling behind the usb controller's monotonic isoc stream. > > > > > > From what I can see, this leaves us in an interesting place: UVC > > > allows the host to configure the camera's output resolution and fps, > > > which effectively controls how fast the camera is generating data. > > > This is at odds with the UVC gadget driver, which currently packs each > > > video frame into as few usb_requests as possible (using the full > > > available size in usb_requests). Effectively, the UVC gadget driver > > > attempts to use the "full" bandwidth of isoc transfers even when the > > > camera isn't generating data fast enough. For example, in my > > > observations: 1 video frame is ~22kB. At 30fps, this represents 1/30 > > > of the amount of data the camera would generate in a second. This 22kB > > > is split into 8 usb_requests which is about 1/1000 the number of > > > requests UVC driver needs to generate per second to prevent isoc > > > failures (assuming 125us monotonic uframes). Assuming some fudge > > > factor from the simplifications in your explanation gives the uvc > > > driver some extra leeway with request queuing, we're still roughly two > > > order of magnitudes out of sync. Even with perfect 'complete' > > > callbacks and video frame encodings, an underrun seems inevitable. > > > Data is being generated at a far slower rate than it is being > > > transferred. Does this reasoning seem valid? > > > > > > Just as a test I'll try updating the UVC driver to consume 266 > > > usb_requests per video frame (~1/30 of 8000), which should be enough > > > to keep the usb controller queue occupied for ~1/30s. Ideally, by the > > > time the controller queue is empty, the camera would have produced a > > > new frame. This doesn't solve the issue with latencies around callback > > > and an isoc failure might still happen, hopefully the failure > > > frequency is reduced because UVC queues enough requests per video > > > frame to not starve the controller's queue while waiting on a new > > > frame and the only way they go out of sync is from 'complete' callback > > > timings. I am assuming this has been tried before, but my LKML search > > > skills are failing and I can't find much on it. > > > > Note that there's nothing wrong with submitting a 0-length isochronous > > transfer. If there's no data left but you still need to send > > _something_ in order to fill out the remaining slots in the controller's > > schedule, this is a good way to do it. > > > Oh, this is very good to know, thank you!! We just need to reach a > steady state of UVC queuing enough requests monotonically (even if > they are empty), and the usb controller calling the 'complete' > callback to give it more requests to queue. Although I wonder how the > host's UVC driver would interpret the zero length packets, if it would > even care. By the usb spec, for IN direction, if there's no data available and the host requests for data, then the device will send a zero-length data packet. This is what the dwc3 controller will do. So regardless whether you prepare and queue a 0-length request or not, the host will receive it. > > I am unfortunately being pulled into some other work for the next few > days, but I will try out both: splitting one frame into many many > requests and just sending 0 length requests, and see what happens on > the host. Will report back with what I find. Any other insights are > welcome. I want to fix this problem for good if possible, and am happy > to try out whatever it takes! That would be great. Thanks. BR, Thinh
