On Thu, 13 Sep 2012, Laurent Pinchart wrote: > Hi Alan, > > Sorry for the late answer, I've been travelling with no internet connection. No problem; thanks for joining the discussion. > For video ISO IN problems can be split in two categories. > > Small delays (< one video frame) should already be properly supported as far > as I can see. Data will be lost, drivers will detect that, either immediately > if the protocol allows such detection, or at some video frame boundary. This > will result in a corrupt video frame with missing data, and drivers will mark > it as such when passing it to userspace through V4L2. Is this true even when the lost packets span a video frame boundary? > Larger delays that result in completely dropped video frames can cause more > serious problems. Drivers won't know how many video frames have been dropped, > and the V4L2 video frame sequence number will get out of sync. Applications > can work around the problem using the video frame timestamps, but that's more > a hack than a proper solution. > > I'm not sure if there is a way to solve the problem properly. ISO IN with > video typically uses the ASAP flag, There's no particular reason for this, is there? Currently the flag doesn't really do anything. > and making the USB core report an > artificial number of dropped ISO packets won't help much as many (most/all ?) > devices transmit video frames in a variable number of ISO packets. If the number of lost packets is small enough, the video driver can get it directly by looking at the status fields in the isochronous packet descriptor structures. If the number is too big, though, it won't be reported at all -- the driver will know only that a big gap occurred. >From what you say, this would be okay. > For video output the situation is currently unclear. The only V4L2 output > driver for USB devices that I know of is the uvcvideo driver, and the only UVC > output device I know of uses bulk transfers. > > Assuming we will get video ISO OUT devices in the future, the right way to fix > the problem will be to drop complete video frames on the host side if a large > delays occurs. Feedback endpoints (currently not supported in the uvcvideo > driver) might help (I've never worked with them, so I'd have to take a closer > look). As Clemens mentioned, feedback endpoints are meant to help with small differences in clock rates between the host and the device. They are not meant to help with large gaps in the data stream. > Another option would be to just check timestamps on the host side when > starting a new video frame. I'm not sure whether the USB core could help with > this problem. The best solution may be, if an URB submission fails with a -EXDEV error, to move forward to the next video frame boundary and restart transmitting from there. Depending on the protocol, you might want to delay the next URB submission so that the device doesn't see two frame boundaries too close together. Either way, you would set the URB_ISO_ASAP flag for the first URB and leave it turned off for all the following URBs. Basically what I'm asking is whether the API described in the documentation RFC (http://marc.info/?l=linux-usb&m=134382746009339&w=2) would be suitable for your needs. The main point is that there should be a way to let class drivers know when a large gap in the isochronous stream has occurred. In the absence of such a mechanism the driver would proceed merrily on its way, totally unaware that the stream had lost synchronization. Alan Stern -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
