Some cameras, such as the Sonix Technology Co. 292A (also presents as
the Microdia Webcam Vitade AF), allow multiple streams to be sent through the
same USB device. It was found during testing that occasionally, when
one H.264 stream sends a new keyframe during an MJPEG stream, a URB
buffer would be lost, and a frame is lost in the resulting capture.
This was because some EOF markers in URB buffers are lost. This particular
use case is not covered by the uvc_video driver, which causes two
consecutive buffers to be erroneously considered to be as part of
the same buffer, and lead to the whole erroneous buffer to be dropped.
This can be seen when using the current driver, as frames occasionally
are double the expected size, containing two images. Furthermore, the
interval between the frames was double that of what it should be (~66ms
as opposed to the usual ~33ms). Upon further investigation, it was found
that the packets within the erroneous buffer were being dropped.
We can detect the end of a frame by checking for a new SOI marker in the
JPEG. We should only check for this in the first two bytes after the
header, as the new packets are not corrupted and should be used at the
start of the next frame as part of a new buffer.
Check the buffer for a new SOI on JPEG streams and complete the
preceding buffer using EAGAIN, and invert the FID to make sure the
erroneous frame is not dropped. It should only do this check when there
is data after the header, as some packets will contain only a header and
no payload.
The changes in this series supply a new quirk for the UVC Driver, which
cause the buffers of MJPEG streams to be closed when a new JPEG SOI
marker is discovered, and be treated as a new buffer. This works around
the missing EOF marker and separates the buffers.
lsusb output:
Bus 001 Device 006: ID 0c45:6366 Microdia Webcam Vitade AF
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 [unknown]
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 64
idVendor 0x0c45 Microdia
idProduct 0x6366 Webcam Vitade AF
bcdDevice 1.00
iManufacturer 2 Sonix Technology Co., Ltd.
iProduct 1 292A-IPC-AR0330
iSerial 3 SN0001
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x05b9
bNumInterfaces 3
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 3
bFunctionClass 14 Video
bFunctionSubClass 3 Video Interface Collection
bFunctionProtocol 0
iFunction 5
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 1 Video Control
bInterfaceProtocol 0
iInterface 5
VideoControl Interface Descriptor:
bLength 14
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdUVC 1.00
wTotalLength 0x0074
dwClockFrequency 15.000000MHz
bInCollection 2
baInterfaceNr( 0) 1
baInterfaceNr( 1) 2
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 5
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 4
iTerminal 0
VideoControl Interface Descriptor:
bLength 28
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 3
guidExtensionCode {28f03370-6311-4a2e-ba2c-6890eb334016}
bNumControls 24
bNrInPins 1
baSourceID( 0) 2
bControlSize 3
bmControls( 0) 0xff
bmControls( 1) 0xff
bmControls( 2) 0xff
iExtension 0
VideoControl Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 6 (EXTENSION_UNIT)
bUnitID 4
guidExtensionCode {dddf7394-973e-4727-bed9-04ed6426dc67}
bNumControls 16
bNrInPins 1
baSourceID( 0) 3
bControlSize 2
bmControls( 0) 0xff
bmControls( 1) 0x01
iExtension 0
VideoControl Interface Descriptor:
bLength 18
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0201 Camera Sensor
bAssocTerminal 0
iTerminal 0
wObjectiveFocalLengthMin 0
wObjectiveFocalLengthMax 0
wOcularFocalLength 0
bControlSize 3
bmControls 0x0000000e
Auto-Exposure Mode
Auto-Exposure Priority
Exposure Time (Absolute)
VideoControl Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 5 (PROCESSING_UNIT)
Warning: Descriptor too short
bUnitID 2
bSourceID 1
wMaxMultiplier 0
bControlSize 2
bmControls 0x0000177f
Brightness
Contrast
Hue
Saturation
Sharpness
Gamma
White Balance Temperature
Backlight Compensation
Gain
Power Line Frequency
White Balance Temperature, Auto
iProcessing 0
bmVideoStandards 0x09
None
SECAM - 625/50
VideoControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 6
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 4
iTerminal 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0010 1x 16 bytes
bInterval 6
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 0
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 15
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 2
wTotalLength 0x02bf
bEndpointAddress 0x81 EP 1 IN
bmInfo 0
bTerminalLink 5
bStillCaptureMethod 0
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 0
bmaControls( 1) 0
VideoStreaming Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 6 (FORMAT_MJPEG)
bFormatIndex 1
bNumFrameDescriptors 7
bFlags 0
Fixed-size samples: No
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 1 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 1
bmCapabilities 0x00
Still image unsupported
wWidth 1920
wHeight 1080
dwMinBitRate 82920440
dwMaxBitRate 497522640
dwMaxVideoFrameBufferSize 2073600
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 2
bmCapabilities 0x00
Still image unsupported
wWidth 1280
wHeight 720
dwMinBitRate 73728000
dwMaxBitRate 442368000
dwMaxVideoFrameBufferSize 1843789
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 3
bmCapabilities 0x00
Still image unsupported
wWidth 640
wHeight 480
dwMinBitRate 24576000
dwMaxBitRate 147456000
dwMaxVideoFrameBufferSize 614989
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 4
bmCapabilities 0x00
Still image unsupported
wWidth 640
wHeight 360
dwMinBitRate 18432000
dwMaxBitRate 110592000
dwMaxVideoFrameBufferSize 461389
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 5
bmCapabilities 0x00
Still image unsupported
wWidth 320
wHeight 240
dwMinBitRate 6144000
dwMaxBitRate 36864000
dwMaxVideoFrameBufferSize 154189
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 6
bmCapabilities 0x00
Still image unsupported
wWidth 320
wHeight 180
dwMinBitRate 4608000
dwMaxBitRate 27648000
dwMaxVideoFrameBufferSize 115789
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 7 (FRAME_MJPEG)
bFrameIndex 7
bmCapabilities 0x00
Still image unsupported
wWidth 1920
wHeight 1080
dwMinBitRate 82920440
dwMaxBitRate 497522640
dwMaxVideoFrameBufferSize 2073600
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 27
bDescriptorType 36
bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED)
bFormatIndex 2
bNumFrameDescriptors 7
guidFormat {32595559-0000-0010-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
VideoStreaming Interface Descriptor:
bLength 30
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 1
bmCapabilities 0x00
Still image unsupported
wWidth 1920
wHeight 1080
dwMinBitRate 165888000
dwMaxBitRate 165888000
dwMaxVideoFrameBufferSize 4147200
dwDefaultFrameInterval 2000000
bFrameIntervalType 1
dwFrameInterval( 0) 2000000
VideoStreaming Interface Descriptor:
bLength 34
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 2
bmCapabilities 0x00
Still image unsupported
wWidth 1280
wHeight 720
dwMinBitRate 73728000
dwMaxBitRate 147456000
dwMaxVideoFrameBufferSize 1843200
dwDefaultFrameInterval 1000000
bFrameIntervalType 2
dwFrameInterval( 0) 1000000
dwFrameInterval( 1) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 3
bmCapabilities 0x00
Still image unsupported
wWidth 640
wHeight 480
dwMinBitRate 24576000
dwMaxBitRate 147456000
dwMaxVideoFrameBufferSize 614400
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 4
bmCapabilities 0x00
Still image unsupported
wWidth 640
wHeight 360
dwMinBitRate 18432000
dwMaxBitRate 110592000
dwMaxVideoFrameBufferSize 460800
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 5
bmCapabilities 0x00
Still image unsupported
wWidth 320
wHeight 240
dwMinBitRate 6144000
dwMaxBitRate 36864000
dwMaxVideoFrameBufferSize 153600
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 6
bmCapabilities 0x00
Still image unsupported
wWidth 320
wHeight 180
dwMinBitRate 4608000
dwMaxBitRate 27648000
dwMaxVideoFrameBufferSize 115200
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 30
bDescriptorType 36
bDescriptorSubtype 5 (FRAME_UNCOMPRESSED)
bFrameIndex 7
bmCapabilities 0x00
Still image unsupported
wWidth 1920
wHeight 1080
dwMinBitRate 165888000
dwMaxBitRate 165888000
dwMaxVideoFrameBufferSize 4147200
dwDefaultFrameInterval 2000000
bFrameIntervalType 1
dwFrameInterval( 0) 2000000
VideoStreaming Interface Descriptor:
bLength 6
bDescriptorType 36
bDescriptorSubtype 13 (COLORFORMAT)
bColorPrimaries 1 (BT.709,sRGB)
bTransferCharacteristics 1 (BT.709)
bMatrixCoefficients 4 (SMPTE 170M (BT.601))
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 1
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0080 1x 128 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 2
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0100 1x 256 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 3
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0320 1x 800 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 4
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0b20 2x 800 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 5
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x1320 3x 800 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 6
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x1400 3x 1024 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 0
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
VideoStreaming Interface Descriptor:
bLength 14
bDescriptorType 36
bDescriptorSubtype 1 (INPUT_HEADER)
bNumFormats 1
wTotalLength 0x018e
bEndpointAddress 0x82 EP 2 IN
bmInfo 0
bTerminalLink 6
bStillCaptureMethod 0
bTriggerSupport 0
bTriggerUsage 0
bControlSize 1
bmaControls( 0) 0
VideoStreaming Interface Descriptor:
bLength 28
bDescriptorType 36
bDescriptorSubtype 16 (FORMAT_FRAME_BASED)
bFormatIndex 1
bNumFrameDescriptors 7
guidFormat {34363248-0000-0010-8000-00aa00389b71}
bBitsPerPixel 16
bDefaultFrameIndex 1
bAspectRatioX 0
bAspectRatioY 0
bmInterlaceFlags 0x00
Interlaced stream or variable: No
Fields per frame: 2 fields
Field 1 first: No
Field pattern: Field 1 only
bCopyProtect 0
bVariableSize 1
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 1
bmCapabilities 0x00
Still image unsupported
wWidth 1920
wHeight 1080
dwMinBitRate 27648000
dwMaxBitRate 165888000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 2
bmCapabilities 0x00
Still image unsupported
wWidth 1280
wHeight 720
dwMinBitRate 24576000
dwMaxBitRate 147456000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 3
bmCapabilities 0x00
Still image unsupported
wWidth 640
wHeight 480
dwMinBitRate 8192000
dwMaxBitRate 49152000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 4
bmCapabilities 0x00
Still image unsupported
wWidth 640
wHeight 360
dwMinBitRate 6144000
dwMaxBitRate 36864000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 5
bmCapabilities 0x00
Still image unsupported
wWidth 320
wHeight 240
dwMinBitRate 2048000
dwMaxBitRate 12288000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 6
bmCapabilities 0x00
Still image unsupported
wWidth 320
wHeight 180
dwMinBitRate 1536000
dwMaxBitRate 9216000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 50
bDescriptorType 36
bDescriptorSubtype 17 (FRAME_FRAME_BASED)
bFrameIndex 7
bmCapabilities 0x00
Still image unsupported
wWidth 1920
wHeight 1080
dwMinBitRate 27648000
dwMaxBitRate 165888000
dwDefaultFrameInterval 333333
bFrameIntervalType 6
dwBytesPerLine 0
dwFrameInterval( 0) 333333
dwFrameInterval( 1) 400000
dwFrameInterval( 2) 500000
dwFrameInterval( 3) 666666
dwFrameInterval( 4) 1000000
dwFrameInterval( 5) 2000000
VideoStreaming Interface Descriptor:
bLength 6
bDescriptorType 36
bDescriptorSubtype 13 (COLORFORMAT)
bColorPrimaries 1 (BT.709,sRGB)
bTransferCharacteristics 1 (BT.709)
bMatrixCoefficients 4 (SMPTE 170M (BT.601))
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 1
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0080 1x 128 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 2
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0100 1x 256 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 3
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0320 1x 800 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 4
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x0b20 2x 800 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 5
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x1320 3x 800 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 6
bNumEndpoints 1
bInterfaceClass 14 Video
bInterfaceSubClass 2 Video Streaming
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 5
Transfer Type Isochronous
Synch Type Asynchronous
Usage Type Data
wMaxPacketSize 0x1400 3x 1024 bytes
bInterval 1
We can see the effect of this patch in GStreamer:
To reproduce this effect we need to launch two streams simultaneously
like so:
gst-launch-1.0 -e v4l2src device=$H264_DEVICE ! video/x-h264,width=1920,height=1080,framerate=30/1 ! fakesink &
gst-launch-1.0 -e v4l2src device=$MJPEG_DEVICE ! image/jpeg,width=1920,height=1080,framerate=30/1 ! jpegdec ! videoconvert ! x264enc tune=zerolatency ! mp4mux ! filesink location=output2.mp4 &
Post-patch:
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1861 kb/s, 29.10 fps, 29.58 tbr, 3k tbn (default)
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1880 kb/s, 29.36 fps, 29.58 tbr, 3k tbn (default)
Pre-patch:
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1819 kb/s, 29.09 fps, 29.58 tbr, 3k tbn (default)
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1765 kb/s, 27.83 fps, 29.58 tbr, 3k tbn (default)
We can see from the results that the second stream (the one with 1 H.264
stream and 1 MJPEG stream, gets lower FPS before the patch, indicating
the lost frames). After the patch we get higher FPS (29.36 as opposed to
27.83).
Isaac Scott (2):
media: uvcvideo: Implement dual stream quirk to fix loss of usb
packets
media: uvcvideo: Add new quirk definition for the Sonix Technology Co.
292a camera
drivers/media/usb/uvc/uvc_driver.c | 9 +++++++++
drivers/media/usb/uvc/uvc_video.c | 27 ++++++++++++++++++++++++++-
drivers/media/usb/uvc/uvcvideo.h | 1 +
3 files changed, 36 insertions(+), 1 deletion(-)
--
2.43.0
