Hello List.
I am having trouble solving a problem with a bunch of USB3-SATA-bridge
connected to 2 USB3-hubs. I was not able finding any useful help for
weeks now.
Perhaps by bringing this issue to linux-usb can help me troubleshot the
problem or sort out a bug or help improve Linux USB stability.
The issue is an ongoing reset of different JMicron (ID 152d:0567) JMS567
USB3-SATA-bridges partitioned equally to 2 USB3-hubs (ID 2109:0812).
I have 8 of theses bridges with 8 3TB WD (WDC WD30EZRX) hard drives
connected to them. That 8 drives are configured running as a raid6.
Reading/writing using 1 hard disk at a time works fine, even with higher
speeds using dd or copying large files. Using several drives
simultaneously is causing ongoing resets on a number of these bridges
randomly (but 1 a time). It is like reading for seconds, then the read
blocks for may by 5 seconds, a reset on one bridge happens and the
reading continues.
The host is a Debian 8 with Linux 3.16.0-4-amd64.
What i noticed today was, by lowering max_sectors to 16 or 8 improves
stability much, but it is still fare from stable.
What could be done improving this?
Are there any quirks worth trying?
Any idea would be much appreciated. Axel
~# dmesg
[2134752.916530] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d55701c0
[2134752.916543] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570208
[2134786.815641] usb 3-3.1.3: reset SuperSpeed USB device number 9 using
xhci_hcd
[2134786.830932] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d55701c0
[2134786.830945] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570208
[2134820.858285] usb 3-3.1.3: reset SuperSpeed USB device number 9 using
xhci_hcd
[2134820.873365] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d55701c0
[2134820.873378] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570208
[2134825.450020] usb 3-3.1.2: reset SuperSpeed USB device number 8 using
xhci_hcd
[2134825.464474] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570340
[2134825.464488] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570388
[2134856.820071] usb 3-3.1.3: reset SuperSpeed USB device number 9 using
xhci_hcd
[2134856.835270] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d55701c0
[2134856.835283] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570208
[2134891.886229] usb 3-3.1.3: reset SuperSpeed USB device number 9 using
xhci_hcd
[2134891.901496] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d55701c0
[2134891.901509] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570208
[2134926.824496] usb 3-3.1.3: reset SuperSpeed USB device number 9 using
xhci_hcd
[2134926.839680] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d55701c0
[2134926.839693] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called
with disabled ep ffff8800d5570208
~# lsblk
sdc 8:32 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sdd 8:48 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sde 8:64 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sdf 8:80 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sdg 8:96 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sdh 8:112 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sdi 8:128 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
sdj 8:144 0 2,7T 0 disk
└─md0 9:0 0 16,4T 0 raid6
└─stor 254:8 0 16,4T 0 crypt /mnt/stor
~# lsusb
Bus 001 Device 002: ID 8087:8000 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 007: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 013: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 012: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 011: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 006: ID 2109:0812
Bus 003 Device 003: ID 2109:0812
Bus 003 Device 005: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 010: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 009: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 008: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Bus 003 Device 004: ID 2109:0812
Bus 003 Device 002: ID 2109:0812
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 005: ID 13d3:3394 IMC Networks
Bus 002 Device 007: ID 2109:2812
Bus 002 Device 004: ID 2109:2812
Bus 002 Device 006: ID 2109:2812
Bus 002 Device 003: ID 2109:2812
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
~# lsusb -t
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
|__ Port 3: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 1: Dev 4, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 2: Dev 8, If 0, Class=Mass Storage,
Driver=usb-storage, 5000M
|__ Port 3: Dev 9, If 0, Class=Mass Storage,
Driver=usb-storage, 5000M
|__ Port 4: Dev 10, If 0, Class=Mass Storage,
Driver=usb-storage, 5000M
|__ Port 3: Dev 5, If 0, Class=Mass Storage, Driver=usb-storage,
5000M
|__ Port 4: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 1: Dev 6, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 2: Dev 11, If 0, Class=Mass Storage,
Driver=usb-storage, 5000M
|__ Port 3: Dev 12, If 0, Class=Mass Storage,
Driver=usb-storage, 5000M
|__ Port 4: Dev 13, If 0, Class=Mass Storage,
Driver=usb-storage, 5000M
|__ Port 3: Dev 7, If 0, Class=Mass Storage, Driver=usb-storage,
5000M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 480M
|__ Port 3: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 6, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 4: Dev 4, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 7, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 7: Dev 5, If 0, Class=Wireless, Driver=btusb, 12M
|__ Port 7: Dev 5, If 1, Class=Wireless, Driver=btusb, 12M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
:~# lsusb -v
Bus 003 Device 007: ID 152d:0567 JMicron Technology Corp. / JMicron USA
Technology Corp.
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 9
idVendor 0x152d JMicron Technology Corp. / JMicron USA
Technology Corp.
idProduct 0x0567
bcdDevice 1.07
iManufacturer 10 JMicron
iProduct 11 USB to ATA/ATAPI Bridge
iSerial 5 WD-WMC4N0E7KS8E
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 44
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xc0
Self Powered
MaxPower 2mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 22
bNumDeviceCaps 2
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000e
Device can operate at Full Speed (12Mbps)
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 1
Lowest fully-functional device speed is Full Speed (12Mbps)
bU1DevExitLat 10 micro seconds
bU2DevExitLat 32 micro seconds
Device Status: 0x0001
Self Powered
00:14.0 USB controller: Intel Corporation 8 Series USB xHCI HC (rev 04)
(prog-if 30 [XHCI])
Subsystem: Gigabyte Technology Co., Ltd Device 5004
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 60
Region 0: Memory at f7e00000 (64-bit, non-prefetchable)
[size=64K]
Capabilities: [70] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
Address: 00000000fee0300c Data: 41c1
Kernel driver in use: xhci_hcd
00:16.0 Communication controller: Intel Corporation 8 Series HECI #0
(rev 04)
Subsystem: Gigabyte Technology Co., Ltd Device 5001
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Interrupt: pin A routed to IRQ 62
Region 0: Memory at f7e1d000 (64-bit, non-prefetchable)
[size=32]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee0300c Data: 41e1
Kernel driver in use: mei_me
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
