On Tue, Jun 14, 2011 at 10:17:05PM +0200, Francesco Castelli wrote: > Hi Sarah, > > many thanks for your answer, not sure about that, > but it seems that you have aimed the spotlight in the correct direction: > > setting CONFIG_PCI_DEBUG=y I found out that the crash > is due to an ARM processor fault, > namely, a data abort due to a PCI access failure: > this happens when, in the very first lines of xhci_event_ring_work(), > your code references the status register > within the XHCI-HCD operational registers set > > therefore, we definitely have no wandering pointers: > most likely, the PCIe transaction for accessing that register > went bad, for some reason, either hardware- or driver-related: > the ti816x_pcie.c driver module is maintained directly from TI, > so I have to refer to TI folks directly... I see. Would that be Felipe's PCI glue layer for the OMAP5 processor? Or is this a different TI USB3 evaluation board? > ...unless, of course, you have some more hints for me > about debugging the PCIe side of an USB3 hub :-) No, I can't help you debug the PCIe layer. :) I'd say ask TI, and then maybe ask on the linux-pci mailing list. Sarah Sharp > Il 14/06/2011 02:04, Sarah Sharp ha scritto: > >On Fri, Jun 10, 2011 at 02:08:27PM +0200, Francesco Castelli wrote: > >>Hi Sarah, > >> > >>let me explain better this time: > >> > >>-first of all, I work as an Embedded Linux consultant > >>on a prototype board with a Texas Instruments TI8168 device > >>(brief data sheet here: http://focus.ti.com/lit/ds/sprs614/sprs614.pdf) > >> > >>-the PCIe controller embedded into TI8168 is connected > >>to the (TI again) TUSB7320 2-ports xHCI-compliant USB3 controller > >>(data manual here: http://focus.ti.com/lit/ds/symlink/tusb7320.pdf) > >> > >>-TI delivers a complete TI81XX SDK covering all the (many) onboard cores: > >>the ARM Cortex-A8 is supported via patching Linux Kernel 2.6.37 > >>basicly with the TI8168 chip support and the TI EVM board support, > >> > >>PLEASE NOTE that the drivers/usb subtree has NOT been patched in any way > >Ok, good to know. > > > >>-I ran 'make menuconfig' after selecting my boards' defconfig, > >>adding the following lines: > >> > >>CONFIG_USB_DEBUG=y > >>CONFIG_USB_XHCI_HCD=m > >>CONFIG_USB_XHCI_HCD_DEBUGGING=y > >> > >>-my runtime routine is to prevent autoloading xhci-hcd.ko via blacklisting > >>and load it later via modprobe, I don't know if this helps, anyway > >>now, among the various issues I raised in the previous email, > >>I would like to focus on what I think is the major problem: the random crash > >>that happens with NO (!!!) USB peripheral plugged in > >>(BTW 'gadget' is the misleading term I used in place of 'peripheral') > >> > >>please find attached a single TXT file with three sections of dmesg dumps: > >> > >>-the first one refers to when I load xhci-hcd.ko via insmod > >Ok, that looks normal since you enabled CONFIG_USB_XHCI_HCD_DEBUGGING. > > > >>-the second one appears randomly and looks like an activity report > >>from the hub; > >Yes, that's the polling loop that runs every minute. I probably should > >change it to check the enqueue and dequeue pointers from the previous > >run, and not print anything if nothing changed, but it's normal. > > > >>-the last one is the crash itself, an illegal access at location 0x00001008 > >> > >>which is not addressable by the ARM processor > >Yeah, that's probably a NULL pointer dereference. > > > >>#################################################################### > >># fault that happened without NO USB peripheral plugged in - BEGIN # > >>#################################################################### > >>xhci_hcd 0000:01:00.0: Poll event ring: 14496 > >>Unhandled fault: Precise External Abort on non-linefetch (0x1008) at 0x8f140024 > >>Internal error: : 1008 [#1] > >>last sysfs file: /sys/devices/pci0000:00/0000:00:00.0/0000:01:00.0/usb3/idVendor > >>Modules linked in: aett ti8168_vpssm3 ti8168_videom3 ti8168_c674xplus TI81xx_hdmi syslink xhci_hcd ipv6 aeremote rvphoenix_board xNCI hal_xNCI pSOSk gmem > >>CPU: 0 Not tainted (2.6.37 #1) > >>PC is at xhci_event_ring_work+0x34/0x240 [xhci_hcd] > >>LR is at release_console_sem+0x198/0x1ac > >>pc : [<7f0825cc>] lr : [<8006516c>] psr: 20000193 > >>sp : 80477e70 ip : 80477d58 fp : 80477e94 > >>r10: 804d934c r9 : 804d914c r8 : 20000113 > >>r7 : 00000100 r6 : 804d8540 r5 : 8ac6c0dc r4 : 000000a0 > >>r3 : 8f140020 r2 : 00000001 r1 : 0000fa58 r0 : 00000034 > >>Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel > >>Control: 10c5387d Table: 8acbc019 DAC: 00000017 > >>Process swapper (pid: 0, stack limit = 0x804762e8) > >>Stack: (0x80477e70 to 0x80478000) > >>7e60: 000000a0 8ac6c564 804d8540 00000100 > >>7e80: 80476000 804d914c 80477ee4 80477e98 80071260 7f0825a4 8008a518 8008a358 > >>7ea0: 8ac6c0dc 7f082598 804d8f4c 804d8d4c 80477eb0 80477eb0 00989680 00000043 > >>7ec0: 80476000 00000001 00000004 00000001 0000000a 00000100 80477f24 80477ee8 > >>7ee0: 8006a42c 80071088 80477f1c 80477ef8 80477f0c 804d8304 8004f574 00000043 > >>7f00: 00000000 8002d8ac 8047a0f4 8002bfa8 413fc082 0000001f 80477f34 80477f28 > >>7f20: 8006a55c 8006a354 80477f4c 80477f38 80037080 8006a520 ffffffff fa200000 > >>7f40: 80477fa4 80477f50 80361d74 8003700c 81600181 40000013 80477f98 00000814 > >>7f60: 80476000 804b7380 8002d8ac 8047a0f4 8002bfa8 413fc082 0000001f 80477fa4 > >>7f80: 80477f98 80477f98 80042f5c 80042f60 80000013 ffffffff 80477fbc 80477fa8 > >>7fa0: 80043500 80042f30 804ff83c 804b7380 80477fcc 80477fc0 80357474 800434bc > >>7fc0: 80477ff4 80477fd0 80008c64 80357420 800087c4 00000000 00000000 8002d8b0 > >>7fe0: 10c53c7d 804b749c 00000000 80477ff8 80008034 80008a20 00000000 00000000 > >>Backtrace: > >>[<7f082598>] (xhci_event_ring_work+0x0/0x240 [xhci_hcd]) from [<80071260>] (run_timer_softirq+0x1e4/0x2bc) > >> r9:804d914c r8:80476000 r7:00000100 r6:804d8540 r5:8ac6c564 > >>r4:000000a0 > >>[<8007107c>] (run_timer_softirq+0x0/0x2bc) from [<8006a42c>] (__do_softirq+0xe4/0x1cc) > >>[<8006a348>] (__do_softirq+0x0/0x1cc) from [<8006a55c>] (irq_exit+0x48/0x94) > >>[<8006a514>] (irq_exit+0x0/0x94) from [<80037080>] (asm_do_IRQ+0x80/0xa0) > >>[<80037000>] (asm_do_IRQ+0x0/0xa0) from [<80361d74>] (__irq_svc+0x34/0xa0) > >>Exception stack(0x80477f50 to 0x80477f98) > >>7f40: 81600181 40000013 80477f98 00000814 > >>7f60: 80476000 804b7380 8002d8ac 8047a0f4 8002bfa8 413fc082 0000001f 80477fa4 > >>7f80: 80477f98 80477f98 80042f5c 80042f60 80000013 ffffffff > >> r5:fa200000 r4:ffffffff > >>[<80042f24>] (default_idle+0x0/0x40) from [<80043500>] (cpu_idle+0x50/0x90) > >>[<800434b0>] (cpu_idle+0x0/0x90) from [<80357474>] (rest_init+0x60/0x78) > >> r5:804b7380 r4:804ff83c > >>[<80357414>] (rest_init+0x0/0x78) from [<80008c64>] (start_kernel+0x250/0x2a4) > >>[<80008a14>] (start_kernel+0x0/0x2a4) from [<80008034>] (stext+0x34/0x3c) > >> r5:804b749c r4:10c53c7d > >>Code: eb45c29a e10f8000 f10c0080 e5953004 (e5934004) > >>---[ end trace b32101bf3e82059a ]--- > >>Kernel panic - not syncing: Fatal exception in interrupt > >>Backtrace: > >>[<80045ce4>] (dump_backtrace+0x0/0x10c) from [<8035fbc0>] (dump_stack+0x18/0x1c) > >> r7:7f0825d0 r6:80477d1f r5:7f0825ce r4:804b79e0 > >>[<8035fba8>] (dump_stack+0x0/0x1c) from [<8035fc24>] (panic+0x60/0x17c) > >>[<8035fbc4>] (panic+0x0/0x17c) from [<80046074>] (die+0x284/0x2d8) > >> r3:00000100 r2:8041ea52 r1:00000000 r0:8040dbc6 > >>[<80045df0>] (die+0x0/0x2d8) from [<80046188>] (arm_notify_die+0x5c/0x60) > >>[<8004612c>] (arm_notify_die+0x0/0x60) from [<800372f0>] (do_DataAbort+0x88/0x9c) > >> r5:8047a7c0 r4:00000007 > >>[<80037268>] (do_DataAbort+0x0/0x9c) from [<80361d2c>] (__dabt_svc+0x4c/0x60) > >>Exception stack(0x80477e28 to 0x80477e70) > >>7e20: 00000034 0000fa58 00000001 8f140020 000000a0 8ac6c0dc > >>7e40: 804d8540 00000100 20000113 804d914c 804d934c 80477e94 80477d58 80477e70 > >>7e60: 8006516c 7f0825cc 20000193 ffffffff > >> r8:20000113 r7:00000100 r6:804d8540 r5:80477e5c r4:ffffffff > >>[<7f082598>] (xhci_event_ring_work+0x0/0x240 [xhci_hcd]) from [<80071260>] (run_timer_softirq+0x1e4/0x2bc) > >> r9:804d914c r8:80476000 r7:00000100 r6:804d8540 r5:8ac6c564 > >>r4:000000a0 > >>[<8007107c>] (run_timer_softirq+0x0/0x2bc) from [<8006a42c>] (__do_softirq+0xe4/0x1cc) > >>[<8006a348>] (__do_softirq+0x0/0x1cc) from [<8006a55c>] (irq_exit+0x48/0x94) > >>[<8006a514>] (irq_exit+0x0/0x94) from [<80037080>] (asm_do_IRQ+0x80/0xa0) > >>[<80037000>] (asm_do_IRQ+0x0/0xa0) from [<80361d74>] (__irq_svc+0x34/0xa0) > >>Exception stack(0x80477f50 to 0x80477f98) > >>7f40: 81600181 40000013 80477f98 00000814 > >>7f60: 80476000 804b7380 8002d8ac 8047a0f4 8002bfa8 413fc082 0000001f 80477fa4 > >>7f80: 80477f98 80477f98 80042f5c 80042f60 80000013 ffffffff > >> r5:fa200000 r4:ffffffff > >>[<80042f24>] (default_idle+0x0/0x40) from [<80043500>] (cpu_idle+0x50/0x90) > >>[<800434b0>] (cpu_idle+0x0/0x90) from [<80357474>] (rest_init+0x60/0x78) > >> r5:804b7380 r4:804ff83c > >>[<80357414>] (rest_init+0x0/0x78) from [<80008c64>] (start_kernel+0x250/0x2a4) > >>[<80008a14>] (start_kernel+0x0/0x2a4) from [<80008034>] (stext+0x34/0x3c) > >> r5:804b749c r4:10c53c7d > >>#################################################################### > >># fault that happened without NO USB peripheral plugged in - END # > >>#################################################################### > >That is very strange looking. Have you tried to run the oops through > >scripts/markup-oops.pl to see where exactly the xHCI driver is oopsing? > >If nothing changed in the driver state, why would this function > >dereference a NULL pointer on some random iteration through > >the polling loop in xhci_event_ring_work? I'm baffled. > > > >Did the xHCI host controller disconnect from the PCIe bus some time > >before, or was there any abnormal activity before this crash? > > > >Sarah Sharp > > -- To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
