[5.15-rc1][PPC/PCI] IPR adapter reset threw EEH and fail to recover while booting mainline on powerpc

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Greeting's

IPR adapter reset triggers EEH errors and recovered 6 times and than task hung on today's mainline kernel on powerpc box with loads of hung task call traces

The problem is not seen with 5.14-rc7 (good)  and is last recreated on 5.15-rc1 (bad) kernel

Console logs
EEH: Recovering PHB#134-PE#800000
EEH: PE location: N/A, PHB location: N/A
EEH: Frozen PHB#134-PE#800000 detected
EEH: Call Trace:
EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
EEH: [c008000000501c08] eeh_readl+0x70/0xa0 [ipr]
EEH: [c00800000050e76c] ipr_reset_enable_ioa+0x474/0x5a0 [ipr]
EEH: [c0080000005049a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
EEH: [c008000000508f04] ipr_pci_slot_reset+0xac/0x140 [ipr]
EEH: [c000000000052ce8] eeh_report_reset+0xb8/0xe0
EEH: [c000000000054834] eeh_pe_report+0x254/0x4b4
EEH: [c000000000053efc] eeh_handle_normal_event+0x9dc/0xa80
EEH: [c000000000054de8] eeh_event_handler+0x108/0x190
EEH: [c00000000018151c] kthread+0x18c/0x1a0
EEH: [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
EEH: PHB#134-PE#800000 has failed 6 times in the last hour and has been permanently disabled.
EEH: Unable to recover from failure from PHB#134-PE#800000.
Please try reseating or replacing it
EEH: of node=0134:80:00.0
EEH: PCI device/vendor: 034a1014
EEH: PCI cmd/status register: 00100546
EEH: PCI-E capabilities and status follow:
EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503
EEH: PCI-E 10: 10830000 00000000 00000000 00000000
EEH: PCI-E 20: 00000000
EEH: PCI-E AER capability register set follows:
EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030
EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000
EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000
EEH: PCI-E AER 30: 00000000 00000000
EEH: Beginning: 'error_detected(permanent failure)'
PCI 0134:80:00.0#800000: EEH: not actionable (1,1,1)
[FAILED] Failed to start udev Wait for Complete Device Initialization.
See 'systemctl status systemd-udev-settle.service' for details.
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
EEH: Finished:'error_detected(permanent failure)'
device-mapper: multipath service-time: version 0.3.0 loaded
INFO: task eehd:137 blocked for more than 122 seconds.
      Not tainted 5.15.0-rc1-autotest #3
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:eehd            state:D stack:    0 pid:  137 ppid:     2 flags:0x00000800
Call Trace:
[c00000000d167650] [c0000000001c329c] __wake_up_common+0xcc/0x290 (unreliable)
[c00000000d167840] [c00000000001e6c8] __switch_to+0x278/0x490
[c00000000d1678a0] [c000000000c8b9bc] __schedule+0x31c/0xa20
[c00000000d167960] [c000000000c8c128] schedule+0x68/0x120
[c00000000d167990] [c00800000050a3c4] __ipr_remove+0x10c/0x4b0 [ipr]
[c00000000d167a60] [c00800000050a828] ipr_remove+0xc0/0x110 [ipr]
[c00000000d167ae0] [c000000000732bd4] pci_device_remove+0x64/0x120
[c00000000d167b20] [c000000000811318] device_release_driver_internal+0x168/0x2c0
[c00000000d167b60] [c000000000724618] pci_stop_bus_device+0xa8/0x100
[c00000000d167ba0] [c000000000724810] pci_stop_and_remove_bus_device+0x20/0x40
[c00000000d167bd0] [c000000000075e60] pci_hp_remove_devices+0x90/0x130
[c00000000d167c60] [c000000000053cd8] eeh_handle_normal_event+0x7b8/0xa80
[c00000000d167d40] [c000000000054de8] eeh_event_handler+0x108/0x190
[c00000000d167da0] [c00000000018151c] kthread+0x18c/0x1a0
[c00000000d167e10] [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
INFO: task systemd-udevd:473 blocked for more than 122 seconds.
      Not tainted 5.15.0-rc1-autotest #3

--
Regard's

Abdul Haleem
IBM Linux Technology Center


OF stdout device is: /vdevice/vty@30000000
Preparing to boot Linux version 5.15.0-rc1-autotest (root@ltcd11) (gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1), GNU ld version 2.30-93.el8) #5 SMP Mon Sep 13 10:16:56 EDT 2021
Detected machine type: 0000000000000101
command line: BOOT_IMAGE=/vmlinuz-5.15.0-rc1-autotest root=/dev/mapper/rhel_ltcd11-root ro crashkernel=auto rd.lvm.lv=rhel_ltcden11/root rd.lvm.lv=rhel_ltcden11-lp3/swap biosdevname=0 ipr.debug=1
Max number of cores passed to firmware: 256 (NR_CPUS = 2048)
Calling ibm,client-architecture-support... done
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 0000000010560000
  alloc_top    : 0000000020000000
  alloc_top_hi : 0000000020000000
  rmo_top      : 0000000020000000
  ram_top      : 0000000020000000
instantiating rtas at 0x000000001ec30000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x0000000010570000 -> 0x0000000010571aaa
Device tree struct  0x0000000010580000 -> 0x0000000010590000
Quiescing Open Firmware ...
Booting Linux via __start() @ 0x000000000a710000 ...
[    0.000000] crashkernel: memory value expected
[    0.000000] radix-mmu: Page sizes from device-tree:
[    0.000000] radix-mmu: Page size shift = 12 AP=0x0
[    0.000000] radix-mmu: Page size shift = 16 AP=0x5
[    0.000000] radix-mmu: Page size shift = 21 AP=0x1
[    0.000000] radix-mmu: Page size shift = 30 AP=0x2
[    0.000000] Activating Kernel Userspace Execution Prevention
[    0.000000] Activating Kernel Userspace Access Prevention
[    0.000000] radix-mmu: Mapped 0x0000000000000000-0x0000000001600000 with 2.00 MiB pages (exec)
[    0.000000] radix-mmu: Mapped 0x0000000001600000-0x0000000c80000000 with 2.00 MiB pages
[    0.000000] lpar: Using radix MMU under hypervisor
[    0.000000] Linux version 5.15.0-rc1-autotest (root@ltcd11) (gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1), GNU ld version 2.30-93.el8) #5 SMP Mon Sep 13 10:16:56 EDT 2021
[    0.000000] Found initrd at 0xc00000000c500000:0xc000000010558bc4
[    0.000000] Using pSeries machine description
[    0.000000] printk: bootconsole [udbg0] enabled
[    0.000000] Partition configured for 24 cpus.
[    0.000000] CPU maps initialized for 8 threads per core
[    0.000000] numa: Partition configured for 32 NUMA nodes.
[    0.000000] -----------------------------------------------------
[    0.000000] phys_mem_size     = 0xc80000000
[    0.000000] dcache_bsize      = 0x80
[    0.000000] icache_bsize      = 0x80
[    0.000000] cpu_features      = 0x000c00eb8f5f9187
[    0.000000]   possible        = 0x000ffbfbcf5fb187
[    0.000000]   always          = 0x0000000380008181
[    0.000000] cpu_user_features = 0xdc0065c2 0xaef60000
[    0.000000] mmu_features      = 0x3c007641
[    0.000000] firmware_features = 0x0000019fc45bfc57
[    0.000000] vmalloc start     = 0xc008000000000000
[    0.000000] IO start          = 0xc00a000000000000
[    0.000000] vmemmap start     = 0xc00c000000000000
[    0.000000] -----------------------------------------------------
[    0.000000] numa:   NODE_DATA [mem 0xc7fafb380-0xc7fb003ff]
[    0.000000] rfi-flush: fallback displacement flush available
[    0.000000] count-cache-flush: hardware flush enabled.
[    0.000000] link-stack-flush: software flush enabled.
[    0.000000] stf-barrier: eieio barrier available
[    0.000000] lpar: H_BLOCK_REMOVE supports base psize:0 psize:0 block size:8
[    0.000000] PPC64 nvram contains 15360 bytes
[    0.000000] barrier-nospec: using ORI speculation barrier
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x0000000c7fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000c7fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000c7fffffff]
[    0.000000] percpu: Embedded 10 pages/cpu s590360 r0 d65000 u655360
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 818400
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-5.15.0-rc1-autotest root=/dev/mapper/rhel_ltcden11--lp3-root ro crashkernel=auto rd.lvm.lv=rhel_ltcden11-lp3/root rd.lvm.lv=rhel_ltcden11-lp3/swap biosdevname=0 ipr.debug=1
[    0.000000] Unknown command line parameters: BOOT_IMAGE=/vmlinuz-5.15.0-rc1-autotest crashkernel=auto biosdevname=0
[    0.000000] Dentry cache hash table entries: 8388608 (order: 10, 67108864 bytes, linear)
[    0.000000] Inode-cache hash table entries: 4194304 (order: 9, 33554432 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 52058944K/52428800K available (12928K kernel code, 5760K rwdata, 4096K rodata, 3904K init, 2625K bss, 369856K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=24, Nodes=32
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=2048 to nr_cpu_ids=24.
[    0.000000] rcu: 	RCU debug extended QS entry/exit.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] 	Rude variant of Tasks RCU enabled.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=24
[    0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16
[    0.000000] xive: Using IRQ range [400000-400017]
[    0.000000] xive: Interrupt handling initialized with spapr backend
[    0.000000] xive: Using priority 7 for all interrupts
[    0.000000] xive: Using 64kB queues
[    0.000000] rcu: 	Offload RCU callbacks from CPUs: (none).
[    0.000000] random: get_random_u64 called from start_kernel+0x65c/0x8bc with crng_init=0
[    0.000001] time_init: 56 bit decrementer (max: 7fffffffffffff)
[    0.000025] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x761537d007, max_idle_ns: 440795202126 ns
[    0.000065] clocksource: timebase mult[1f40000] shift[24] registered
[    0.000126] Console: colour dummy device 80x25
[    0.000146] printk: console [hvc0] enabled
[    0.000146] printk: console [hvc0] enabled
[    0.000164] printk: bootconsole [udbg0] disabled
[    0.000164] printk: bootconsole [udbg0] disabled
[    0.000202] pid_max: default: 32768 minimum: 301
[    0.000228] LSM: Security Framework initializing
[    0.000238] Yama: becoming mindful.
[    0.000245] SELinux:  Initializing.
[    0.000332] Mount-cache hash table entries: 131072 (order: 4, 1048576 bytes, linear)
[    0.000397] Mountpoint-cache hash table entries: 131072 (order: 4, 1048576 bytes, linear)
[    0.000993] POWER10 performance monitor hardware support registered
[    0.001012] rcu: Hierarchical SRCU implementation.
[    0.010106] Kprobe smoke test: started
[    0.017789] Kprobe smoke test: passed successfully
[    0.017976] smp: Bringing up secondary CPUs ...
[    0.022951] smp: Brought up 1 node, 24 CPUs
[    0.022959] numa: Node 0 CPUs: 0-23
[    0.022963] Big cores detected but using small core scheduling
[    0.023618] devtmpfs: initialized
[    0.025671] PCI host bridge /pci@800000020000010  ranges:
[    0.025681]  MEM 0x0000040000000000..0x000004007effffff -> 0x0000000080000000 
[    0.025686]  MEM 0x0000044000000000..0x0000047fffffffff -> 0x0006020000000000 
[    0.025704] PCI host bridge /pci@800000020000134  ranges:
[    0.025709]  MEM 0x0000040080000000..0x000004008fffffff -> 0x00000000e0000000 
[    0.025714]  MEM 0x0000041000000000..0x0000041fffffffff -> 0x0006202000000000 
[    0.025751] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.025759] futex hash table entries: 8192 (order: 4, 1048576 bytes, linear)
[    0.025968] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.026113] audit: initializing netlink subsys (disabled)
[    0.026159] audit: type=2000 audit(1631801812.020:1): state=initialized audit_enabled=0 res=1
[    0.026203] thermal_sys: Registered thermal governor 'fair_share'
[    0.026204] thermal_sys: Registered thermal governor 'step_wise'
[    0.026282] cpuidle: using governor menu
[    0.026448] pstore: Registered nvram as persistent store backend
[    0.026873] EEH: pSeries platform initialized
[    0.029200] software IO TLB: tearing down default memory pool
[    0.029262] PCI: Probing PCI hardware
[    0.029321] PCI host bridge to bus 0010:01
[    0.029326] pci_bus 0010:01: root bus resource [mem 0x40000000000-0x4007effffff] (bus address [0x80000000-0xfeffffff])
[    0.029332] pci_bus 0010:01: root bus resource [mem 0x44000000000-0x47fffffffff 64bit] (bus address [0x6020000000000-0x6023fffffffff])
[    0.029339] pci_bus 0010:01: root bus resource [bus 01-ff]
[    0.029855] pci 0010:01:00.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled.
[    0.031328] pci 0010:01:00.0: PME# supported from D0 D3hot D3cold
[    0.032699] pci 0010:01:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 0010:01:00.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
[    0.037434] IOMMU table initialized, virtual merging enabled
[    0.037490] PCI host bridge to bus 0134:80
[    0.037493] pci_bus 0134:80: Unknown NUMA node; performance will be reduced
[    0.037497] pci_bus 0134:80: root bus resource [mem 0x40080000000-0x4008fffffff] (bus address [0xe0000000-0xefffffff])
[    0.037503] pci_bus 0134:80: root bus resource [mem 0x41000000000-0x41fffffffff 64bit] (bus address [0x6202000000000-0x6202fffffffff])
[    0.037508] pci_bus 0134:80: root bus resource [bus 80-ff]
[    0.038166] pci 0134:80:00.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled.
[    0.040026] pci 0134:80:00.0: PME# supported from D0 D3hot D3cold
[    0.041666] pci 0134:80:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 0134:80:00.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
[    0.046896] pci 0010:01:00.0: Adding to iommu group 0
[    0.048009] pci 0134:80:00.0: Adding to iommu group 1
[    0.049538] EEH: Capable adapter found: recovery enabled.
[    0.049607] pseries-rng: Registering arch random hook.
[    0.050488] Kprobes globally optimized
[    0.050610] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.050615] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.079264] iommu: Default domain type: Translated 
[    0.079269] iommu: DMA domain TLB invalidation policy: strict mode 
[    0.079308] vgaarb: loaded
[    0.079398] SCSI subsystem initialized
[    0.079425] usbcore: registered new interface driver usbfs
[    0.079433] usbcore: registered new interface driver hub
[    0.079442] usbcore: registered new device driver usb
[    0.079496] EDAC MC: Ver: 3.0.0
[    0.079838] NetLabel: Initializing
[    0.079841] NetLabel:  domain hash size = 128
[    0.079843] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.079856] NetLabel:  unlabeled traffic allowed by default
[    0.079978] clocksource: Switched to clocksource timebase
[    0.086272] VFS: Disk quotas dquot_6.6.0
[    0.086308] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
[    0.087857] NET: Registered PF_INET protocol family
[    0.088002] IP idents hash table entries: 262144 (order: 5, 2097152 bytes, linear)
[    0.090039] tcp_listen_portaddr_hash hash table entries: 32768 (order: 3, 524288 bytes, linear)
[    0.090105] TCP established hash table entries: 524288 (order: 6, 4194304 bytes, linear)
[    0.090810] TCP bind hash table entries: 65536 (order: 4, 1048576 bytes, linear)
[    0.090887] TCP: Hash tables configured (established 524288 bind 65536)
[    0.090934] UDP hash table entries: 32768 (order: 4, 1048576 bytes, linear)
[    0.091014] UDP-Lite hash table entries: 32768 (order: 4, 1048576 bytes, linear)
[    0.091153] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.091282] PCI: CLS 128 bytes, default 128
[    0.091355] Trying to unpack rootfs image as initramfs...
[    0.117522] vio_register_device_node: node lid missing 'reg'
[    0.117633] vas: GZIP feature is available
[    0.118526] hv-24x7: read 548 catalog entries, created 387 event attrs (0 failures), 387 descs
[    0.120504] Initialise system trusted keyrings
[    0.120560] workingset: timestamp_bits=38 max_order=20 bucket_order=0
[    0.121816] zbud: loaded
[    0.132036] NET: Registered PF_ALG protocol family
[    0.132043] Key type asymmetric registered
[    0.132047] Asymmetric key parser 'x509' registered
[    0.132063] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
[    0.132107] io scheduler mq-deadline registered
[    0.132112] io scheduler kyber registered
[    0.132382] atomic64_test: passed
[    0.132699] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.132892] Non-volatile memory driver v1.3
[    0.132916] Linux agpgart interface v0.103
[    0.133081] rdac: device handler registered
[    0.133119] hp_sw: device handler registered
[    0.133123] emc: device handler registered
[    0.133156] alua: device handler registered
[    0.133239] libphy: Fixed MDIO Bus: probed
[    0.133271] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.133281] ehci-pci: EHCI PCI platform driver
[    0.133290] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.133299] ohci-pci: OHCI PCI platform driver
[    0.133307] uhci_hcd: USB Universal Host Controller Interface driver
[    0.133339] usbcore: registered new interface driver usbserial_generic
[    0.133347] usbserial: USB Serial support registered for generic
[    0.133402] mousedev: PS/2 mouse device common for all mice
[    0.133506] rtc-generic rtc-generic: registered as rtc0
[    0.133532] rtc-generic rtc-generic: setting system clock to 2021-09-16T14:16:53 UTC (1631801813)
[    0.133611] xcede: xcede_record_size = 10
[    0.133615] xcede: Record 0 : hint = 1, latency = 0x1400 tb ticks, Wake-on-irq = 1
[    0.133620] xcede: Record 1 : hint = 2, latency = 0x3c00 tb ticks, Wake-on-irq = 0
[    0.133625] cpuidle: Skipping the 2 Extended CEDE idle states
[    0.133628] cpuidle: Fixed up CEDE exit latency to 10 us
[    0.133938] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sync_size new:65536 old:0
[    0.133945] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sync_sg new:510 old:0
[    0.133950] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sg_len new:4080 old:0
[    0.134062] alg: No test for 842 (842-nx)
[    0.134137] hid: raw HID events driver (C) Jiri Kosina
[    0.134250] usbcore: registered new interface driver usbhid
[    0.134256] usbhid: USB HID core driver
[    0.134271] drop_monitor: Initializing network drop monitor service
[    0.134350] Initializing XFRM netlink socket
[    0.134466] NET: Registered PF_INET6 protocol family
[    0.134706] Segment Routing with IPv6
[    0.134714] In-situ OAM (IOAM) with IPv6
[    0.134732] NET: Registered PF_PACKET protocol family
[    0.134786] Running instruction emulation self-tests ...
[    0.134792] emulate_step_test: ld             : PASS
[    0.134797] emulate_step_test: pld            : PASS
[    0.134801] emulate_step_test: lwz            : PASS
[    0.134806] emulate_step_test: plwz           : PASS
[    0.134810] emulate_step_test: lwzx           : PASS
[    0.134814] emulate_step_test: std            : PASS
[    0.134818] emulate_step_test: pstd           : PASS
[    0.134823] emulate_step_test: ldarx / stdcx. : PASS
[    0.134830] emulate_step_test: lfsx           : PASS
[    0.134834] emulate_step_test: stfsx          : PASS
[    0.134838] emulate_step_test: plfs           : PASS
[    0.134842] emulate_step_test: pstfs          : PASS
[    0.134846] emulate_step_test: lfdx           : PASS
[    0.134849] emulate_step_test: stfdx          : PASS
[    0.134853] emulate_step_test: plfd           : PASS
[    0.134856] emulate_step_test: pstfd          : PASS
[    0.134861] emulate_step_test: lvx            : PASS
[    0.134865] emulate_step_test: stvx           : PASS
[    0.134870] emulate_step_test: lxvd2x         : PASS
[    0.134875] emulate_step_test: stxvd2x        : PASS
[    0.134879] emulate_step_test: lxvp           : PASS
[    0.134884] emulate_step_test: stxvp          : PASS
[    0.134888] emulate_step_test: lxvpx          : PASS
[    0.134892] emulate_step_test: stxvpx         : PASS
[    0.134896] emulate_step_test: plxvp          : PASS
[    0.134900] emulate_step_test: pstxvp         : PASS
[    0.134904] emulate_step_test: nop            : R0 = LONG_MAX                                      PASS
[    0.134910] emulate_step_test: setb           : BFA = 1, CR = GT                                   PASS
[    0.134916] emulate_step_test: setb           : BFA = 4, CR = LT                                   PASS
[    0.134921] emulate_step_test: setb           : BFA = 5, CR = EQ                                   PASS
[    0.134926] emulate_step_test: add            : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.134931] emulate_step_test: add            : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.134936] emulate_step_test: add            : RA = LONG_MAX, RB = LONG_MAX                       PASS
[    0.134941] emulate_step_test: add            : RA = ULONG_MAX, RB = ULONG_MAX                     PASS
[    0.134946] emulate_step_test: add            : RA = ULONG_MAX, RB = 0x1                           PASS
[    0.134951] emulate_step_test: add            : RA = INT_MIN, RB = INT_MIN                         PASS
[    0.134956] emulate_step_test: add            : RA = INT_MIN, RB = INT_MAX                         PASS
[    0.134961] emulate_step_test: add            : RA = INT_MAX, RB = INT_MAX                         PASS
[    0.134967] emulate_step_test: add            : RA = UINT_MAX, RB = UINT_MAX                       PASS
[    0.134972] emulate_step_test: add            : RA = UINT_MAX, RB = 0x1                            PASS
[    0.134977] emulate_step_test: add.           : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.134982] emulate_step_test: add.           : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.134987] emulate_step_test: add.           : RA = LONG_MAX, RB = LONG_MAX                       PASS
[    0.134992] emulate_step_test: add.           : RA = ULONG_MAX, RB = ULONG_MAX                     PASS
[    0.134997] emulate_step_test: add.           : RA = ULONG_MAX, RB = 0x1                           PASS
[    0.135002] emulate_step_test: add.           : RA = INT_MIN, RB = INT_MIN                         PASS
[    0.135007] emulate_step_test: add.           : RA = INT_MIN, RB = INT_MAX                         PASS
[    0.135013] emulate_step_test: add.           : RA = INT_MAX, RB = INT_MAX                         PASS
[    0.135018] emulate_step_test: add.           : RA = UINT_MAX, RB = UINT_MAX                       PASS
[    0.135022] emulate_step_test: add.           : RA = UINT_MAX, RB = 0x1                            PASS
[    0.135028] emulate_step_test: addc           : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.135033] emulate_step_test: addc           : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.135038] emulate_step_test: addc           : RA = LONG_MAX, RB = LONG_MAX                       PASS
[    0.135043] emulate_step_test: addc           : RA = ULONG_MAX, RB = ULONG_MAX                     PASS
[    0.135048] emulate_step_test: addc           : RA = ULONG_MAX, RB = 0x1                           PASS
[    0.135053] emulate_step_test: addc           : RA = INT_MIN, RB = INT_MIN                         PASS
[    0.135058] emulate_step_test: addc           : RA = INT_MIN, RB = INT_MAX                         PASS
[    0.135064] emulate_step_test: addc           : RA = INT_MAX, RB = INT_MAX                         PASS
[    0.135069] emulate_step_test: addc           : RA = UINT_MAX, RB = UINT_MAX                       PASS
[    0.135074] emulate_step_test: addc           : RA = UINT_MAX, RB = 0x1                            PASS
[    0.135079] emulate_step_test: addc           : RA = LONG_MIN | INT_MIN, RB = LONG_MIN | INT_MIN   PASS
[    0.135085] emulate_step_test: addc.          : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.135090] emulate_step_test: addc.          : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.135098] emulate_step_test: addc.          : RA = LONG_MAX, RB = LONG_MAX                       PASS
[    0.135103] emulate_step_test: addc.          : RA = ULONG_MAX, RB = ULONG_MAX                     PASS
[    0.135109] emulate_step_test: addc.          : RA = ULONG_MAX, RB = 0x1                           PASS
[    0.135114] emulate_step_test: addc.          : RA = INT_MIN, RB = INT_MIN                         PASS
[    0.135119] emulate_step_test: addc.          : RA = INT_MIN, RB = INT_MAX                         PASS
[    0.135124] emulate_step_test: addc.          : RA = INT_MAX, RB = INT_MAX                         PASS
[    0.135130] emulate_step_test: addc.          : RA = UINT_MAX, RB = UINT_MAX                       PASS
[    0.135135] emulate_step_test: addc.          : RA = UINT_MAX, RB = 0x1                            PASS
[    0.135140] emulate_step_test: addc.          : RA = LONG_MIN | INT_MIN, RB = LONG_MIN | INT_MIN   PASS
[    0.135146] emulate_step_test: divde          : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.135151] emulate_step_test: divde          : RA = 1L, RB = 0                                    PASS
[    0.135156] emulate_step_test: divde          : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.135162] emulate_step_test: divde.         : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.135167] emulate_step_test: divde.         : RA = 1L, RB = 0                                    PASS
[    0.135172] emulate_step_test: divde.         : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.135177] emulate_step_test: divdeu         : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.135183] emulate_step_test: divdeu         : RA = 1L, RB = 0                                    PASS
[    0.135188] emulate_step_test: divdeu         : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.135193] emulate_step_test: divdeu         : RA = LONG_MAX - 1, RB = LONG_MAX                   PASS
[    0.135198] emulate_step_test: divdeu         : RA = LONG_MIN + 1, RB = LONG_MIN                   PASS
[    0.135203] emulate_step_test: divdeu.        : RA = LONG_MIN, RB = LONG_MIN                       PASS
[    0.135208] emulate_step_test: divdeu.        : RA = 1L, RB = 0                                    PASS
[    0.135213] emulate_step_test: divdeu.        : RA = LONG_MIN, RB = LONG_MAX                       PASS
[    0.135219] emulate_step_test: divdeu.        : RA = LONG_MAX - 1, RB = LONG_MAX                   PASS
[    0.135224] emulate_step_test: divdeu.        : RA = LONG_MIN + 1, RB = LONG_MIN                   PASS
[    0.135229] emulate_step_test: paddi          : RA = LONG_MIN, SI = SI_MIN, R = 0                  PASS
[    0.135234] emulate_step_test: paddi          : RA = LONG_MIN, SI = SI_MAX, R = 0                  PASS
[    0.135239] emulate_step_test: paddi          : RA = LONG_MAX, SI = SI_MAX, R = 0                  PASS
[    0.135245] emulate_step_test: paddi          : RA = ULONG_MAX, SI = SI_UMAX, R = 0                PASS
[    0.135250] emulate_step_test: paddi          : RA = ULONG_MAX, SI = 0x1, R = 0                    PASS
[    0.135255] emulate_step_test: paddi          : RA = INT_MIN, SI = SI_MIN, R = 0                   PASS
[    0.135260] emulate_step_test: paddi          : RA = INT_MIN, SI = SI_MAX, R = 0                   PASS
[    0.135265] emulate_step_test: paddi          : RA = INT_MAX, SI = SI_MAX, R = 0                   PASS
[    0.135270] emulate_step_test: paddi          : RA = UINT_MAX, SI = 0x1, R = 0                     PASS
[    0.135276] emulate_step_test: paddi          : RA = UINT_MAX, SI = SI_MAX, R = 0                  PASS
[    0.135281] emulate_step_test: paddi          : RA is r0, SI = SI_MIN, R = 0                       PASS
[    0.135286] emulate_step_test: paddi          : RA = 0, SI = SI_MIN, R = 0                         PASS
[    0.135292] emulate_step_test: paddi          : RA is r0, SI = 0, R = 1                            PASS
[    0.135297] emulate_step_test: paddi          : RA is r0, SI = SI_MIN, R = 1                       PASS
[    0.135302] emulate_step_test: paddi          : RA = R22(0), SI = 0, R = 1                         PASS
[    0.136064] registered taskstats version 1
[    0.136315] Loading compiled-in X.509 certificates
[    0.402845] random: fast init done
[    0.863696] Freeing initrd memory: 65856K
[    0.865419] alg: No test for pkcs1pad(rsa,sha1) (pkcs1pad(rsa-generic,sha1))
[    0.865939] Loaded X.509 cert 'Build time autogenerated kernel key: a5f791480b3ed63caeaff187da537ab36b5500c1'
[    0.866007] zswap: loaded using pool lzo/zbud
[    0.867046] pstore: Using crash dump compression: deflate
[    0.920244] Freeing unused kernel image (initmem) memory: 3904K
[    0.920254] Kernel memory protection not selected by kernel config.
[    0.920261] Run /init as init process
[    0.926916] systemd[1]: systemd 239 (239-45.el8) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)
[    0.927020] systemd[1]: Detected architecture ppc64-le.
[    0.927026] systemd[1]: Running in initial RAM disk.

Welcome to Red Hat Enterprise Linux 8.4 (Ootpa) dracut-049-135.git20210121.el8 (Initramfs)!

[    0.960343] systemd[1]: Set hostname to <ltcden11-lp3.aus.stglabs.ibm.com>.
[    0.993161] random: systemd: uninitialized urandom read (16 bytes read)
[    0.993191] systemd[1]: Reached target Swap.
[  OK  ] Reached target Swap.
[    0.993296] random: systemd: uninitialized urandom read (16 bytes read)
[    0.993304] systemd[1]: Reached target Timers.
[  OK  ] Reached target Timers.
[    0.993343] random: systemd: uninitialized urandom read (16 bytes read)
[    0.993352] systemd[1]: Reached target Slices.
[  OK  ] Reached target Slices.
[    0.993449] systemd[1]: Listening on udev Control Socket.
[  OK  ] Listening on udev Control Socket.
[    0.993519] systemd[1]: Listening on udev Kernel Socket.
[  OK  ] Listening on udev Kernel Socket.
[    0.993601] systemd[1]: Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket.
         Starting Journal Service...
[  OK  ] Started Memstrack Anylazing Service.
[  OK  ] Reached target Sockets.
[  OK  ] Started Hardware RNG Entropy Gatherer Daemon.
         Starting Load Kernel Modules...
         Starting Create list of required st…ce nodes for the current kernel...
         Starting Setup Virtual Console...
[  OK  ] Started Create list of required sta…vice nodes for the current kernel.
         Starting Create Static Device Nodes in /dev...
[  OK  ] Started Create Static Device Nodes in /dev.
[    1.015760] fuse: init (API version 7.34)
[  OK  ] Started Load Kernel Modules.
         Starting Apply Kernel Variables...
[  OK  ] Started Journal Service.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Setup Virtual Console.
         Starting dracut ask for additional cmdline parameters...
[  OK  ] Started dracut ask for additional cmdline parameters.
         Starting dracut cmdline hook...
[  OK  ] Started dracut cmdline hook.
         Starting dracut pre-udev hook...
[    1.150387] device-mapper: uevent: version 1.0.3
[    1.150462] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@xxxxxxxxxx
[  OK  ] Started dracut pre-udev hook.
         Starting udev Kernel Device Manager...
[  OK  ] Started udev Kernel Device Manager.
         Starting udev Coldplug all Devices...
         Mounting Kernel Configuration File System...
[  OK  ] Mounted Kernel Configuration File System.
[    1.234519] synth uevent: /devices/vio: failed to send uevent
[    1.234534] vio vio: uevent: failed to send synthetic uevent
[    1.234589] synth uevent: /devices/vio/4000: failed to send uevent
[    1.234595] vio 4000: uevent: failed to send synthetic uevent
[    1.234610] synth uevent: /devices/vio/4001: failed to send uevent
[    1.234616] vio 4001: uevent: failed to send synthetic uevent
[    1.234630] synth uevent: /devices/vio/4002: failed to send uevent
[    1.234636] vio 4002: uevent: failed to send synthetic uevent
[    1.234650] synth uevent: /devices/vio/4004: failed to send uevent
[    1.234656] vio 4004: uevent: failed to send synthetic uevent
[  OK  ] Started udev Coldplug all Devices.
         Starting Show Plymouth Boot Screen...
         Starting udev Wait for Complete Device Initialization...
         Starting dracut initqueue hook...
[    1.256579] ipr: IBM Power RAID SCSI Device Driver version: 2.6.4 (March 14, 2017)
[    1.256627] ipr: Entering ipr_probe_ioa
[    1.256636] ipr 0010:01:00.0: Found IOA with IRQ: 0
[    1.256925] ipr 0010:01:00.0: enabling device (0140 -> 0142)
[    1.257138] ipr 0010:01:00.0: ibm,query-pe-dma-windows(53) 10000 8000000 20000010 returned 0
[    1.257893] ipr 0010:01:00.0: ibm,create-pe-dma-window(54) 10000 8000000 20000010 10 24 returned 0 (liobn = 0x70000010 starting addr = 8000000 0)
[    1.260344] ibmveth 30000002 net0: renamed from eth0
[    1.260696] ibmvfc: IBM Virtual Fibre Channel Driver version: 1.0.11 (April 12, 2013)
[    1.261087] scsi host1: IBM POWER Virtual FC Adapter
[  OK  ] Started Show Plymouth Boot Screen.
[    1.261836] ibmvfc 300000cb: Partner initialization complete
[  OK  ] Reached target Paths.
[  OK  ] Started Forward Password Requests to Plymouth Directory Watch.
[    1.270767] ibmvfc 300000cb: Host partition: ltcden11-vios1, device: vfchost2 U78D8.ND0.FGD004S-P0-C1-C0-T1 U9080.HEX.134C488-V100-C203 max sectors 2048
[    1.284176] scsi 1:0:0:0: Direct-Access     IBM      2145             0000 PQ: 0 ANSI: 6
[    1.286246] scsi 1:0:0:0: alua: supports implicit TPGS
[    1.286254] scsi 1:0:0:0: alua: device naa.60050768108001b3a8000000000000c5 port group 10 rel port 680
[    1.286893] scsi 1:0:1:0: Direct-Access     IBM      2145             0000 PQ: 0 ANSI: 6
[    1.287436] scsi 1:0:1:0: alua: supports implicit TPGS
[    1.287442] scsi 1:0:1:0: alua: device naa.60050768108001b3a8000000000000c5 port group 11 rel port e80
[    1.290119] scsi 1:0:0:0: Attached scsi generic sg0 type 0
[    1.290170] scsi 1:0:1:0: Attached scsi generic sg1 type 0
[    1.293817] sd 1:0:1:0: Power-on or device reset occurred
[    1.293972] sd 1:0:0:0: Power-on or device reset occurred
[    1.341504] sd 1:0:1:0: alua: transition timeout set to 60 seconds
[    1.341512] sd 1:0:1:0: alua: port group 11 state A non-preferred supports tolusna
[    1.341631] sd 1:0:0:0: alua: transition timeout set to 60 seconds
[    1.341641] sd 1:0:0:0: alua: port group 10 state N non-preferred supports tolusna
[    1.341966] sd 1:0:1:0: [sdb] 209715200 512-byte logical blocks: (107 GB/100 GiB)
[    1.342053] sd 1:0:1:0: [sdb] Write Protect is off
[    1.342271] sd 1:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[    1.343010] sd 1:0:0:0: [sda] 209715200 512-byte logical blocks: (107 GB/100 GiB)
[    1.343543] sd 1:0:0:0: [sda] Write Protect is off
[    1.344294] sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[    1.382396] ipr: Entering ipr_test_msi
[    1.382482] ipr 0010:01:00.0: IRQ assigned: 60
[    1.382493] ipr 0010:01:00.0: Received IRQ : 60
[    1.389983] ipr 0010:01:00.0: MSI test succeeded.
[    1.390005] ipr: Leaving ipr_test_msi
[    1.390010] ipr 0010:01:00.0: Request for 16 MSI-Xs succeeded.
[    1.390485] ipr: Entering ipr_alloc_mem
[    1.390894] ipr: Leaving ipr_alloc_mem
[    1.392712] ipr: Leaving ipr_probe_ioa
[    1.392716] ipr: Entering ipr_probe_ioa_part2
[    1.392721] ipr: Entering ipr_reset_enable_ioa
[    1.392777] ipr: Entering ipr_ioafp_identify_hrrq
[    1.392782] ipr 0010:01:00.0: Starting IOA initialization sequence.
[    1.392787] ipr: Leaving ipr_ioafp_identify_hrrq
[    1.392792] ipr: Leaving ipr_probe_ioa_part2
[    1.392797] scsi host0: IBM 0 Storage Adapter
[    1.392974] ipr: Entering ipr_worker_thread
[    1.392978] ipr: Leaving ipr_worker_thread
[    1.392982] ipr: Entering ipr_add_remove_thread
[    1.393003] ipr: Entering ipr_probe_ioa
[    1.393008] ipr 0134:80:00.0: Found IOA with IRQ: 0
[    1.393228] ipr 0134:80:00.0: enabling device (0140 -> 0142)
[    1.393424] ipr 0134:80:00.0: ibm,query-pe-dma-windows(53) 800000 8000000 20000134 returned 0
[    1.394214] ipr 0134:80:00.0: ibm,create-pe-dma-window(54) 800000 8000000 20000134 10 24 returned 0 (liobn = 0x70000134 starting addr = 8000000 0)
[    1.442260] ipr: Entering ipr_test_msi
[    1.442381] ipr 0134:80:00.0: IRQ assigned: 76
[    1.442388] ipr 0134:80:00.0: Received IRQ : 76
[    1.442395] ipr 0134:80:00.0: MSI test succeeded.
[    1.442407] ipr: Leaving ipr_test_msi
[    1.442410] ipr 0134:80:00.0: Request for 16 MSI-Xs succeeded.
[    1.442858] ipr: Entering ipr_alloc_mem
[    1.443204] ipr: Leaving ipr_alloc_mem
[    1.445029] ipr: Leaving ipr_probe_ioa
[    1.445032] ipr: Entering ipr_probe_ioa_part2
[    1.445035] ipr: Entering ipr_reset_enable_ioa
[    1.445128] ipr: Entering ipr_ioafp_identify_hrrq
[    1.445131] ipr 0134:80:00.0: Starting IOA initialization sequence.
[    1.445134] ipr: Leaving ipr_ioafp_identify_hrrq
[    1.445137] ipr: Leaving ipr_probe_ioa_part2
[    1.445140] scsi host2: IBM 0 Storage Adapter
[    1.445276] ipr: Entering ipr_worker_thread
[    1.445279] ipr: Leaving ipr_worker_thread
[    1.445282] ipr: Entering ipr_add_remove_thread
[    1.463025]  sdb: sdb1 sdb2 sdb3
[    1.464074] sd 1:0:1:0: [sdb] Attached SCSI disk
[    1.485033]  sda: sda1 sda2 sda3
[    1.488382] sd 1:0:0:0: [sda] Attached SCSI disk
[    3.180796] random: crng init done
[    3.180803] random: 7 urandom warning(s) missed due to ratelimiting
[   31.839978] ipr: Entering ipr_timeout
[   31.839978] ipr: Entering ipr_timeout
[   31.839982] ipr 0010:01:00.0: Adapter being reset due to command timeout.
[   31.839991] ipr 0134:80:00.0: Adapter being reset due to command timeout.
[   31.839993] ipr: Entering ipr_reset_shutdown_ioa
[   31.839997] ipr: Entering ipr_reset_shutdown_ioa
[   31.839998] ipr: Leaving ipr_reset_shutdown_ioa
[   31.840001] ipr: Leaving ipr_reset_shutdown_ioa
[   31.840003] ipr: Entering ipr_reset_alert
[   31.840005] ipr: Entering ipr_reset_alert
[   31.840161] ipr: Entering ipr_reset_start_timer
[   31.840165] ipr: Leaving ipr_reset_alert
[   31.840167] EEH: Recovering PHB#10-PE#10000
[   31.840169] ipr: Leaving ipr_timeout
[   31.840177] EEH: PE location: N/A, PHB location: N/A
[   31.840180] EEH: Frozen PHB#10-PE#10000 detected
[   31.840184] EEH: Call Trace:
[   31.840186] EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
[   31.840195] EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
[   31.840200] EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
[   31.840204] EEH: [c0080000008cd4c0] ipr_mask_and_clear_interrupts+0x1d8/0x200 [ipr]
[   31.840213] EEH: [c0080000008cd620] ipr_reset_alert+0x138/0x180 [ipr]
[   31.840220] EEH: [c0080000008c49a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
[   31.840226] EEH: [c0080000008c7f00] ipr_timeout+0xd8/0x160 [ipr]
[   31.840232] EEH: [c0000000002201f0] call_timer_fn+0x50/0x200
[   31.840237] EEH: [c000000000221b90] run_timer_softirq+0x340/0x7c0
[   31.840241] EEH: [c000000000c95c7c] __do_softirq+0x15c/0x3d0
[   31.840246] EEH: [c000000000152058] irq_exit+0x168/0x1b0
[   31.840251] EEH: [c000000000027458] timer_interrupt+0x178/0x3e0
[   31.840256] EEH: [c000000000009a60] decrementer_common_virt+0x210/0x220
[   31.840260] EEH: [c0000000019afd40] init_stack+0x3d40/0x4000
[   31.840264] EEH: [c0000000009bc424] dedicated_cede_loop+0x94/0x1a0
[   31.840270] EEH: [c0000000009b8da4] cpuidle_enter_state+0x2d4/0x4e0
[   31.840274] EEH: [c0000000009b9048] cpuidle_enter+0x48/0x70
[   31.840277] EEH: [c0000000001a0ad4] call_cpuidle+0x44/0x80
[   31.840282] EEH: [c0000000001a10b0] do_idle+0x340/0x390
[   31.840286] EEH: [c0000000001a1334] cpu_startup_entry+0x34/0x40
[   31.840291] EEH: [c000000000012974] rest_init+0xf4/0x10c
[   31.840294] EEH: [c0000000010b4220] start_kernel+0x868/0x8bc
[   31.840299] ipr: Entering ipr_reset_start_timer
[   31.840299] EEH: [c00000000000d49c] start_here_common+0x1c/0x600
[   31.840303] ipr: Leaving ipr_reset_alert
[   31.840305] EEH: This PCI device has failed 1 times in the last hour and will be permanently disabled after 5 failures.
[   31.840307] EEH: Notify device drivers to shutdown
[   31.840311] ipr: Leaving ipr_timeout
[   31.840313] EEH: Beginning: 'error_detected(IO frozen)'
[   31.840315] PCI 0010:01:00.0#10000: EEH: Invoking ipr->error_detected(IO frozen)
[   31.840335] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'can recover'
[   31.840341] EEH: Finished:'error_detected(IO frozen)' with aggregate recovery state:'can recover'
[   31.840392] EEH: Collect temporary log
[   31.840891] EEH: of node=0010:01:00.0
[   31.840923] EEH: PCI device/vendor: 034a1014
[   31.840954] EEH: PCI cmd/status register: 00100546
[   31.840958] EEH: PCI-E capabilities and status follow:
[   31.841102] EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503 
[   31.841218] EEH: PCI-E 10: 10830000 00000000 00000000 00000000 
[   31.841222] EEH: PCI-E 20: 00000000 
[   31.841224] EEH: PCI-E AER capability register set follows:
[   31.841367] EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030 
[   31.841483] EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000 
[   31.841598] EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000 
[   31.841630] EEH: PCI-E AER 30: 00000000 00000000 
[   31.842516] EEH: Enable I/O for affected devices
[   31.842551] EEH: Notify device drivers to resume I/O
[   31.842554] EEH: Beginning: 'mmio_enabled'
[   31.842557] PCI 0010:01:00.0#10000: EEH: Invoking ipr->mmio_enabled()
[   31.842559] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'need reset'
[   31.842562] EEH: Finished:'mmio_enabled' with aggregate recovery state:'need reset'
[   31.842568] EEH: Reset without hotplug activity
[   31.949977] ipr: Entering ipr_reset_start_timer
[   32.059975] ipr: Entering ipr_reset_start_timer
[   32.169974] ipr: Entering ipr_reset_start_timer
[   32.279975] ipr: Entering ipr_reset_start_timer
[   32.389975] ipr: Entering ipr_reset_start_timer
[   32.499975] ipr: Entering ipr_reset_start_timer
[   32.609975] ipr: Entering ipr_reset_start_timer
[   32.719976] ipr: Entering ipr_reset_start_timer
[   32.829975] ipr: Entering ipr_reset_start_timer
[   32.939975] ipr: Entering ipr_reset_start_timer
[   33.049975] ipr: Entering ipr_reset_start_timer
[   33.159975] ipr: Entering ipr_reset_start_timer
[   33.269975] ipr: Entering ipr_reset_start_timer
[   33.379976] ipr: Entering ipr_reset_start_timer
[   33.489975] ipr: Entering ipr_reset_start_timer
[   33.599975] ipr: Entering ipr_reset_start_timer
[   33.709975] ipr: Entering ipr_reset_start_timer
[   33.819975] ipr: Entering ipr_reset_start_timer
[   33.929975] ipr: Entering ipr_reset_start_timer
[   34.000356] ipr 0010:01:00.0: enabling device (0140 -> 0142)
[   34.023909] EEH: Beginning: 'slot_reset'
[   34.023913] PCI 0010:01:00.0#10000: EEH: Invoking ipr->slot_reset()
[   34.023915] ipr: Entering ipr_reset_restore_cfg_space
[   34.025673] ipr: Entering ipr_fail_all_ops
[   34.025677] ipr: Leaving ipr_fail_all_ops
[   34.025680] ipr: Leaving ipr_reset_restore_cfg_space
[   34.025683] ipr: Entering ipr_reset_enable_ioa
[   34.025697] ipr 0010:01:00.0: Initializing IOA.
[   34.025897] ipr: IPL stage = 0xff000000, IPL stage time = 65535
[   34.025901] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'recovered'
[   34.025903] EEH: Finished:'slot_reset' with aggregate recovery state:'recovered'
[   34.025908] EEH: Notify device driver to resume
[   34.025911] EEH: Beginning: 'resume'
[   34.025913] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'none'
[   34.025915] EEH: Finished:'resume'
[   34.025918] EEH: Recovery successful.
[   34.025922] EEH: Recovering PHB#10-PE#10000
[   34.025925] EEH: PE location: N/A, PHB location: N/A
[   34.025927] EEH: Frozen PHB#10-PE#10000 detected
[   34.025930] EEH: Call Trace:
[   34.025932] EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
[   34.025938] EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
[   34.025942] EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
[   34.025946] EEH: [c0080000008d1c70] ipr_reset_next_stage+0x198/0x370 [ipr]
[   34.025953] EEH: [c0080000008c49a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
[   34.025960] EEH: [c0080000008c8f04] ipr_pci_slot_reset+0xac/0x140 [ipr]
[   34.025966] EEH: [c000000000052ce8] eeh_report_reset+0xb8/0xe0
[   34.025970] EEH: [c000000000054834] eeh_pe_report+0x254/0x4b4
[   34.025974] EEH: [c000000000053efc] eeh_handle_normal_event+0x9dc/0xa80
[   34.025979] EEH: [c000000000054de8] eeh_event_handler+0x108/0x190
[   34.025983] EEH: [c00000000018151c] kthread+0x18c/0x1a0
[   34.025987] EEH: [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[   34.025991] EEH: This PCI device has failed 2 times in the last hour and will be permanently disabled after 5 failures.
[   34.025995] EEH: Notify device drivers to shutdown
[   34.025998] EEH: Beginning: 'error_detected(IO frozen)'
[   34.026000] PCI 0010:01:00.0#10000: EEH: Invoking ipr->error_detected(IO frozen)
[   34.026011] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'can recover'
[   34.026016] EEH: Finished:'error_detected(IO frozen)' with aggregate recovery state:'can recover'
[   34.026059] EEH: Collect temporary log
[   34.026537] EEH: of node=0010:01:00.0
[   34.026568] EEH: PCI device/vendor: 034a1014
[   34.026600] EEH: PCI cmd/status register: 00100546
[   34.026603] EEH: PCI-E capabilities and status follow:
[   34.026745] EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503 
[   34.026860] EEH: PCI-E 10: 10830000 00000000 00000000 00000000 
[   34.026863] EEH: PCI-E 20: 00000000 
[   34.026866] EEH: PCI-E AER capability register set follows:
[   34.027010] EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030 
[   34.027125] EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000 
[   34.027239] EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000 
[   34.027270] EEH: PCI-E AER 30: 00000000 00000000 
[   34.028118] EEH: Enable I/O for affected devices
[   34.028153] EEH: Notify device drivers to resume I/O
[   34.028155] EEH: Beginning: 'mmio_enabled'
[   34.028158] PCI 0010:01:00.0#10000: EEH: Invoking ipr->mmio_enabled()
[   34.028159] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'need reset'
[   34.028163] EEH: Finished:'mmio_enabled' with aggregate recovery state:'need reset'
[   34.028168] EEH: Reset without hotplug activity
[   34.039978] ipr: Entering ipr_reset_start_timer
[   34.149976] ipr: Entering ipr_reset_start_bist
[   34.149981] ipr: Entering ipr_reset_start_timer
[   34.149985] ipr: Leaving ipr_reset_start_bist
[   36.159976] ipr: Entering ipr_reset_bist_done
[   36.159981] ipr: Leaving ipr_reset_bist_done
[   36.159984] ipr: Entering ipr_reset_restore_cfg_space
[   36.162229] ipr: Entering ipr_fail_all_ops
[   36.162232] ipr: Leaving ipr_fail_all_ops
[   36.162234] ipr: Leaving ipr_reset_restore_cfg_space
[   36.162237] ipr: Entering ipr_reset_enable_ioa
[   36.162251] ipr: Entering ipr_ioafp_identify_hrrq
[   36.162254] ipr 0134:80:00.0: Starting IOA initialization sequence.
[   36.162258] ipr: Leaving ipr_ioafp_identify_hrrq
[   36.162637] ipr 0010:01:00.0: enabling device (0140 -> 0142)
[   36.185512] EEH: Beginning: 'slot_reset'
[   36.185516] PCI 0010:01:00.0#10000: EEH: Invoking ipr->slot_reset()
[   36.185518] ipr: Entering ipr_reset_restore_cfg_space
[   36.187270] ipr: Entering ipr_fail_all_ops
[   36.187274] ipr: Leaving ipr_fail_all_ops
[   36.187276] ipr: Leaving ipr_reset_restore_cfg_space
[   36.187279] ipr: Entering ipr_reset_enable_ioa
[   36.187286] ipr 0010:01:00.0: Initializing IOA.
[   36.187480] ipr: IPL stage = 0xff000000, IPL stage time = 65535
[   36.187484] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'recovered'
[   36.187486] EEH: Finished:'slot_reset' with aggregate recovery state:'recovered'
[   36.187491] EEH: Notify device driver to resume
[   36.187494] EEH: Beginning: 'resume'
[   36.187496] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'none'
[   36.187497] EEH: Finished:'resume'
[   36.187500] EEH: Recovery successful.
[   36.187504] EEH: Recovering PHB#10-PE#10000
[   36.187507] EEH: PE location: N/A, PHB location: N/A
[   36.187509] EEH: Frozen PHB#10-PE#10000 detected
[   36.187512] EEH: Call Trace:
[   36.187514] EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
[   36.187519] EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
[   36.187523] EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
[   36.187527] EEH: [c0080000008d1c70] ipr_reset_next_stage+0x198/0x370 [ipr]
[   36.187534] EEH: [c0080000008c49a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
[   36.187541] EEH: [c0080000008c8f04] ipr_pci_slot_reset+0xac/0x140 [ipr]
[   36.187547] EEH: [c000000000052ce8] eeh_report_reset+0xb8/0xe0
[   36.187552] EEH: [c000000000054834] eeh_pe_report+0x254/0x4b4
[   36.187556] EEH: [c000000000053efc] eeh_handle_normal_event+0x9dc/0xa80
[   36.187560] EEH: [c000000000054de8] eeh_event_handler+0x108/0x190
[   36.187564] EEH: [c00000000018151c] kthread+0x18c/0x1a0
[   36.187568] EEH: [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[   36.187572] EEH: This PCI device has failed 3 times in the last hour and will be permanently disabled after 5 failures.
[   36.187576] EEH: Notify device drivers to shutdown
[   36.187578] EEH: Beginning: 'error_detected(IO frozen)'
[   36.187581] PCI 0010:01:00.0#10000: EEH: Invoking ipr->error_detected(IO frozen)
[   36.187591] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'can recover'
[   36.187595] EEH: Finished:'error_detected(IO frozen)' with aggregate recovery state:'can recover'
[   36.187636] EEH: Collect temporary log
[   36.188115] EEH: of node=0010:01:00.0
[   36.188146] EEH: PCI device/vendor: 034a1014
[   36.188177] EEH: PCI cmd/status register: 00100546
[   36.188180] EEH: PCI-E capabilities and status follow:
[   36.188324] EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503 
[   36.188439] EEH: PCI-E 10: 10830000 00000000 00000000 00000000 
[   36.188442] EEH: PCI-E 20: 00000000 
[   36.188444] EEH: PCI-E AER capability register set follows:
[   36.188587] EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030 
[   36.188702] EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000 
[   36.188816] EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000 
[   36.188847] EEH: PCI-E AER 30: 00000000 00000000 
[   36.189681] EEH: Enable I/O for affected devices
[   36.189716] EEH: Notify device drivers to resume I/O
[   36.189719] EEH: Beginning: 'mmio_enabled'
[   36.189721] PCI 0010:01:00.0#10000: EEH: Invoking ipr->mmio_enabled()
[   36.189723] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'need reset'
[   36.189726] EEH: Finished:'mmio_enabled' with aggregate recovery state:'need reset'
[   36.189732] EEH: Reset without hotplug activity
[   38.320356] ipr 0010:01:00.0: enabling device (0140 -> 0142)
[   38.343242] EEH: Beginning: 'slot_reset'
[   38.343247] PCI 0010:01:00.0#10000: EEH: Invoking ipr->slot_reset()
[   38.343249] ipr: Entering ipr_reset_restore_cfg_space
[   38.345002] ipr: Entering ipr_fail_all_ops
[   38.345005] ipr: Leaving ipr_fail_all_ops
[   38.345008] ipr: Leaving ipr_reset_restore_cfg_space
[   38.345011] ipr: Entering ipr_reset_enable_ioa
[   38.345018] ipr 0010:01:00.0: Initializing IOA.
[   38.345220] ipr: IPL stage = 0xff000000, IPL stage time = 65535
[   38.345224] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'recovered'
[   38.345226] EEH: Finished:'slot_reset' with aggregate recovery state:'recovered'
[   38.345232] EEH: Notify device driver to resume
[   38.345234] EEH: Beginning: 'resume'
[   38.345236] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'none'
[   38.345238] EEH: Finished:'resume'
[   38.345241] EEH: Recovery successful.
[   38.345245] EEH: Recovering PHB#10-PE#10000
[   38.345247] EEH: PE location: N/A, PHB location: N/A
[   38.345250] EEH: Frozen PHB#10-PE#10000 detected
[   38.345253] EEH: Call Trace:
[   38.345254] EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
[   38.345259] EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
[   38.345264] EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
[   38.345268] EEH: [c0080000008d1c70] ipr_reset_next_stage+0x198/0x370 [ipr]
[   38.345274] EEH: [c0080000008c49a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
[   38.345281] EEH: [c0080000008c8f04] ipr_pci_slot_reset+0xac/0x140 [ipr]
[   38.345287] EEH: [c000000000052ce8] eeh_report_reset+0xb8/0xe0
[   38.345291] EEH: [c000000000054834] eeh_pe_report+0x254/0x4b4
[   38.345295] EEH: [c000000000053efc] eeh_handle_normal_event+0x9dc/0xa80
[   38.345299] EEH: [c000000000054de8] eeh_event_handler+0x108/0x190
[   38.345303] EEH: [c00000000018151c] kthread+0x18c/0x1a0
[   38.345307] EEH: [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[   38.345311] EEH: This PCI device has failed 4 times in the last hour and will be permanently disabled after 5 failures.
[   38.345315] EEH: Notify device drivers to shutdown
[   38.345317] EEH: Beginning: 'error_detected(IO frozen)'
[   38.345319] PCI 0010:01:00.0#10000: EEH: Invoking ipr->error_detected(IO frozen)
[   38.345330] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'can recover'
[   38.345334] EEH: Finished:'error_detected(IO frozen)' with aggregate recovery state:'can recover'
[   38.345376] EEH: Collect temporary log
[   38.345854] EEH: of node=0010:01:00.0
[   38.345885] EEH: PCI device/vendor: 034a1014
[   38.345916] EEH: PCI cmd/status register: 00100546
[   38.345919] EEH: PCI-E capabilities and status follow:
[   38.346062] EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503 
[   38.346177] EEH: PCI-E 10: 10830000 00000000 00000000 00000000 
[   38.346180] EEH: PCI-E 20: 00000000 
[   38.346182] EEH: PCI-E AER capability register set follows:
[   38.346325] EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030 
[   38.346440] EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000 
[   38.346556] EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000 
[   38.346587] EEH: PCI-E AER 30: 00000000 00000000 
[   38.347423] EEH: Enable I/O for affected devices
[   38.347458] EEH: Notify device drivers to resume I/O
[   38.347461] EEH: Beginning: 'mmio_enabled'
[   38.347463] PCI 0010:01:00.0#10000: EEH: Invoking ipr->mmio_enabled()
[   38.347464] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'need reset'
[   38.347468] EEH: Finished:'mmio_enabled' with aggregate recovery state:'need reset'
[   38.347473] EEH: Reset without hotplug activity
[   40.480353] ipr 0010:01:00.0: enabling device (0140 -> 0142)
[   40.503235] EEH: Beginning: 'slot_reset'
[   40.503239] PCI 0010:01:00.0#10000: EEH: Invoking ipr->slot_reset()
[   40.503241] ipr: Entering ipr_reset_restore_cfg_space
[   40.504991] ipr: Entering ipr_fail_all_ops
[   40.504995] ipr: Leaving ipr_fail_all_ops
[   40.504997] ipr: Leaving ipr_reset_restore_cfg_space
[   40.505000] ipr: Entering ipr_reset_enable_ioa
[   40.505007] ipr 0010:01:00.0: Initializing IOA.
[   40.505201] ipr: IPL stage = 0xff000000, IPL stage time = 65535
[   40.505205] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'recovered'
[   40.505207] EEH: Finished:'slot_reset' with aggregate recovery state:'recovered'
[   40.505212] EEH: Notify device driver to resume
[   40.505215] EEH: Beginning: 'resume'
[   40.505217] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'none'
[   40.505218] EEH: Finished:'resume'
[   40.505221] EEH: Recovery successful.
[   40.505225] EEH: Recovering PHB#10-PE#10000
[   40.505227] EEH: PE location: N/A, PHB location: N/A
[   40.505230] EEH: Frozen PHB#10-PE#10000 detected
[   40.505233] EEH: Call Trace:
[   40.505234] EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
[   40.505239] EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
[   40.505244] EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
[   40.505248] EEH: [c0080000008d1c70] ipr_reset_next_stage+0x198/0x370 [ipr]
[   40.505254] EEH: [c0080000008c49a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
[   40.505261] EEH: [c0080000008c8f04] ipr_pci_slot_reset+0xac/0x140 [ipr]
[   40.505267] EEH: [c000000000052ce8] eeh_report_reset+0xb8/0xe0
[   40.505271] EEH: [c000000000054834] eeh_pe_report+0x254/0x4b4
[   40.505275] EEH: [c000000000053efc] eeh_handle_normal_event+0x9dc/0xa80
[   40.505279] EEH: [c000000000054de8] eeh_event_handler+0x108/0x190
[   40.505284] EEH: [c00000000018151c] kthread+0x18c/0x1a0
[   40.505287] EEH: [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[   40.505291] EEH: This PCI device has failed 5 times in the last hour and will be permanently disabled after 5 failures.
[   40.505295] EEH: Notify device drivers to shutdown
[   40.505298] EEH: Beginning: 'error_detected(IO frozen)'
[   40.505300] PCI 0010:01:00.0#10000: EEH: Invoking ipr->error_detected(IO frozen)
[   40.505310] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'can recover'
[   40.505314] EEH: Finished:'error_detected(IO frozen)' with aggregate recovery state:'can recover'
[   40.505355] EEH: Collect temporary log
[   40.505833] EEH: of node=0010:01:00.0
[   40.505864] EEH: PCI device/vendor: 034a1014
[   40.505895] EEH: PCI cmd/status register: 00100546
[   40.505898] EEH: PCI-E capabilities and status follow:
[   40.506041] EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503 
[   40.506157] EEH: PCI-E 10: 10830000 00000000 00000000 00000000 
[   40.506161] EEH: PCI-E 20: 00000000 
[   40.506163] EEH: PCI-E AER capability register set follows:
[   40.506305] EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030 
[   40.506420] EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000 
[   40.506534] EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000 
[   40.506565] EEH: PCI-E AER 30: 00000000 00000000 
[   40.507411] EEH: Enable I/O for affected devices
[   40.507446] EEH: Notify device drivers to resume I/O
[   40.507449] EEH: Beginning: 'mmio_enabled'
[   40.507451] PCI 0010:01:00.0#10000: EEH: Invoking ipr->mmio_enabled()
[   40.507452] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'need reset'
[   40.507456] EEH: Finished:'mmio_enabled' with aggregate recovery state:'need reset'
[   40.507461] EEH: Reset without hotplug activity
[   42.640358] ipr 0010:01:00.0: enabling device (0140 -> 0142)
[   42.663240] EEH: Beginning: 'slot_reset'
[   42.663244] PCI 0010:01:00.0#10000: EEH: Invoking ipr->slot_reset()
[   42.663246] ipr: Entering ipr_reset_restore_cfg_space
[   42.664997] ipr: Entering ipr_fail_all_ops
[   42.665001] ipr: Leaving ipr_fail_all_ops
[   42.665003] ipr: Leaving ipr_reset_restore_cfg_space
[   42.665006] ipr: Entering ipr_reset_enable_ioa
[   42.665013] ipr 0010:01:00.0: Initializing IOA.
[   42.665213] ipr: IPL stage = 0xff000000, IPL stage time = 65535
[   42.665217] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'recovered'
[   42.665219] EEH: Finished:'slot_reset' with aggregate recovery state:'recovered'
[   42.665224] EEH: Notify device driver to resume
[   42.665227] EEH: Beginning: 'resume'
[   42.665229] PCI 0010:01:00.0#10000: EEH: ipr driver reports: 'none'
[   42.665230] EEH: Finished:'resume'
[   42.665233] EEH: Recovery successful.
[   42.665237] EEH: Recovering PHB#10-PE#10000
[   42.665240] EEH: PE location: N/A, PHB location: N/A
[   42.665242] EEH: Frozen PHB#10-PE#10000 detected
[   42.665245] EEH: Call Trace:
[   42.665247] EEH: [c000000000054f90] __eeh_send_failure_event+0x70/0x150
[   42.665252] EEH: [c00000000004e148] eeh_dev_check_failure+0x2e8/0x680
[   42.665256] EEH: [c00000000004e5a8] eeh_check_failure+0xc8/0x100
[   42.665260] EEH: [c0080000008d1c70] ipr_reset_next_stage+0x198/0x370 [ipr]
[   42.665267] EEH: [c0080000008c49a0] ipr_reset_ioa_job+0x78/0x150 [ipr]
[   42.665273] EEH: [c0080000008c8f04] ipr_pci_slot_reset+0xac/0x140 [ipr]
[   42.665279] EEH: [c000000000052ce8] eeh_report_reset+0xb8/0xe0
[   42.665284] EEH: [c000000000054834] eeh_pe_report+0x254/0x4b4
[   42.665288] EEH: [c000000000053efc] eeh_handle_normal_event+0x9dc/0xa80
[   42.665292] EEH: [c000000000054de8] eeh_event_handler+0x108/0x190
[   42.665296] EEH: [c00000000018151c] kthread+0x18c/0x1a0
[   42.665299] EEH: [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[   42.665303] EEH: PHB#10-PE#10000 has failed 6 times in the last hour and has been permanently disabled.
[   42.665307] EEH: Unable to recover from failure from PHB#10-PE#10000.
[   42.665307] Please try reseating or replacing it
[   42.665788] EEH: of node=0010:01:00.0
[   42.665819] EEH: PCI device/vendor: 034a1014
[   42.665851] EEH: PCI cmd/status register: 00100546
[   42.665854] EEH: PCI-E capabilities and status follow:
[   42.665996] EEH: PCI-E 00: 0002a410 00008fc2 0000585e 0003f503 
[   42.666112] EEH: PCI-E 10: 10830000 00000000 00000000 00000000 
[   42.666115] EEH: PCI-E 20: 00000000 
[   42.666117] EEH: PCI-E AER capability register set follows:
[   42.666260] EEH: PCI-E AER 00: 18820001 00000000 00400000 00462030 
[   42.666375] EEH: PCI-E AER 10: 00000000 0000e000 000001e0 00000000 
[   42.666489] EEH: PCI-E AER 20: 00000000 00000000 00000000 00000000 
[   42.666520] EEH: PCI-E AER 30: 00000000 00000000 
[   42.667476] EEH: Beginning: 'error_detected(permanent failure)'
[   42.667479] PCI 0010:01:00.0#10000: EEH: not actionable (1,1,1)
[   42.667481] EEH: Finished:'error_detected(permanent failure)'
[   42.667492] ipr: Entering ipr_remove
[   42.667542] ipr: Entering __ipr_remove
[   66.399976] ipr: Entering ipr_timeout
[   66.399983] ipr 0134:80:00.0: Adapter being reset due to command timeout.
[   66.399989] ipr: Entering ipr_reset_shutdown_ioa
[   66.399992] ipr: Leaving ipr_reset_shutdown_ioa
[   66.399995] ipr: Entering ipr_reset_alert
[   66.400049] ipr: Entering ipr_reset_start_timer
[   66.400054] ipr: Leaving ipr_reset_alert
[   66.400057] ipr: Leaving ipr_timeout
[   66.509975] ipr: Entering ipr_reset_start_timer
[   66.619975] ipr: Entering ipr_reset_start_timer
[   66.729975] ipr: Entering ipr_reset_start_timer
[   66.839975] ipr: Entering ipr_reset_start_timer
[   66.949975] ipr: Entering ipr_reset_start_timer
[   67.059975] ipr: Entering ipr_reset_start_timer
[   67.169975] ipr: Entering ipr_reset_start_timer
[   67.279975] ipr: Entering ipr_reset_start_timer
[   67.389975] ipr: Entering ipr_reset_start_timer
[   67.499975] ipr: Entering ipr_reset_start_timer
[   67.609975] ipr: Entering ipr_reset_start_timer
[   67.719975] ipr: Entering ipr_reset_start_timer
[   67.829975] ipr: Entering ipr_reset_start_timer
[   67.939975] ipr: Entering ipr_reset_start_timer
[   68.049976] ipr: Entering ipr_reset_start_timer
[   68.159975] ipr: Entering ipr_reset_start_timer
[   68.269975] ipr: Entering ipr_reset_start_timer
[   68.379977] ipr: Entering ipr_reset_start_timer
[   68.489975] ipr: Entering ipr_reset_start_timer
[   68.599975] ipr: Entering ipr_reset_start_timer
[   68.709975] ipr: Entering ipr_reset_start_bist
[   68.709979] ipr: Entering ipr_reset_start_timer
[   68.709983] ipr: Leaving ipr_reset_start_bist
[   70.719975] ipr: Entering ipr_reset_bist_done
[   70.719980] ipr: Leaving ipr_reset_bist_done
[   70.719983] ipr: Entering ipr_reset_restore_cfg_space
[   70.722256] ipr: Entering ipr_fail_all_ops
[   70.722260] ipr: Leaving ipr_fail_all_ops
[   70.722263] ipr: Leaving ipr_reset_restore_cfg_space
[   70.722266] ipr: Entering ipr_reset_enable_ioa
[   70.722280] ipr: Entering ipr_ioafp_identify_hrrq
[   70.722283] ipr 0134:80:00.0: Starting IOA initialization sequence.
[   70.722288] ipr: Leaving ipr_ioafp_identify_hrrq
[  100.959975] ipr: Entering ipr_timeout
[  100.959988] ipr 0134:80:00.0: Adapter being reset due to command timeout.
[  100.959995] ipr: Entering ipr_reset_shutdown_ioa
[  100.959998] ipr: Leaving ipr_reset_shutdown_ioa
[  100.960000] ipr: Entering ipr_reset_alert
[  100.960066] ipr: Entering ipr_reset_start_timer
[  100.960070] ipr: Leaving ipr_reset_alert
[  100.960072] ipr: Leaving ipr_timeout
[  101.069976] ipr: Entering ipr_reset_start_timer
[  101.179975] ipr: Entering ipr_reset_start_timer
[  101.289975] ipr: Entering ipr_reset_start_timer
[  101.399977] ipr: Entering ipr_reset_start_timer
[  101.509976] ipr: Entering ipr_reset_start_timer
[  101.619975] ipr: Entering ipr_reset_start_timer
[  101.729975] ipr: Entering ipr_reset_start_timer
[  101.839975] ipr: Entering ipr_reset_start_timer
[  101.949975] ipr: Entering ipr_reset_start_timer
[  102.059975] ipr: Entering ipr_reset_start_timer
[  102.169975] ipr: Entering ipr_reset_start_timer
[  102.279975] ipr: Entering ipr_reset_start_timer
[  102.389976] ipr: Entering ipr_reset_start_timer
[  102.499975] ipr: Entering ipr_reset_start_timer
[  102.609975] ipr: Entering ipr_reset_start_timer
[  102.719975] ipr: Entering ipr_reset_start_timer
[  102.829975] ipr: Entering ipr_reset_start_timer
[  102.939975] ipr: Entering ipr_reset_start_timer
[  103.049975] ipr: Entering ipr_reset_start_timer
[  103.159975] ipr: Entering ipr_reset_start_timer
[  103.269976] ipr: Entering ipr_reset_start_bist
[  103.269980] ipr: Entering ipr_reset_start_timer
[  103.269984] ipr: Leaving ipr_reset_start_bist
[  105.279975] ipr: Entering ipr_reset_bist_done
[  105.279980] ipr: Leaving ipr_reset_bist_done
[  105.279983] ipr: Entering ipr_reset_restore_cfg_space
[  105.282241] ipr: Entering ipr_fail_all_ops
[  105.282245] ipr: Leaving ipr_fail_all_ops
[  105.282248] ipr: Leaving ipr_reset_restore_cfg_space
[  105.282251] ipr: Entering ipr_reset_enable_ioa
[  105.282265] ipr: Entering ipr_ioafp_identify_hrrq
[  105.282268] ipr 0134:80:00.0: Starting IOA initialization sequence.
[  105.282273] ipr: Leaving ipr_ioafp_identify_hrrq
[FAILED] Failed to start udev Wait for Complete Device Initialization.
See 'systemctl status systemd-udev-settle.service' for details.
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[  121.492754] device-mapper: multipath service-time: version 0.3.0 loaded
[  135.519977] ipr: Entering ipr_timeout
[  135.519991] ipr 0134:80:00.0: Adapter being reset due to command timeout.
[  135.519996] ipr 0134:80:00.0: IOA taken offline - error recovery failed
[  135.520002] ipr: Entering ipr_reset_shutdown_ioa
[  135.520005] ipr: Leaving ipr_reset_shutdown_ioa
[  135.520008] ipr: Entering ipr_reset_alert
[  135.520083] ipr: Entering ipr_reset_start_timer
[  135.520088] ipr: Leaving ipr_reset_alert
[  135.520092] ipr: Leaving ipr_timeout
[  135.629975] ipr: Entering ipr_reset_start_timer
[  135.739975] ipr: Entering ipr_reset_start_timer
[  135.849975] ipr: Entering ipr_reset_start_timer
[  135.959975] ipr: Entering ipr_reset_start_timer
[  136.069977] ipr: Entering ipr_reset_start_timer
[  136.179975] ipr: Entering ipr_reset_start_timer
[  136.289975] ipr: Entering ipr_reset_start_timer
[  136.399975] ipr: Entering ipr_reset_start_timer
[  136.509975] ipr: Entering ipr_reset_start_timer
[  136.619976] ipr: Entering ipr_reset_start_timer
[  136.729976] ipr: Entering ipr_reset_start_timer
[  136.839976] ipr: Entering ipr_reset_start_timer
[  136.949975] ipr: Entering ipr_reset_start_timer
[  137.059977] ipr: Entering ipr_reset_start_timer
[  137.169975] ipr: Entering ipr_reset_start_timer
[  137.279975] ipr: Entering ipr_reset_start_timer
[  137.389975] ipr: Entering ipr_reset_start_timer
[  137.499975] ipr: Entering ipr_reset_start_timer
[  137.609975] ipr: Entering ipr_reset_start_timer
[  137.719975] ipr: Entering ipr_reset_start_timer
[  137.829976] ipr: Entering ipr_reset_start_bist
[  137.829980] ipr: Entering ipr_reset_start_timer
[  137.829984] ipr: Leaving ipr_reset_start_bist
[  139.839975] ipr: Entering ipr_reset_bist_done
[  139.839980] ipr: Leaving ipr_reset_bist_done
[  139.839983] ipr: Entering ipr_reset_restore_cfg_space
[  139.842243] ipr: Entering ipr_fail_all_ops
[  139.842247] ipr: Leaving ipr_fail_all_ops
[  139.842249] ipr: Leaving ipr_reset_restore_cfg_space
[  139.842252] ipr: Entering ipr_ioa_bringdown_done
[  139.842255] ipr: drivers/scsi/ipr.c: ipr_ioa_bringdown_done: Line: 7243
[  139.842259] ipr: Leaving ipr_ioa_bringdown_done
[  139.842266] ipr: Entering ipr_worker_thread
[  139.842270] ipr: Leaving ipr_worker_thread
[  139.842273] ipr: Entering ipr_add_remove_thread
[  243.679987] INFO: task eehd:137 blocked for more than 122 seconds.
[  243.680001]       Not tainted 5.15.0-rc1-autotest #5
[  243.680005] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  243.680009] task:eehd            state:D stack:    0 pid:  137 ppid:     2 flags:0x00000800
[  243.680016] Call Trace:
[  243.680018] [c000000008167650] [c0000000001dd368] info_print_prefix+0xa8/0x130 (unreliable)
[  243.680027] [c000000008167840] [c00000000001e6c8] __switch_to+0x278/0x490
[  243.680033] [c0000000081678a0] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  243.680039] [c000000008167960] [c000000000c8c128] schedule+0x68/0x120
[  243.680043] [c000000008167990] [c0080000008ca3c4] __ipr_remove+0x10c/0x4b0 [ipr]
[  243.680053] [c000000008167a60] [c0080000008ca828] ipr_remove+0xc0/0x110 [ipr]
[  243.680061] [c000000008167ae0] [c000000000732bd4] pci_device_remove+0x64/0x120
[  243.680067] [c000000008167b20] [c000000000811318] device_release_driver_internal+0x168/0x2c0
[  243.680073] [c000000008167b60] [c000000000724618] pci_stop_bus_device+0xa8/0x100
[  243.680079] [c000000008167ba0] [c000000000724810] pci_stop_and_remove_bus_device+0x20/0x40
[  243.680083] [c000000008167bd0] [c000000000075e60] pci_hp_remove_devices+0x90/0x130
[  243.680089] [c000000008167c60] [c000000000053cd8] eeh_handle_normal_event+0x7b8/0xa80
[  243.680095] [c000000008167d40] [c000000000054de8] eeh_event_handler+0x108/0x190
[  243.680100] [c000000008167da0] [c00000000018151c] kthread+0x18c/0x1a0
[  243.680105] [c000000008167e10] [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[  243.680117] INFO: task systemd-udevd:476 blocked for more than 122 seconds.
[  243.680120]       Not tainted 5.15.0-rc1-autotest #5
[  243.680123] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  243.680126] task:systemd-udevd   state:D stack:    0 pid:  476 ppid:   455 flags:0x00040402
[  243.680131] Call Trace:
[  243.680133] [c00000000c4df620] [c0000000001c350c] __wake_up_common_lock+0xac/0x100 (unreliable)
[  243.680139] [c00000000c4df810] [c00000000001e6c8] __switch_to+0x278/0x490
[  243.680144] [c00000000c4df870] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  243.680148] [c00000000c4df930] [c000000000c8c128] schedule+0x68/0x120
[  243.680152] [c00000000c4df960] [c000000000186cd4] async_synchronize_cookie_domain+0x144/0x1a0
[  243.680158] [c00000000c4dfa10] [c00000000024f2e4] do_init_module+0x1a4/0x320
[  243.680164] [c00000000c4dfa90] [c000000000252b88] load_module+0x2098/0x22f0
[  243.680169] [c00000000c4dfc90] [c000000000253100] __do_sys_finit_module+0xd0/0x150
[  243.680174] [c00000000c4dfdb0] [c00000000002ffe8] system_call_exception+0x158/0x2c0
[  243.680178] [c00000000c4dfe10] [c00000000000c764] system_call_common+0xf4/0x258
[  243.680184] --- interrupt: c00 at 0x7fff86a5a504
[  243.680188] NIP:  00007fff86a5a504 LR: 00007fff86caea9c CTR: 0000000000000000
[  243.680192] REGS: c00000000c4dfe80 TRAP: 0c00   Not tainted  (5.15.0-rc1-autotest)
[  243.680196] MSR:  800000000000d033 <SF,EE,PR,ME,IR,DR,RI,LE>  CR: 28222244  XER: 00000000
[  243.680204] IRQMASK: 0 
[  243.680204] GPR00: 0000000000000161 00007ffff1dcdfd0 00007fff86b47100 000000000000000e 
[  243.680204] GPR04: 0000000136b9b720 0000000000000000 000000000000000e 0000000000000000 
[  243.680204] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[  243.680204] GPR12: 0000000000000000 00007fff856784a0 0000000000020000 00000001109cd5a0 
[  243.680204] GPR16: 0000000110a10130 00000001109d57d8 0000000000000000 0000000000000000 
[  243.680204] GPR20: 0000000000000000 0000000000000000 0000000136b70de0 0000000000000008 
[  243.680204] GPR24: 0000000000000007 0000000000000000 0000000000000000 0000000136b94860 
[  243.680204] GPR28: 0000000136b9b720 0000000000020000 0000000000000000 0000000136b70de0 
[  243.680244] NIP [00007fff86a5a504] 0x7fff86a5a504
[  243.680247] LR [00007fff86caea9c] 0x7fff86caea9c
[  243.680250] --- interrupt: c00
[  243.680252] INFO: task systemd-udevd:478 blocked for more than 122 seconds.
[  243.680255]       Not tainted 5.15.0-rc1-autotest #5
[  243.680258] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  243.680261] task:systemd-udevd   state:D stack:    0 pid:  478 ppid:   455 flags:0x00040402
[  243.680266] Call Trace:
[  243.680268] [c00000000c4e7620] [c000000000000000] start_first_256B+0x0/0x38 (unreliable)
[  243.680273] [c00000000c4e7810] [c00000000001e6c8] __switch_to+0x278/0x490
[  243.680278] [c00000000c4e7870] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  243.680282] [c00000000c4e7930] [c000000000c8c128] schedule+0x68/0x120
[  243.680287] [c00000000c4e7960] [c000000000186cd4] async_synchronize_cookie_domain+0x144/0x1a0
[  243.680292] [c00000000c4e7a10] [c00000000024f2e4] do_init_module+0x1a4/0x320
[  243.680297] [c00000000c4e7a90] [c000000000252b88] load_module+0x2098/0x22f0
[  243.680302] [c00000000c4e7c90] [c000000000253100] __do_sys_finit_module+0xd0/0x150
[  243.680307] [c00000000c4e7db0] [c00000000002ffe8] system_call_exception+0x158/0x2c0
[  243.680311] [c00000000c4e7e10] [c00000000000c764] system_call_common+0xf4/0x258
[  243.680317] --- interrupt: c00 at 0x7fff86a5a504
[  243.680320] NIP:  00007fff86a5a504 LR: 00007fff86caea9c CTR: 0000000000000000
[  243.680323] REGS: c00000000c4e7e80 TRAP: 0c00   Not tainted  (5.15.0-rc1-autotest)
[  243.680327] MSR:  800000000000d033 <SF,EE,PR,ME,IR,DR,RI,LE>  CR: 28222244  XER: 00000000
[  243.680335] IRQMASK: 0 
[  243.680335] GPR00: 0000000000000161 00007ffff1dcdfd0 00007fff86b47100 000000000000000e 
[  243.680335] GPR04: 00007fff86cbad30 0000000000000000 000000000000000e 0000000000000000 
[  243.680335] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[  243.680335] GPR12: 0000000000000000 00007fff856784a0 0000000000020000 00000001109cd5a0 
[  243.680335] GPR16: 0000000110a10130 00000001109d57d8 0000000000000000 0000000000000000 
[  243.680335] GPR20: 0000000000000000 00000001109d5810 0000000136b926b0 00007ffff1dce1f0 
[  243.680335] GPR24: 0000000110a10150 0000000000020000 0000000000000000 0000000136b730e0 
[  243.680335] GPR28: 00007fff86cbad30 0000000000020000 0000000000000000 0000000136b926b0 
[  243.680374] NIP [00007fff86a5a504] 0x7fff86a5a504
[  243.680377] LR [00007fff86caea9c] 0x7fff86caea9c
[  243.680380] --- interrupt: c00
[  366.559988] INFO: task eehd:137 blocked for more than 245 seconds.
[  366.560004]       Not tainted 5.15.0-rc1-autotest #5
[  366.560007] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  366.560012] task:eehd            state:D stack:    0 pid:  137 ppid:     2 flags:0x00000800
[  366.560018] Call Trace:
[  366.560020] [c000000008167650] [c0000000001dd368] info_print_prefix+0xa8/0x130 (unreliable)
[  366.560029] [c000000008167840] [c00000000001e6c8] __switch_to+0x278/0x490
[  366.560034] [c0000000081678a0] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  366.560040] [c000000008167960] [c000000000c8c128] schedule+0x68/0x120
[  366.560045] [c000000008167990] [c0080000008ca3c4] __ipr_remove+0x10c/0x4b0 [ipr]
[  366.560054] [c000000008167a60] [c0080000008ca828] ipr_remove+0xc0/0x110 [ipr]
[  366.560061] [c000000008167ae0] [c000000000732bd4] pci_device_remove+0x64/0x120
[  366.560067] [c000000008167b20] [c000000000811318] device_release_driver_internal+0x168/0x2c0
[  366.560074] [c000000008167b60] [c000000000724618] pci_stop_bus_device+0xa8/0x100
[  366.560079] [c000000008167ba0] [c000000000724810] pci_stop_and_remove_bus_device+0x20/0x40
[  366.560084] [c000000008167bd0] [c000000000075e60] pci_hp_remove_devices+0x90/0x130
[  366.560090] [c000000008167c60] [c000000000053cd8] eeh_handle_normal_event+0x7b8/0xa80
[  366.560095] [c000000008167d40] [c000000000054de8] eeh_event_handler+0x108/0x190
[  366.560101] [c000000008167da0] [c00000000018151c] kthread+0x18c/0x1a0
[  366.560105] [c000000008167e10] [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[  366.560112] INFO: task kworker/u48:2:207 blocked for more than 122 seconds.
[  366.560115]       Not tainted 5.15.0-rc1-autotest #5
[  366.560118] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  366.560121] task:kworker/u48:2   state:D stack:    0 pid:  207 ppid:     2 flags:0x00000800
[  366.560127] Workqueue: events_unbound async_run_entry_fn
[  366.560131] Call Trace:
[  366.560133] [c000000008bd36c0] [c000000008bd3700] 0xc000000008bd3700 (unreliable)
[  366.560137] [c000000008bd38b0] [c00000000001e6c8] __switch_to+0x278/0x490
[  366.560142] [c000000008bd3910] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  366.560147] [c000000008bd39d0] [c000000000c8c128] schedule+0x68/0x120
[  366.560151] [c000000008bd3a00] [c000000000c93ee8] schedule_timeout+0x348/0x3f0
[  366.560156] [c000000008bd3ae0] [c000000000c8e2cc] wait_for_completion+0xbc/0x270
[  366.560161] [c000000008bd3b50] [c000000000872670] do_scan_async+0x60/0x2b0
[  366.560167] [c000000008bd3bd0] [c000000000186a2c] async_run_entry_fn+0x5c/0x1c0
[  366.560171] [c000000008bd3c60] [c000000000175468] process_one_work+0x288/0x570
[  366.560177] [c000000008bd3d00] [c0000000001757c8] worker_thread+0x78/0x660
[  366.560182] [c000000008bd3da0] [c00000000018151c] kthread+0x18c/0x1a0
[  366.560187] [c000000008bd3e10] [c00000000000cfd4] ret_from_kernel_thread+0x5c/0x64
[  366.560196] INFO: task systemd-udevd:476 blocked for more than 245 seconds.
[  366.560199]       Not tainted 5.15.0-rc1-autotest #5
[  366.560202] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  366.560205] task:systemd-udevd   state:D stack:    0 pid:  476 ppid:   455 flags:0x00040402
[  366.560210] Call Trace:
[  366.560212] [c00000000c4df620] [c0000000001c350c] __wake_up_common_lock+0xac/0x100 (unreliable)
[  366.560218] [c00000000c4df810] [c00000000001e6c8] __switch_to+0x278/0x490
[  366.560224] [c00000000c4df870] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  366.560228] [c00000000c4df930] [c000000000c8c128] schedule+0x68/0x120
[  366.560232] [c00000000c4df960] [c000000000186cd4] async_synchronize_cookie_domain+0x144/0x1a0
[  366.560238] [c00000000c4dfa10] [c00000000024f2e4] do_init_module+0x1a4/0x320
[  366.560243] [c00000000c4dfa90] [c000000000252b88] load_module+0x2098/0x22f0
[  366.560249] [c00000000c4dfc90] [c000000000253100] __do_sys_finit_module+0xd0/0x150
[  366.560254] [c00000000c4dfdb0] [c00000000002ffe8] system_call_exception+0x158/0x2c0
[  366.560258] [c00000000c4dfe10] [c00000000000c764] system_call_common+0xf4/0x258
[  366.560264] --- interrupt: c00 at 0x7fff86a5a504
[  366.560268] NIP:  00007fff86a5a504 LR: 00007fff86caea9c CTR: 0000000000000000
[  366.560271] REGS: c00000000c4dfe80 TRAP: 0c00   Not tainted  (5.15.0-rc1-autotest)
[  366.560275] MSR:  800000000000d033 <SF,EE,PR,ME,IR,DR,RI,LE>  CR: 28222244  XER: 00000000
[  366.560284] IRQMASK: 0 
[  366.560284] GPR00: 0000000000000161 00007ffff1dcdfd0 00007fff86b47100 000000000000000e 
[  366.560284] GPR04: 0000000136b9b720 0000000000000000 000000000000000e 0000000000000000 
[  366.560284] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[  366.560284] GPR12: 0000000000000000 00007fff856784a0 0000000000020000 00000001109cd5a0 
[  366.560284] GPR16: 0000000110a10130 00000001109d57d8 0000000000000000 0000000000000000 
[  366.560284] GPR20: 0000000000000000 0000000000000000 0000000136b70de0 0000000000000008 
[  366.560284] GPR24: 0000000000000007 0000000000000000 0000000000000000 0000000136b94860 
[  366.560284] GPR28: 0000000136b9b720 0000000000020000 0000000000000000 0000000136b70de0 
[  366.560324] NIP [00007fff86a5a504] 0x7fff86a5a504
[  366.560327] LR [00007fff86caea9c] 0x7fff86caea9c
[  366.560329] --- interrupt: c00
[  366.560332] INFO: task systemd-udevd:478 blocked for more than 245 seconds.
[  366.560335]       Not tainted 5.15.0-rc1-autotest #5
[  366.560338] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  366.560341] task:systemd-udevd   state:D stack:    0 pid:  478 ppid:   455 flags:0x00040402
[  366.560345] Call Trace:
[  366.560347] [c00000000c4e7620] [c000000000000000] start_first_256B+0x0/0x38 (unreliable)
[  366.560353] [c00000000c4e7810] [c00000000001e6c8] __switch_to+0x278/0x490
[  366.560357] [c00000000c4e7870] [c000000000c8b9bc] __schedule+0x31c/0xa20
[  366.560362] [c00000000c4e7930] [c000000000c8c128] schedule+0x68/0x120
[  366.560366] [c00000000c4e7960] [c000000000186cd4] async_synchronize_cookie_domain+0x144/0x1a0
[  366.560372] [c00000000c4e7a10] [c00000000024f2e4] do_init_module+0x1a4/0x320
[  366.560376] [c00000000c4e7a90] [c000000000252b88] load_module+0x2098/0x22f0
[  366.560381] [c00000000c4e7c90] [c000000000253100] __do_sys_finit_module+0xd0/0x150
[  366.560386] [c00000000c4e7db0] [c00000000002ffe8] system_call_exception+0x158/0x2c0
[  366.560391] [c00000000c4e7e10] [c00000000000c764] system_call_common+0xf4/0x258
[  366.560396] --- interrupt: c00 at 0x7fff86a5a504
[  366.560399] NIP:  00007fff86a5a504 LR: 00007fff86caea9c CTR: 0000000000000000
[  366.560403] REGS: c00000000c4e7e80 TRAP: 0c00   Not tainted  (5.15.0-rc1-autotest)
[  366.560407] MSR:  800000000000d033 <SF,EE,PR,ME,IR,DR,RI,LE>  CR: 28222244  XER: 00000000
[  366.560414] IRQMASK: 0 
[  366.560414] GPR00: 0000000000000161 00007ffff1dcdfd0 00007fff86b47100 000000000000000e 
[  366.560414] GPR04: 00007fff86cbad30 0000000000000000 000000000000000e 0000000000000000 
[  366.560414] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
[  366.560414] GPR12: 0000000000000000 00007fff856784a0 0000000000020000 00000001109cd5a0 
[  366.560414] GPR16: 0000000110a10130 00000001109d57d8 0000000000000000 0000000000000000 
[  366.560414] GPR20: 0000000000000000 00000001109d5810 0000000136b926b0 00007ffff1dce1f0 
[  366.560414] GPR24: 0000000110a10150 0000000000020000 0000000000000000 0000000136b730e0 
[  366.560414] GPR28: 00007fff86cbad30 0000000000020000 0000000000000000 0000000136b926b0 
[  366.560453] NIP [00007fff86a5a504] 0x7fff86a5a504
[  366.560456] LR [00007fff86caea9c] 0x7fff86caea9c
[  366.560459] --- interrupt: c00



[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [SCSI Target Devel]     [Linux SCSI Target Infrastructure]     [Kernel Newbies]     [IDE]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux ATA RAID]     [Linux IIO]     [Samba]     [Device Mapper]

  Powered by Linux