https://bugzilla.kernel.org/show_bug.cgi?id=199435 --- Comment #21 from Anthony Hausman (anthonyhaussmann@xxxxxxxxx) --- I have reproduced the problem. Here the condition that I have done: Kernel: 4.16.3-041603-generic hpsa: 3.4.20-125 with patch to use local work-queue instead of system work-queue. I needed to execute a badblocks in a read-only test on a disk who has failed before: ~# while :; do badblocks -v -b 4096 -s /dev/sdt; done And several days after, the bug raised. You'll find a graph of the load in an attachment. Before the reset, I have a hpsa_update_device_info: inquiry failed and a stack trace on badblocks (this one seems to be logical) Load: 850 [Tue May 1 06:27:37 2018] hpsa 0000:08:00.0: aborted: LUN:000000c000003901 CDB:12000000310000000000000000000000 [Tue May 1 06:27:37 2018] hpsa 0000:08:00.0: hpsa_update_device_info: inquiry failed, device will be skipped. [Tue May 1 06:27:37 2018] hpsa 0000:08:00.0: scsi 0:0:50:0: removed Direct-Access ATA MB4000GCWDC PHYS DRV SSDSmartPathCap- En- Exp=0 [Tue May 1 06:28:24 2018] hpsa 0000:08:00.0: aborted: LUN:000000c000003901 CDB:12000000310000000000000000000000 [Tue May 1 06:28:24 2018] hpsa 0000:08:00.0: hpsa_update_device_info: inquiry failed, device will be skipped. [Tue May 1 06:29:51 2018] INFO: task badblocks:46824 blocked for more than 120 seconds. [Tue May 1 06:29:51 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:29:51 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:29:51 2018] badblocks D 0 46824 48728 0x00000004 [Tue May 1 06:29:51 2018] Call Trace: [Tue May 1 06:29:51 2018] __schedule+0x297/0x880 [Tue May 1 06:29:51 2018] ? iov_iter_get_pages+0xc0/0x2c0 [Tue May 1 06:29:51 2018] schedule+0x2c/0x80 [Tue May 1 06:29:51 2018] io_schedule+0x16/0x40 [Tue May 1 06:29:51 2018] __blkdev_direct_IO_simple+0x1ff/0x360 [Tue May 1 06:29:51 2018] ? bdget+0x120/0x120 [Tue May 1 06:29:51 2018] blkdev_direct_IO+0x3a2/0x3f0 [Tue May 1 06:29:51 2018] ? blkdev_direct_IO+0x3a2/0x3f0 [Tue May 1 06:29:51 2018] ? current_time+0x32/0x70 [Tue May 1 06:29:51 2018] ? __atime_needs_update+0x7f/0x190 [Tue May 1 06:29:51 2018] generic_file_read_iter+0xc6/0xc10 [Tue May 1 06:29:51 2018] ? __blkdev_direct_IO_simple+0x360/0x360 [Tue May 1 06:29:51 2018] ? generic_file_read_iter+0xc6/0xc10 [Tue May 1 06:29:51 2018] ? __wake_up+0x13/0x20 [Tue May 1 06:29:51 2018] ? tty_ldisc_deref+0x16/0x20 [Tue May 1 06:29:51 2018] ? tty_write+0x1fb/0x320 [Tue May 1 06:29:51 2018] blkdev_read_iter+0x35/0x40 [Tue May 1 06:29:51 2018] __vfs_read+0xfb/0x170 [Tue May 1 06:29:51 2018] vfs_read+0x8e/0x130 [Tue May 1 06:29:51 2018] SyS_read+0x55/0xc0 [Tue May 1 06:29:51 2018] do_syscall_64+0x73/0x130 [Tue May 1 06:29:51 2018] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [Tue May 1 06:29:51 2018] RIP: 0033:0x7fe31b97c330 [Tue May 1 06:29:51 2018] RSP: 002b:00007fffcea10258 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 [Tue May 1 06:29:51 2018] RAX: ffffffffffffffda RBX: 0000026e19800000 RCX: 00007fe31b97c330 [Tue May 1 06:29:51 2018] RDX: 0000000000040000 RSI: 00007fe31c26e000 RDI: 0000000000000003 [Tue May 1 06:29:51 2018] RBP: 0000000000001000 R08: 0000000026e19800 R09: 00007fffcea10008 [Tue May 1 06:29:51 2018] R10: 00007fffcea10020 R11: 0000000000000246 R12: 0000000000000003 [Tue May 1 06:29:51 2018] R13: 00007fe31c26e000 R14: 0000000000000040 R15: 0000000000040000 [Tue May 1 06:31:52 2018] INFO: task badblocks:46824 blocked for more than 120 seconds. [Tue May 1 06:31:52 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:31:52 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:31:52 2018] badblocks D 0 46824 48728 0x00000004 [Tue May 1 06:31:52 2018] Call Trace: [Tue May 1 06:31:52 2018] __schedule+0x297/0x880 [Tue May 1 06:31:52 2018] ? iov_iter_get_pages+0xc0/0x2c0 [Tue May 1 06:31:52 2018] schedule+0x2c/0x80 [Tue May 1 06:31:52 2018] io_schedule+0x16/0x40 [Tue May 1 06:31:52 2018] __blkdev_direct_IO_simple+0x1ff/0x360 [Tue May 1 06:31:52 2018] ? bdget+0x120/0x120 [Tue May 1 06:31:52 2018] blkdev_direct_IO+0x3a2/0x3f0 [Tue May 1 06:31:52 2018] ? blkdev_direct_IO+0x3a2/0x3f0 [Tue May 1 06:31:52 2018] ? current_time+0x32/0x70 [Tue May 1 06:31:52 2018] ? __atime_needs_update+0x7f/0x190 [Tue May 1 06:31:52 2018] generic_file_read_iter+0xc6/0xc10 [Tue May 1 06:31:52 2018] ? __blkdev_direct_IO_simple+0x360/0x360 [Tue May 1 06:31:52 2018] ? generic_file_read_iter+0xc6/0xc10 [Tue May 1 06:31:52 2018] ? __wake_up+0x13/0x20 [Tue May 1 06:31:52 2018] ? tty_ldisc_deref+0x16/0x20 [Tue May 1 06:31:52 2018] ? tty_write+0x1fb/0x320 [Tue May 1 06:31:52 2018] blkdev_read_iter+0x35/0x40 [Tue May 1 06:31:52 2018] __vfs_read+0xfb/0x170 [Tue May 1 06:31:52 2018] vfs_read+0x8e/0x130 [Tue May 1 06:31:52 2018] SyS_read+0x55/0xc0 [Tue May 1 06:31:52 2018] do_syscall_64+0x73/0x130 [Tue May 1 06:31:52 2018] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [Tue May 1 06:31:52 2018] RIP: 0033:0x7fe31b97c330 [Tue May 1 06:31:52 2018] RSP: 002b:00007fffcea10258 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 [Tue May 1 06:31:52 2018] RAX: ffffffffffffffda RBX: 0000026e19800000 RCX: 00007fe31b97c330 [Tue May 1 06:31:52 2018] RDX: 0000000000040000 RSI: 00007fe31c26e000 RDI: 0000000000000003 [Tue May 1 06:31:52 2018] RBP: 0000000000001000 R08: 0000000026e19800 R09: 00007fffcea10008 [Tue May 1 06:31:52 2018] R10: 00007fffcea10020 R11: 0000000000000246 R12: 0000000000000003 [Tue May 1 06:31:52 2018] R13: 00007fe31c26e000 R14: 0000000000000040 R15: 0000000000040000 [Tue May 1 06:32:55 2018] hpsa 0000:08:00.0: scsi 0:1:0:19: resetting logical Direct-Access HP LOGICAL VOLUME RAID-0 SSDSmartPathCap- E n- Exp=1 I have done a ps like you said before this time, every 30 seconds: ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 1 TS /sbin/init 0.0 3680 101792 0 0 0.0 poll_schedule_timeout init 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 9 TS [rcu_sched] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_sched 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2034 TS /usr/sbin/syslog-ng --process-mode=background -f / 0.0 56936 637344 0 0 1.3 ep_poll syslog-ng 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2333 TS cat filer-01-24-1.keys 0.0 324 4384 0 0 0.0 pipe_wait cat 0 2334 TS /usr/bin/python /usr/local/scality-walker/scality- 0.0 11208 131444 0 0 1.2 wait_woken scality-walker. 0 2740 TS /opt/datadog-agent/embedded/bin/python /opt/datado 0.0 42160 289140 0 0 0.6 poll_schedule_timeout python ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2333 TS cat filer-01-24-1.keys 0.0 324 4384 0 0 0.0 pipe_wait cat 0 2334 TS /usr/bin/python /usr/local/scality-walker/scality- 0.0 11208 131444 0 0 1.2 wait_woken scality-walker. 0 2740 TS /opt/datadog-agent/embedded/bin/python /opt/datado 0.0 42160 289140 0 0 0.6 poll_schedule_timeout python 1 minute later, I had a task trace about cmaeventd (logical) and jbd2 tasks: Load: 2000 [Tue May 1 06:33:53 2018] INFO: task cmaeventd:3405 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] cmaeventd D 0 3405 1 0x00000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] scsi_block_when_processing_errors+0xd4/0x110 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] sg_open+0x14c/0x5d0 [Tue May 1 06:33:53 2018] chrdev_open+0xc4/0x1b0 [Tue May 1 06:33:53 2018] do_dentry_open+0x1c2/0x310 [Tue May 1 06:33:53 2018] ? cdev_put.part.3+0x20/0x20 [Tue May 1 06:33:53 2018] vfs_open+0x4f/0x80 [Tue May 1 06:33:53 2018] path_openat+0x66e/0x1770 [Tue May 1 06:33:53 2018] ? unlazy_walk+0x3b/0xb0 [Tue May 1 06:33:53 2018] ? terminate_walk+0x8e/0xf0 [Tue May 1 06:33:53 2018] do_filp_open+0x9b/0x110 [Tue May 1 06:33:53 2018] ? __check_object_size+0xac/0x1a0 [Tue May 1 06:33:53 2018] ? __check_object_size+0xac/0x1a0 [Tue May 1 06:33:53 2018] ? __alloc_fd+0x46/0x170 [Tue May 1 06:33:53 2018] do_sys_open+0x1ba/0x250 [Tue May 1 06:33:53 2018] ? do_sys_open+0x1ba/0x250 [Tue May 1 06:33:53 2018] ? SyS_access+0x13d/0x230 [Tue May 1 06:33:53 2018] SyS_open+0x1e/0x20 [Tue May 1 06:33:53 2018] do_syscall_64+0x73/0x130 [Tue May 1 06:33:53 2018] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [Tue May 1 06:33:53 2018] RIP: 0033:0x7fdfbc6b0be0 [Tue May 1 06:33:53 2018] RSP: 002b:00007ffe1f418728 EFLAGS: 00000246 ORIG_RAX: 0000000000000002 [Tue May 1 06:33:53 2018] RAX: ffffffffffffffda RBX: 00000000018b8640 RCX: 00007fdfbc6b0be0 [Tue May 1 06:33:53 2018] RDX: 0000000000000008 RSI: 0000000000000002 RDI: 00007ffe1f418760 [Tue May 1 06:33:53 2018] RBP: 00007ffe1f418760 R08: 0000000000000001 R09: 0000000000000000 [Tue May 1 06:33:53 2018] R10: 00007fdfbc699760 R11: 0000000000000246 R12: 0000000000000002 [Tue May 1 06:33:53 2018] R13: 0000000000000001 R14: 00007ffe1f418870 R15: 00007ffe1f4189a0 [Tue May 1 06:33:53 2018] INFO: task cmaidad:3507 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] cmaidad D 0 3507 1 0x00000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? __find_get_block+0xb6/0x2f0 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] scsi_block_when_processing_errors+0xd4/0x110 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] sg_open+0x14c/0x5d0 [Tue May 1 06:33:53 2018] chrdev_open+0xc4/0x1b0 [Tue May 1 06:33:53 2018] do_dentry_open+0x1c2/0x310 [Tue May 1 06:33:53 2018] ? cdev_put.part.3+0x20/0x20 [Tue May 1 06:33:53 2018] vfs_open+0x4f/0x80 [Tue May 1 06:33:53 2018] path_openat+0x66e/0x1770 [Tue May 1 06:33:53 2018] ? unlazy_walk+0x3b/0xb0 [Tue May 1 06:33:53 2018] ? terminate_walk+0x8e/0xf0 [Tue May 1 06:33:53 2018] do_filp_open+0x9b/0x110 [Tue May 1 06:33:53 2018] ? __check_object_size+0xac/0x1a0 [Tue May 1 06:33:53 2018] ? __check_object_size+0xac/0x1a0 [Tue May 1 06:33:53 2018] ? __alloc_fd+0x46/0x170 [Tue May 1 06:33:53 2018] do_sys_open+0x1ba/0x250 [Tue May 1 06:33:53 2018] ? do_sys_open+0x1ba/0x250 [Tue May 1 06:33:53 2018] ? SyS_access+0x13d/0x230 [Tue May 1 06:33:53 2018] SyS_open+0x1e/0x20 [Tue May 1 06:33:53 2018] do_syscall_64+0x73/0x130 [Tue May 1 06:33:53 2018] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [Tue May 1 06:33:53 2018] RIP: 0033:0x7f5dec322be0 [Tue May 1 06:33:53 2018] RSP: 002b:00007ffee82dccc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000002 [Tue May 1 06:33:53 2018] RAX: ffffffffffffffda RBX: 00000000021c4c60 RCX: 00007f5dec322be0 [Tue May 1 06:33:53 2018] RDX: 0000000000000008 RSI: 0000000000000002 RDI: 00007ffee82dcd00 [Tue May 1 06:33:53 2018] RBP: 00007ffee82dcd00 R08: 0000000000000001 R09: 0000000000000003 [Tue May 1 06:33:53 2018] R10: 00007f5dec30b760 R11: 0000000000000246 R12: 0000000000000002 [Tue May 1 06:33:53 2018] R13: 0000000000000001 R14: 00007ffee82dce10 R15: 00007ffee82dcf40 [Tue May 1 06:33:53 2018] INFO: task jbd2/sdas-8:9924 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] jbd2/sdas-8 D 0 9924 2 0x80000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] jbd2_journal_commit_transaction+0x244/0x1740 [Tue May 1 06:33:53 2018] ? update_curr+0xf5/0x1d0 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] ? lock_timer_base+0x6b/0x90 [Tue May 1 06:33:53 2018] kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] kthread+0x121/0x140 [Tue May 1 06:33:53 2018] ? commit_timeout+0x20/0x20 [Tue May 1 06:33:53 2018] ? kthread_create_worker_on_cpu+0x70/0x70 [Tue May 1 06:33:53 2018] ret_from_fork+0x35/0x40 [Tue May 1 06:33:53 2018] INFO: task jbd2/sdan-8:9955 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] jbd2/sdan-8 D 0 9955 2 0x80000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] jbd2_journal_commit_transaction+0x244/0x1740 [Tue May 1 06:33:53 2018] ? update_curr+0xf5/0x1d0 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] ? lock_timer_base+0x6b/0x90 [Tue May 1 06:33:53 2018] kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] kthread+0x121/0x140 [Tue May 1 06:33:53 2018] ? commit_timeout+0x20/0x20 [Tue May 1 06:33:53 2018] ? kthread_create_worker_on_cpu+0x70/0x70 [Tue May 1 06:33:53 2018] ? do_syscall_64+0x73/0x130 [Tue May 1 06:33:53 2018] ? SyS_exit_group+0x14/0x20 [Tue May 1 06:33:53 2018] ret_from_fork+0x35/0x40 [Tue May 1 06:33:53 2018] INFO: task jbd2/sdaq-8:9965 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] jbd2/sdaq-8 D 0 9965 2 0x80000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] jbd2_journal_commit_transaction+0x244/0x1740 [Tue May 1 06:33:53 2018] ? update_curr+0xf5/0x1d0 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] ? lock_timer_base+0x6b/0x90 [Tue May 1 06:33:53 2018] kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] kthread+0x121/0x140 [Tue May 1 06:33:53 2018] ? commit_timeout+0x20/0x20 [Tue May 1 06:33:53 2018] ? kthread_create_worker_on_cpu+0x70/0x70 [Tue May 1 06:33:53 2018] ret_from_fork+0x35/0x40 [Tue May 1 06:33:53 2018] INFO: task jbd2/sdaj-8:10082 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] jbd2/sdaj-8 D 0 10082 2 0x80000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] jbd2_journal_commit_transaction+0x244/0x1740 [Tue May 1 06:33:53 2018] ? update_curr+0xf5/0x1d0 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] ? lock_timer_base+0x6b/0x90 [Tue May 1 06:33:53 2018] kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] kthread+0x121/0x140 [Tue May 1 06:33:53 2018] ? commit_timeout+0x20/0x20 [Tue May 1 06:33:53 2018] ? kthread_create_worker_on_cpu+0x70/0x70 [Tue May 1 06:33:53 2018] ? do_syscall_64+0x73/0x130 [Tue May 1 06:33:53 2018] ? SyS_exit_group+0x14/0x20 [Tue May 1 06:33:53 2018] ret_from_fork+0x35/0x40 [Tue May 1 06:33:53 2018] INFO: task jbd2/sdao-8:10109 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] jbd2/sdao-8 D 0 10109 2 0x80000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? bit_wait+0x60/0x60 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] io_schedule+0x16/0x40 [Tue May 1 06:33:53 2018] bit_wait_io+0x11/0x60 [Tue May 1 06:33:53 2018] __wait_on_bit+0x4c/0x90 [Tue May 1 06:33:53 2018] out_of_line_wait_on_bit+0x90/0xb0 [Tue May 1 06:33:53 2018] ? bit_waitqueue+0x40/0x40 [Tue May 1 06:33:53 2018] __wait_on_buffer+0x32/0x40 [Tue May 1 06:33:53 2018] jbd2_journal_commit_transaction+0xf59/0x1740 [Tue May 1 06:33:53 2018] kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] kthread+0x121/0x140 [Tue May 1 06:33:53 2018] ? commit_timeout+0x20/0x20 [Tue May 1 06:33:53 2018] ? kthread_create_worker_on_cpu+0x70/0x70 [Tue May 1 06:33:53 2018] ret_from_fork+0x35/0x40 [Tue May 1 06:33:53 2018] INFO: task jbd2/sdag-8:10135 blocked for more than 120 seconds. [Tue May 1 06:33:53 2018] Tainted: G OE 4.16.3-041603-generic #201804190730 [Tue May 1 06:33:53 2018] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue May 1 06:33:53 2018] jbd2/sdag-8 D 0 10135 2 0x80000000 [Tue May 1 06:33:53 2018] Call Trace: [Tue May 1 06:33:53 2018] __schedule+0x297/0x880 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] schedule+0x2c/0x80 [Tue May 1 06:33:53 2018] jbd2_journal_commit_transaction+0x244/0x1740 [Tue May 1 06:33:53 2018] ? update_curr+0xf5/0x1d0 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] ? lock_timer_base+0x6b/0x90 [Tue May 1 06:33:53 2018] kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? kjournald2+0xc8/0x270 [Tue May 1 06:33:53 2018] ? wait_woken+0x80/0x80 [Tue May 1 06:33:53 2018] kthread+0x121/0x140 [Tue May 1 06:33:53 2018] ? commit_timeout+0x20/0x20 [Tue May 1 06:33:53 2018] ? kthread_create_worker_on_cpu+0x70/0x70 [Tue May 1 06:33:53 2018] ret_from_fork+0x35/0x40 Some other ps after that message: ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2333 TS cat filer-01-24-1.keys 0.0 324 4384 0 0 0.0 pipe_wait cat 0 2427 TS /usr/bin/python /usr/bin/salt-minion KeepAlive Mul 0.0 109868 719424 0 0 0.1 poll_schedule_timeout /usr/bin/python 0 3555 TS cmascsid -p 15 -s OK -l /var/log/hp-snmp-agents/cm 0.0 396 12880 0 0 0.0 msgrcv cmascsid ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 1865 TS /usr/sbin/sshd -D 0.0 836 61392 0 0 0.0 poll_schedule_timeout sshd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2333 TS cat filer-01-24-1.keys 0.0 324 4384 0 0 0.0 pipe_wait cat 0 2399 TS /usr/bin/python /usr/sbin/exabgp /etc/exabgp/exabg 0.0 9552 50888 0 0 0.0 poll_schedule_timeout exabgp ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 436 TS [md0_raid1] 0.0 0 0 0 0 0.0 md_thread md0_raid1 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2333 TS cat filer-01-24-1.keys 0.0 324 4384 0 0 0.0 pipe_wait cat 0 2662 TS asynctask-worker [disable] : 1 0.0 14856 135372 0 0 0.0 poll_schedule_timeout asynctask-worke ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2034 TS /usr/sbin/syslog-ng --process-mode=background -f / 0.0 56936 637344 0 0 1.3 ep_poll syslog-ng 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2333 TS cat filer-01-24-1.keys 0.0 324 4384 0 0 0.0 pipe_wait cat 0 2471 TS python /etc/exabgp/processes/exasrv.py /etc/exabgp 0.0 6316 35120 0 0 0.0 poll_schedule_timeout python ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2471 TS python /etc/exabgp/processes/exasrv.py /etc/exabgp 0.0 6316 35120 0 0 0.0 poll_schedule_timeout python 0 3275 TS cmahealthd -p 30 -s OK -t OK -i -l /var/log/hp-snm 0.0 972 22236 0 0 0.0 msgrcv cmahealthd 0 3487 TS cmasasd -p 15 -s OK -l /var/log/hp-snmp-agents/cma 0.0 388 10820 0 0 0.0 msgrcv cmasasd ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 458 TS [jbd2/md0-8] 0.0 0 0 0 0 0.0 kjournald2 jbd2/md0-8 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 3275 TS cmahealthd -p 30 -s OK -t OK -i -l /var/log/hp-snm 0.0 972 22236 0 0 0.0 msgrcv cmahealthd 0 3487 TS cmasasd -p 15 -s OK -l /var/log/hp-snmp-agents/cma 0.0 388 10820 0 0 0.0 msgrcv cmasasd ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kwor ker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kwor ker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 436 TS [md0_raid1] 0.0 0 0 0 0 0.0 md_thread md0_raid1 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 1462 TS lldpd: monitor 0.0 2160 48672 0 0 0.0 skb_wait_for_more_packets lldpd 0 1865 TS /usr/sbin/sshd -D 0.0 836 61392 0 0 0.0 poll_schedule_timeout sshd 0 2033 TS lldpd: 2 neighbors 0.0 2488 49000 0 0 0.0 ep_poll lldpd 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty Few minutes later before reboot: ps -deo psr,pid,cls,cmd:50,pmem,size,vsz,nice,psr,pcpu,wchan:30,comm:30 | sort -nk1 | head -20 0 3 TS [kworker/0:0] 0.0 0 0 0 0 0.0 worker_thread kworker/0:0 0 4 TS [kworker/0:0H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:0H 0 7 TS [mm_percpu_wq] 0.0 0 0 -20 0 0.0 rescuer_thread mm_percpu_wq 0 8 TS [ksoftirqd/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn ksoftirqd/0 0 10 TS [rcu_bh] 0.0 0 0 0 0 0.0 rcu_gp_kthread rcu_bh 0 11 FF [migration/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn migration/0 0 12 FF [watchdog/0] 0.0 0 0 - 0 0.0 smpboot_thread_fn watchdog/0 0 13 TS [cpuhp/0] 0.0 0 0 0 0 0.0 smpboot_thread_fn cpuhp/0 0 71 TS [kblockd] 0.0 0 0 -20 0 0.0 rescuer_thread kblockd 0 76 FF [watchdogd] 0.0 0 0 - 0 0.0 kthread_worker_fn watchdogd 0 128 TS [nvme-delete-wq] 0.0 0 0 -20 0 0.0 rescuer_thread nvme-delete-wq 0 245 TS [kworker/0:2] 0.0 0 0 0 0 0.0 worker_thread kworker/0:2 0 271 TS [raid5wq] 0.0 0 0 -20 0 0.0 rescuer_thread raid5wq 0 427 TS [kworker/u129:0] 0.0 0 0 0 0 0.0 worker_thread kworker/u129:0 0 477 TS [kworker/0:1H] 0.0 0 0 -20 0 0.0 worker_thread kworker/0:1H 0 2080 TS logger -p daemon.info -t docker_daemon_events 0.0 328 4360 0 0 0.0 pipe_wait logger 0 2248 TS /sbin/getty -8 38400 tty6 0.0 356 15836 0 0 0.0 wait_woken getty 0 2427 TS /usr/bin/python /usr/bin/salt-minion KeepAlive Mul 0.0 109868 719424 0 0 0.1 poll_schedule_timeout /usr/bin/python 0 3326 TS cmasm2d -p 30 -l /var/log/hp-snmp-agents/cma.log 0.0 948 24176 0 0 0.0 msgrcv cmasm2d 0 3364 TS cmaperfd -p 30 -s OK -l /var/log/hp-snmp-agents/cm 0.0 1628 22724 0 0 0.0 msgrcv cmaperfd So here it is, I hope we have now enough thing to track down this weird behavior. If you need some other informations or more thing, I can make a little to script to pass some commands if the reset raise without returns. -- You are receiving this mail because: You are the assignee for the bug.
