On Sun, 2018-03-04 at 20:01 +0100, Jean-Louis Dupond wrote:
> I'm running indeed CentOS 6 with the Virt SIG kernels. Already updated
> to 4.9.75, but recently hit the problem again.
>
> The first PID that was in D-state (root 27157 0.0 0.0 127664 5196
> ? D 06:19 0:00 \_ vgdisplay -c --ignorelockingfailure), had
> the following stack:
> # cat /proc/27157/stack
> [<ffffffff813fc62f>] blk_mq_freeze_queue_wait+0x6f/0xd0
> [<ffffffff813fe5be>] blk_freeze_queue+0x1e/0x30
> [<ffffffff813fe5de>] blk_mq_freeze_queue+0xe/0x10
> [<ffffffff815fa46e>] loop_switch+0x1e/0xd0
> [<ffffffff815fb2ba>] lo_release+0x7a/0x80
> [<ffffffff812a75f7>] __blkdev_put+0x1a7/0x200
> [<ffffffff812a76a6>] blkdev_put+0x56/0x140
> [<ffffffff812a77b4>] blkdev_close+0x24/0x30
> [<ffffffff8126b7b8>] __fput+0xc8/0x240
> [<ffffffff8126b9de>] ____fput+0xe/0x10
> [<ffffffff810c4ab8>] task_work_run+0x68/0xa0
> [<ffffffff81003546>] exit_to_usermode_loop+0xc6/0xd0
> [<ffffffff81003f85>] do_syscall_64+0x185/0x240
> [<ffffffff818df3aa>] entry_SYSCALL64_slow_path+0x25/0x25
> [<ffffffffffffffff>] 0xffffffffffffffff
>
> Other procs show the following:
> # cat /proc/7803/stack
> [<ffffffff812a782c>] __blkdev_get+0x6c/0x3f0
> [<ffffffff812a7dfc>] blkdev_get+0x5c/0x1c0
> [<ffffffff812a8342>] blkdev_open+0x62/0x80
> [<ffffffff812669aa>] do_dentry_open+0x22a/0x340
> [<ffffffff81266b11>] vfs_open+0x51/0x80
> [<ffffffff81279fe5>] do_last+0x435/0x7a0
> [<ffffffff8127a3d7>] path_openat+0x87/0x1c0
> [<ffffffff8127a595>] do_filp_open+0x85/0xe0
> [<ffffffff812681ec>] do_sys_open+0x11c/0x210
> [<ffffffff8126831e>] SyS_open+0x1e/0x20
> [<ffffffff81003e7a>] do_syscall_64+0x7a/0x240
> [<ffffffff818df3aa>] entry_SYSCALL64_slow_path+0x25/0x25
> [<ffffffffffffffff>] 0xffffffffffffffff
>
> An strace hangs again on loop0 open:
> stat("/dev/loop0", {st_mode=S_IFBLK|0660, st_rdev=makedev(7, 0), ...}) =
> 0
> open("/dev/loop0", O_RDONLY|O_DIRECT|O_NOATIME
>
> And it seems like indeed alot is hanging on loop0:
> # cat /sys/block/loop0/mq/0/queued
> 5957
Hello Jean-Louis,
Is the system still in this state? If so, can you provide the output of the
following command (as an attachment):
find /sys/kernel/debug/block/ -type f \! \( -name poll_stat -o -name dispatched -o -name merged -o -name completed \)
Thanks,
Bart.
