On Tue, Jul 11, 2023 at 10:13:31AM +1000, Chris Dunlop wrote:
On Tue, Jul 11, 2023 at 07:53:54AM +1000, Chris Dunlop wrote:
Hi,
This box is newly booted into linux v6.1.35 (2 days ago), it was
previously running v5.15.118 without any problems (other than that
fixed by "5e672cd69f0a xfs: non-blocking inodegc pushes", the reason
for the upgrade).
I have rm operations on two files that have been stuck for in excess
of 22 hours and 18 hours respectively:
...
...subsequent to starting writing all this down I have another two
sets of rms stuck, again on unremarkable files, and on two more
separate filesystems.
...oh. And an 'ls' on those files is hanging. The reboot has become
more urgent.
FYI, it's not 'ls' that's hanging, it's bash, because I used a
wildcard on the command line. The bash stack:
$ cat /proc/24779/stack
[<0>] iterate_dir+0x3e/0x180
[<0>] __x64_sys_getdents64+0x71/0x100
[<0>] do_syscall_64+0x34/0x80
[<0>] entry_SYSCALL_64_after_hwframe+0x46/0xb0
'lsof' shows me it's trying to read one of the directories holding the
file that one of the newer hanging "rm"s is trying to remove.
Ugh. It wasn't just the "rm"s and bash hanging (and as it turns out,
xfs_logprint), they were just obvious because that's what I was looking
at. It turns out there was a whole lot more hanging.
Full sysrq-w output at:
https://file.io/tg7F5OqIWo1B
Sorry if there's more I could be looking at, but I've gotta reboot this
thing NOW...
Cheers,
Chris
