Hi, ugh. Sorry, looks like I jumped the gun. Mea culpa. We experienced a hang like this: Apr 05 11:51:27 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 05 11:51:27 kernel: task:xfs-conv/vdc1 state:D stack: 0 pid: 606 ppid: 2 flags:0x00004080 Apr 05 11:51:27 kernel: Workqueue: xfs-conv/vdc1 xfs_end_io [xfs] Apr 05 11:51:27 kernel: Call Trace: Apr 05 11:51:27 kernel: __schedule+0x274/0x870 Apr 05 11:51:27 kernel: schedule+0x46/0xb0 Apr 05 11:51:27 kernel: xlog_grant_head_wait+0xc5/0x1d0 [xfs] Apr 05 11:51:27 kernel: xlog_grant_head_check+0xde/0x100 [xfs] Apr 05 11:51:27 kernel: xfs_log_reserve+0xbe/0x1b0 [xfs] Apr 05 11:51:27 kernel: xfs_trans_reserve+0x143/0x180 [xfs] Apr 05 11:51:27 kernel: xfs_trans_alloc+0xee/0x1a0 [xfs] Apr 05 11:51:27 kernel: xfs_iomap_write_unwritten+0x120/0x2e0 [xfs] Apr 05 11:51:27 kernel: ? record_times+0x15/0x90 Apr 05 11:51:27 kernel: xfs_end_ioend+0xd8/0x140 [xfs] Apr 05 11:51:27 kernel: xfs_end_io+0xb8/0xf0 [xfs] Apr 05 11:51:27 kernel: process_one_work+0x1b6/0x350 Apr 05 11:51:27 kernel: rescuer_thread+0x1d1/0x3a0 Apr 05 11:51:27 kernel: ? worker_thread+0x3e0/0x3e0 Apr 05 11:51:27 kernel: kthread+0x11b/0x140 Apr 05 11:51:27 kernel: ? kthread_associate_blkcg+0xb0/0xb0 Apr 05 11:51:27 kernel: ret_from_fork+0x22/0x30 Which seems to be similar to this: https://bugs.launchpad.net/bugs/1996269 I followed their patchset here: https://review.opendev.org/c/starlingx/kernel/+/864257 And I was under the impression that I picked the right one to ask for backporting, but it seems that was incorrect. I went through the list again and I think the following patches are the ones missing from 5.10: 8182ec00803085354761bbadf0287cad7eac0e2f - https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0035-xfs-drop-submit-side-trans-alloc-for-append-ioends.patch edbf1eb9032b84631031d9b43570e262f3461c24 - https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0036-xfs-open-code-ioend-needs-workqueue-helper.patch 170e31793806ce5e5a9647b6340954536244518e - https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0037-xfs-drop-unused-ioend-private-merge-and-setfilesize-.patch 2fd609b6c90a88630a50fb317473b210759b3873 - https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0038-xfs-drop-unnecessary-setfilesize-helper.patch The first one in the series was applied in 5.10.129 afaict and landed in mainstream in 5.12 already. Cheers, Christian > On 12. Apr 2023, at 16:12, Amir Goldstein <amir73il@xxxxxxxxx> wrote: > > On Wed, Apr 12, 2023 at 5:06 PM Christian Theune <ct@xxxxxxxxxxxxxxx> wrote: >> >> Hi, >> >> afaict this was fixed in 5.13 but hasn’t been backported. I’ve seen one of our VMs running 5.10.169 crash with this. >> >> Anybody willing to backport this? It’s only triggered a single time so far and we are rolling out 5.15 anyways, but maybe this was an oversight … ? > > What do you mean by "crash with this"? > > There is no logic change in the commit mentioned below. > > Thanks, > Amir. > >> >> commit 7adb8f14e134d5f885d47c4ccd620836235f0b7f >> Author: Brian Foster <bfoster@xxxxxxxxxx> >> Date: Fri Apr 9 10:27:55 2021 -0700 >> >> xfs: open code ioend needs workqueue helper >> >> Open code xfs_ioend_needs_workqueue() into the only remaining >> caller. >> >> Signed-off-by: Brian Foster <bfoster@xxxxxxxxxx> >> Reviewed-by: Christoph Hellwig <hch@xxxxxx> >> Reviewed-by: Darrick J. Wong <djwong@xxxxxxxxxx> >> Signed-off-by: Darrick J. Wong <djwong@xxxxxxxxxx> >> >> >> Kind regards, >> Christian >> >> -- >> Christian Theune · ct@xxxxxxxxxxxxxxx · +49 345 219401 0 >> Flying Circus Internet Operations GmbH · https://flyingcircus.io >> Leipziger Str. 70/71 · 06108 Halle (Saale) · Deutschland >> HR Stendal HRB 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick >> Liebe Grüße, Christian Theune -- Christian Theune · ct@xxxxxxxxxxxxxxx · +49 345 219401 0 Flying Circus Internet Operations GmbH · https://flyingcircus.io Leipziger Str. 70/71 · 06108 Halle (Saale) · Deutschland HR Stendal HRB 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick
