Hi Amir and Chandan, much appreciated! I’m building up a history of being a truffle pig WRT forgotten fixes. :) Christian > On 13. Apr 2023, at 12:31, Chandan Babu R <chandan.babu@xxxxxxxxxx> wrote: > > On Thu, Apr 13, 2023 at 10:44:43 AM +0300, Amir Goldstein wrote: >> On Wed, Apr 12, 2023 at 6:58 PM Christian Theune <ct@xxxxxxxxxxxxxxx> wrote: >>> >>> Hi, >>> >>> ugh. Sorry, looks like I jumped the gun. Mea culpa. >>> >>> We experienced a hang like this: >>> >>> Apr 05 11:51:27 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. >>> Apr 05 11:51:27 kernel: task:xfs-conv/vdc1 state:D stack: 0 pid: 606 ppid: 2 flags:0x00004080 >>> Apr 05 11:51:27 kernel: Workqueue: xfs-conv/vdc1 xfs_end_io [xfs] >>> Apr 05 11:51:27 kernel: Call Trace: >>> Apr 05 11:51:27 kernel: __schedule+0x274/0x870 >>> Apr 05 11:51:27 kernel: schedule+0x46/0xb0 >>> Apr 05 11:51:27 kernel: xlog_grant_head_wait+0xc5/0x1d0 [xfs] >>> Apr 05 11:51:27 kernel: xlog_grant_head_check+0xde/0x100 [xfs] >>> Apr 05 11:51:27 kernel: xfs_log_reserve+0xbe/0x1b0 [xfs] >>> Apr 05 11:51:27 kernel: xfs_trans_reserve+0x143/0x180 [xfs] >>> Apr 05 11:51:27 kernel: xfs_trans_alloc+0xee/0x1a0 [xfs] >>> Apr 05 11:51:27 kernel: xfs_iomap_write_unwritten+0x120/0x2e0 [xfs] >>> Apr 05 11:51:27 kernel: ? record_times+0x15/0x90 >>> Apr 05 11:51:27 kernel: xfs_end_ioend+0xd8/0x140 [xfs] >>> Apr 05 11:51:27 kernel: xfs_end_io+0xb8/0xf0 [xfs] >>> Apr 05 11:51:27 kernel: process_one_work+0x1b6/0x350 >>> Apr 05 11:51:27 kernel: rescuer_thread+0x1d1/0x3a0 >>> Apr 05 11:51:27 kernel: ? worker_thread+0x3e0/0x3e0 >>> Apr 05 11:51:27 kernel: kthread+0x11b/0x140 >>> Apr 05 11:51:27 kernel: ? kthread_associate_blkcg+0xb0/0xb0 >>> Apr 05 11:51:27 kernel: ret_from_fork+0x22/0x30 >>> >>> Which seems to be similar to this: >>> https://bugs.launchpad.net/bugs/1996269 >>> >>> I followed their patchset here: >>> https://review.opendev.org/c/starlingx/kernel/+/864257 >>> >>> And I was under the impression that I picked the right one to ask >>> for backporting, but it seems that was incorrect. I went through the >>> list again and I think the following patches are the ones missing >>> from 5.10: >>> >>> 8182ec00803085354761bbadf0287cad7eac0e2f - >>> https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0035-xfs-drop-submit-side-trans-alloc-for-append-ioends.patch >>> edbf1eb9032b84631031d9b43570e262f3461c24 - >>> https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0036-xfs-open-code-ioend-needs-workqueue-helper.patch >>> 170e31793806ce5e5a9647b6340954536244518e - >>> https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0037-xfs-drop-unused-ioend-private-merge-and-setfilesize-.patch >>> 2fd609b6c90a88630a50fb317473b210759b3873 - >>> https://review.opendev.org/c/starlingx/kernel/+/864257/5/kernel-std/centos/patches/0038-xfs-drop-unnecessary-setfilesize-helper.patch >>> >> >> The only commit that fixes the bug is: >> 7cd3099f4925 xfs: drop submit side trans alloc for append ioends >> >> The rest are just code cleanups. >> >> That fix was missed in my original backports from v5.13 because of a tool error, >> so thank you for pointing it out. >> >> I have added it to my test branch and will follow up with posting to >> stable later on. >> >> Chandan, >> >> Please make sure you include this fix when you get to considering >> fixes from v5.13 to 5.4.y. >> > > Sure, I will do that. However ... > >> I will wait with posting this fix to 5.10.y until I get the v5.13 >> backports wish list from you. >> > > Since I am working on another XFS work item there will be some delay before I > share the list of patches to be backported from v5.13 to 5.4.y. > > -- > chandan Liebe Grüße, Christian Theune -- Christian Theune · ct@xxxxxxxxxxxxxxx · +49 345 219401 0 Flying Circus Internet Operations GmbH · https://flyingcircus.io Leipziger Str. 70/71 · 06108 Halle (Saale) · Deutschland HR Stendal HRB 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick