Hi!
On Wed 23-10-24 11:19:24, John Garry wrote:
> I have been seeing lockups reliably occur on v6.12-rc1, 3, 4 and linus'
> master branch:
>
> Message from syslogd@jgarry-atomic-write-exp-e4-8-instance-20231214-1221 at
> Oct 22 09:07:15 ...
> kernel:watchdog: BUG: soft lockup - CPU#12 stuck for 26s! [khugepaged:154]
>
> Message from syslogd@jgarry-atomic-write-exp-e4-8-instance-20231214-1221 at
> Oct 22 09:08:07 ...
> kernel:BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck
> for 44s!
>
> Message from syslogd@jgarry-atomic-write-exp-e4-8-instance-20231214-1221 at
> Oct 22 09:08:07 ...
> kernel:BUG: workqueue lockup - pool cpus=4 node=0 flags=0x0 nice=0 stuck
> for 35s!
>
> Message from syslogd@jgarry-atomic-write-exp-e4-8-instance-20231214-1221 at
> Oct 22 09:08:07 ...
> kernel:BUG: workqueue lockup - pool cpus=10 node=0 flags=0x0 nice=0 stuck
> for 33s!
>
> This is while doing some server MySQL performance testing. v6.11 has no such
> issue.
>
> I added some debug, and we seem to be spending a lot of time in FS
> writeback, specifically wb_workfn() -> wb_do_writeback() - ring any bells?
Thanks for report. This doesn't remind me of anything but checking the
writeback changes we have merged 532980cb1bf ("inode: port __I_SYNC to var
event") which could have odd consequences if we made mistake somewhere...
It probably won't be easy to revert but you could try whether the problem
reproduces before / after this commit.
Honza
--
Jan Kara <jack@xxxxxxxx>
SUSE Labs, CR
