Re: [bug report] INFO: task mdX_resync:42168 blocked for more than 122 seconds

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, May 19, 2024 at 2:44 PM Changhui Zhong <czhong@xxxxxxxxxx> wrote:
>
> On Fri, May 17, 2024 at 10:49 AM Yu Kuai <yukuai1@xxxxxxxxxxxxxxx> wrote:
> >
> >
> > Thanks for the test, this do look like a deadlock, beside
> > raise_barrier(), is there any other victim? I can't reporduce this,
> > and I have no clue yet. The possible next step might be bisect to
> > locate the blame commit first. Maybe related to dm-raid1.
> >
> > Thanks,
> > Kuai
> >
>
> Hi,Yu Kuai
>
> I tried to do git bisect and got the following result, please help check,
>
> [czhong@vm linux-block]$ git bisect start
> [czhong@vm linux-block]$ git bisect bad
> [czhong@vm linux-block]$ git bisect good
> d0487577e6e0b640d71375a6ec2f9e8a2d3555f2
> Bisecting: 2652 revisions left to test after this (roughly 11 steps)
> [895621f1c81695da7660fe909173e9f98619e89c] bnxt_en: Don't support
> offline self test when RoCE driver is loaded
> [czhong@vm linux-block]$
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 1219 revisions left to test after this (roughly 10 steps)
> [6c60000f0b9ae7da630a5715a9ba33042d87e7fd] Merge tag 'soc-dt-6.10' of
> git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> [czhong@vm linux-block]$
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 612 revisions left to test after this (roughly 9 steps)
> [87caef42200cd44f8b808ec2f8ac2257f3e0a8c1] Merge tag
> 'hardening-6.10-rc1' of
> git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux
> [czhong@vm linux-block]$
> [czhong@vm linux-block]$ git bisect bad
> Bisecting: 303 revisions left to test after this (roughly 8 steps)
> [25c73642cc5baea5b91bbb9b1f5fcd93672bfa08] Merge tag
> 'keys-next-6.10-rc1' of
> git://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 155 revisions left to test after this (roughly 7 steps)
> [f4e8d80292859809ea135e9f4c43bae47e4f58bc] Merge tag 'vfs-6.10.rw' of
> git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
> [czhong@vm linux-block]$
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 77 revisions left to test after this (roughly 6 steps)
> [ac5f71a3d9d7eb540f6bf7e794eb4a3e4c3f11dd] io_uring/net: add provided
> buffer support for IORING_OP_SEND
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 37 revisions left to test after this (roughly 5 steps)
> [0c9f4ac808b017a0013cee92a30de980550145d5] Merge tag
> 'for-6.10/block-20240511' of git://git.kernel.dk/linux
> [czhong@vm linux-block]$ git bisect bad
> Bisecting: 19 revisions left to test after this (roughly 4 steps)
> [a3166c51702bb00b8f8b84022090cbab8f37be1a] blk-throttle: delay
> initialization until configuration
> [czhong@vm linux-block]$
> [czhong@vm linux-block]$ git bisect bad
> Bisecting: 9 revisions left to test after this (roughly 3 steps)
> [e8b4869bc78da1a71f2a2ab476caf50c1dcfeed0] block: add a
> blk_alloc_discard_bio helper
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 4 revisions left to test after this (roughly 2 steps)
> [3a861560ccb35f2a4f0a4b8207fa7c2a35fc7f31] bcache: fix variable length
> array abuse in btree_iter
> [czhong@vm linux-block]$ git bisect good
> Bisecting: 2 revisions left to test after this (roughly 1 step)
> [99dc422335d8b2bd4d105797241d3e715bae90e9] block: support to account
> io_ticks precisely
> [czhong@vm linux-block]$
> [czhong@vm linux-block]$ git bisect bad
> Bisecting: 0 revisions left to test after this (roughly 0 steps)
> [060406c61c7cb4bbd82a02d179decca9c9bb3443] block: add plug while submitting IO
> [czhong@vm linux-block]$ git bisect bad
> 060406c61c7cb4bbd82a02d179decca9c9bb3443 is the first bad commit
> commit 060406c61c7cb4bbd82a02d179decca9c9bb3443
> Author: Yu Kuai <yukuai3@xxxxxxxxxx>
> Date:   Thu May 9 20:38:25 2024 +0800
>
>     block: add plug while submitting IO
>
>     So that if caller didn't use plug, for example, __blkdev_direct_IO_simple()
>     and __blkdev_direct_IO_async(), block layer can still benefit from caching
>     nsec time in the plug.
>
>     Signed-off-by: Yu Kuai <yukuai3@xxxxxxxxxx>
>     Link: https://lore.kernel.org/r/20240509123825.3225207-1-yukuai1@xxxxxxxxxxxxxxx
>     Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
>
>  block/blk-core.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> [czhong@vm linux-block]$
>
> Thanks,
> Changhui

update with  git bisect log:
[czhong@vm linux-block]$ git bisect log
git bisect start
# bad: [59ef8180748269837975c9656b586daa16bb9def] Merge branch
'block-6.10' into for-next
git bisect bad 59ef8180748269837975c9656b586daa16bb9def
# good: [d0487577e6e0b640d71375a6ec2f9e8a2d3555f2] Merge tag
'md-6.10-20240502' of
https://git.kernel.org/pub/scm/linux/kernel/git/song/md into
for-6.10/block
git bisect good d0487577e6e0b640d71375a6ec2f9e8a2d3555f2
# good: [895621f1c81695da7660fe909173e9f98619e89c] bnxt_en: Don't
support offline self test when RoCE driver is loaded
git bisect good 895621f1c81695da7660fe909173e9f98619e89c
# good: [6c60000f0b9ae7da630a5715a9ba33042d87e7fd] Merge tag
'soc-dt-6.10' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good 6c60000f0b9ae7da630a5715a9ba33042d87e7fd
# bad: [87caef42200cd44f8b808ec2f8ac2257f3e0a8c1] Merge tag
'hardening-6.10-rc1' of
git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux
git bisect bad 87caef42200cd44f8b808ec2f8ac2257f3e0a8c1
# good: [25c73642cc5baea5b91bbb9b1f5fcd93672bfa08] Merge tag
'keys-next-6.10-rc1' of
git://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd
git bisect good 25c73642cc5baea5b91bbb9b1f5fcd93672bfa08
# good: [f4e8d80292859809ea135e9f4c43bae47e4f58bc] Merge tag
'vfs-6.10.rw' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
git bisect good f4e8d80292859809ea135e9f4c43bae47e4f58bc
# good: [ac5f71a3d9d7eb540f6bf7e794eb4a3e4c3f11dd] io_uring/net: add
provided buffer support for IORING_OP_SEND
git bisect good ac5f71a3d9d7eb540f6bf7e794eb4a3e4c3f11dd
# bad: [0c9f4ac808b017a0013cee92a30de980550145d5] Merge tag
'for-6.10/block-20240511' of git://git.kernel.dk/linux
git bisect bad 0c9f4ac808b017a0013cee92a30de980550145d5
# bad: [a3166c51702bb00b8f8b84022090cbab8f37be1a] blk-throttle: delay
initialization until configuration
git bisect bad a3166c51702bb00b8f8b84022090cbab8f37be1a
# good: [e8b4869bc78da1a71f2a2ab476caf50c1dcfeed0] block: add a
blk_alloc_discard_bio helper
git bisect good e8b4869bc78da1a71f2a2ab476caf50c1dcfeed0
# good: [3a861560ccb35f2a4f0a4b8207fa7c2a35fc7f31] bcache: fix
variable length array abuse in btree_iter
git bisect good 3a861560ccb35f2a4f0a4b8207fa7c2a35fc7f31
# bad: [99dc422335d8b2bd4d105797241d3e715bae90e9] block: support to
account io_ticks precisely
git bisect bad 99dc422335d8b2bd4d105797241d3e715bae90e9
# bad: [060406c61c7cb4bbd82a02d179decca9c9bb3443] block: add plug
while submitting IO
git bisect bad 060406c61c7cb4bbd82a02d179decca9c9bb3443
# first bad commit: [060406c61c7cb4bbd82a02d179decca9c9bb3443] block:
add plug while submitting IO
[czhong@vm linux-block]$

Thanks,






[Index of Archives]     [DM Crypt]     [Fedora Desktop]     [ATA RAID]     [Fedora Marketing]     [Fedora Packaging]     [Fedora SELinux]     [Yosemite Discussion]     [KDE Users]     [Fedora Docs]

  Powered by Linux