On Wed 07-09-11 13:56:08, Greg Freemyer wrote: > On Wed, Sep 7, 2011 at 1:34 PM, Jan Kara <jack@xxxxxxx> wrote: > > Hello, > > > > Thanks for report! > > > > On Wed 07-09-11 12:29:30, Masayoshi MIZUMA wrote: > >> When I checked the freeze feature for ext3 filesystem using fsfreeze > >> command at 3.1.0-rc4, I think the following deadlock problem happened. > >> > >> How to reproduce: > >> # mkfs -t ext3 /dev/sdd1 > >> # mount /dev/sdd1 /MNT > >> # ./fsstress -d /MNT/tmp -n 10 -p 1000 > /dev/null 2>&1 & > >> # fsfreeze -f /MNT > >> # fsfreeze -u /MNT > >> > >> If this deadlock is reproduced, "fsfreeze -u /MNT" does not return. > >> > >> The detail of deadlock: > >> o [flush-8:16:1523] > >> wb_do_writeback > >> wb_writeback > >> ... > >> ext3_journalled_writepage > >> journal_start > >> start_this_handle > >> # waiting until journal->j_barrier_count turns 0... > >> # j_barrier_count was incremented by journal_lock_updates() > >> # via ext3_freeze(). > >> > >> o [fsstress:2673] > >> sys_sync > >> sync_filesystems > >> iterate_supers > >> down_read(sb->s_umount) > >> sync_one_sb > >> __sync_filesystem > >> writeback_inodes_sb > >> writeback_inodes_sb_nr > >> wait_for_completion > >> wait_for_common > >> # waiting for completion of [flush-8:16:1523]... > >> > >> o [fsfreeze:2749] > >> sys_ioctl > >> do_vfs_ioctl > >> thaw_super > >> # waiting for down_write(sb->s_umount)... > >> # [fsfreeze:2673] did down_read(sb->s_umount). > > Yes, this is a classical deadlock that can happen for any filesystem. The > > problem is flusher thread holds s_umount semaphore (either directly, or as > > in your case, indirectly via blocked sync) and tries to do some IO which > > blocks on frozen filesystem. It's particularly easy to hit for ext3 because > > it doesn't do vfs_check_frozen() checks but all other filesystems have the > > race window as well. Val Henson is working on fixing the problem - she even > > has some first version of patches I believe. > > > > Honza > > xfstests test 068 has been around since kernel 2.4 days and should > have caught it if xfs is impacted. > > I know I ran the 2002 version many times to prove to myself that > fsfreeze for xfs was stable when teamed with LVM. (It wasn't when I > first wrote 068 way back then). > > 068 has been greatly simplified since 2002, but it still looks like it > should do a good job. > > Is there a problem with 068? Does it need extra test coverage even for xfs? I believe at least mmapped writes can trigger the deadlock even for xfs and fsstress (slightly surprisingly) does not test that. It's a narrow race window but it is there and it has been triggered in practice (for ext4 but it's a race in VFS code used by both XFS and ext4). So maybe extending fsstress would be a way to go? Honza -- Jan Kara <jack@xxxxxxx> SUSE Labs, CR -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
