Hi,
> (switched to email. Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> > SysRq : Show Blocked State
> > task PC stack pid father
> > kjournald D c01384df 0 2525 2
> > cfcb5f0c 00000082 de27d500 c01384df cfcb5ef4 c02cb5c0 00000001 de32ca00
> > de324814 dd037ebc de324814 de324934 dd037ebc cfcb5f5c cfcb5f90 c01bd4bb
> > 00000046 c0424110 de324a0c de324814 de324800 00000000 00000002 dd037e80
> > Call Trace:
> > [<c01384df>] ? trace_hardirqs_on+0xb/0xd
> > [<c01bd4bb>] journal_commit_transaction+0xea/0xeaf
> > [<c02c534a>] ? _spin_unlock_irqrestore+0x38/0x3f
> > [<c0138489>] ? trace_hardirqs_on_caller+0x145/0x190
> > [<c012b6ee>] ? autoremove_wake_function+0x0/0x38
> > [<c0122a25>] ? del_timer+0x50/0x59
> > [<c01c0c75>] kjournald+0xad/0x1bb
> > [<c012b6ee>] ? autoremove_wake_function+0x0/0x38
> > [<c01c0bc8>] ? kjournald+0x0/0x1bb
> > [<c012b442>] kthread+0x37/0x59
> > [<c012b40b>] ? kthread+0x0/0x59
> > [<c0103667>] kernel_thread_helper+0x7/0x10
>
> I assume this is
>
> while (commit_transaction->t_updates) {
> DEFINE_WAIT(wait);
>
> prepare_to_wait(&journal->j_wait_updates, &wait,
> TASK_UNINTERRUPTIBLE);
> if (commit_transaction->t_updates) {
> spin_unlock(&commit_transaction->t_handle_lock);
> spin_unlock(&journal->j_state_lock);
> schedule();
Yes.
> I'm wondering about
>
> : commit e219cca082f52e7dfea41f3be264b7b5eb204227
> : Author: Theodore Ts'o <tytso@xxxxxxx>
> : AuthorDate: Thu Nov 6 22:37:59 2008 -0500
> : Commit: Theodore Ts'o <tytso@xxxxxxx>
> : CommitDate: Thu Nov 6 22:37:59 2008 -0500
> :
> : jbd: don't give up looking for space so easily in __log_wait_for_space
>
> but that patch was present in 2.6.28.
Hmm, I don't see what made this deadlock happening - as far as I can
see it's there for quite some time. See below...
> > pickup D c01384df 0 2597 2594
> > cfaa9e5c 00000086 df9faa80 c01384df cfaa9e44 00000282 cfaa9e74 de32ca00
> > cfaa9e5c c012b8b7 00000002 de324800 0000014f cfaa9e74 cfaa9e94 c01c0539
> > 00000000 de3248c8 de324910 de324814 00000000 df9faa80 c012b6ee de3248e4
> > Call Trace:
> > [<c01384df>] ? trace_hardirqs_on+0xb/0xd
> > [<c012b8b7>] ? prepare_to_wait+0x42/0x4a
> > [<c01c0539>] log_wait_commit+0x90/0xf7
> > [<c012b6ee>] ? autoremove_wake_function+0x0/0x38
> > [<c01bba9d>] journal_stop+0x1c8/0x288
> > [<c01b4236>] __ext3_journal_stop+0x1c/0x38
> > [<c01aeb96>] ext3_delete_inode+0x90/0xc2
> > [<c01aeb06>] ? ext3_delete_inode+0x0/0xc2
> > [<c017ab82>] generic_delete_inode+0x72/0x100
> > [<c02c4ee1>] ? _spin_lock+0x3a/0x40
> > [<c017ad4c>] generic_drop_inode+0x13c/0x1da
> > [<c01d4068>] ? _atomic_dec_and_lock+0x10/0x38
> > [<c017a4e7>] iput+0x47/0x4e
> > [<c0173db0>] do_unlinkat+0xc1/0x137
> > [<c0102f87>] ? sysenter_exit+0xf/0x18
> > [<c0138489>] ? trace_hardirqs_on_caller+0x145/0x190
> > [<c0173e36>] sys_unlink+0x10/0x12
> > [<c0102f55>] sysenter_do_call+0x12/0x35
In generic_delete_inode() we mark inode as I_FREEING. Then
ext3_delete_inode() is called and because of O_SYNC it starts a
transaction commit and waits for it.
> > postdrop D c01384df 0 2664 2663
> > cfcbfd6c 00000086 dd13f700 c01384df cfcbfd54 c02cb5c0 00000001 deedc780
> > c03a1690 c1402348 c03a1690 cfcbfd7c c1402348 cfcbfd90 cfcbfd9c c017a55b
> > dd758c48 00000007 00000000 dd13f700 c012b726 c1402364 c1402364 00152b13
> > Call Trace:
> > [<c01384df>] ? trace_hardirqs_on+0xb/0xd
> > [<c017a55b>] __wait_on_freeing_inode+0x6d/0x88
> > [<c012b726>] ? wake_bit_function+0x0/0x47
> > [<c017a5b5>] find_inode_fast+0x3f/0x4a
> > [<c017ba05>] insert_inode_locked+0x50/0xeb
> > [<c01ab90b>] ext3_new_inode+0x738/0x88f
> > [<c01bc550>] ? journal_start+0xab/0x100
> > [<c01b259a>] ext3_create+0x59/0xbf
> > [<c01722c4>] vfs_create+0x75/0xb0
> > [<c02c4dda>] ? _spin_unlock+0x1d/0x20
> > [<c01b2541>] ? ext3_create+0x0/0xbf
> > [<c0174bc3>] do_filp_open+0x644/0x713
> > [<c02c4dda>] ? _spin_unlock+0x1d/0x20
> > [<c01692ce>] do_sys_open+0x45/0xce
> > [<c0102f87>] ? sysenter_exit+0xf/0x18
> > [<c0138489>] ? trace_hardirqs_on_caller+0x145/0x190
> > [<c01693a3>] sys_open+0x23/0x2b
> > [<c0102f55>] sysenter_do_call+0x12/0x35
Here, we have started a transaction in ext3_create() and then wait in
find_inode_fast() for I_FREEING to be cleared (obviously we have
reallocated the inode and squeezed the allocation before journal_stop()
from the delete was called).
Nasty deadlock and I don't see how to fix it now - have to go home for
today... Tomorrow I'll have a look what we can do about it.
Honza
--
Jan Kara <jack@xxxxxxx>
SuSE CR Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
