Re: CentOS 5 - problem with kernel/process: task blocked for more than 120 seconds.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



The host is:
Linux server 2.6.18-238.19.1.el5 #1 SMP Fri Jul 15 07:32:29 EDT 2011 i686
i686 i386 GNU/Linux
CentOS release 5.6 (Final)
dovecot 1.2.16

Best regards,
R.

W dniu 11 kwietnia 2012 11:48 użytkownik Rafał Radecki <
radecki.rafal@xxxxxxxxx> napisał:

> Hi All.
>
> I have a server which hanged two times because we could not connect to the
> box, it was not responding. In /var/log/messages I saw:
>
> Apr 11 10:13:29 server kernel: INFO: task imap:5855 blocked for more than
> 120 seconds.
> Apr 11 10:13:29 server kernel: "echo 0 >
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Apr 11 10:13:29 server kernel: imap          D 000124D0  1716  5855
> 14393          6320  5839 (NOTLB)
> Apr 11 10:13:29 server kernel: d1adeed0 00000082 632f304d 000124d0
> 000124cf 0000000e 00000000 00000009
> Apr 11 10:13:29 server kernel: c750b000 632f379a 000124d0 0000074d
> 00000000 c750b10c c17f6d00 d2637580
> Apr 11 10:13:29 server kernel: f75d3e68 00000000 00000000 d1adeecc
> c041ec40 00000000 d1adeed8 f75d3e50
> Apr 11 10:13:29 server kernel: Call Trace:
> Apr 11 10:13:29 server kernel: [<c041ec40>] __wake_up+0x2a/0x3d
> Apr 11 10:13:29 server kernel: [<c0436d23>] prepare_to_wait+0x24/0x46
> Apr 11 10:13:29 server kernel: [<f885f1e2>] log_wait_commit+0x80/0xc7 [jbd]
> Apr 11 10:13:29 server kernel: [<c0436bd7>]
> autoremove_wake_function+0x0/0x2d
> Apr 11 10:13:29 server kernel: [<f885a661>] journal_stop+0x195/0x1ba [jbd]
> Apr 11 10:13:29 server kernel: [<c0495caa>]
> __writeback_single_inode+0x197/0x2a3
> Apr 11 10:13:29 server kernel: [<c045ee18>] do_writepages+0x2b/0x32
> Apr 11 10:13:29 server kernel: [<c045a96c>]
> __filemap_fdatawrite_range+0x66/0x72
> Apr 11 10:13:29 server kernel: [<c0496346>] sync_inode+0x19/0x24
> Apr 11 10:13:29 server kernel: [<f8983009>] ext3_sync_file+0xb1/0xdc [ext3]
> Apr 11 10:13:29 server kernel: [<c047956c>] do_fsync+0x41/0x83
> Apr 11 10:13:29 server kernel: [<c04795cb>] __do_fsync+0x1d/0x2b
> Apr 11 10:13:29 server kernel: [<c0404f4b>] syscall_call+0x7/0xb
> Apr 11 10:13:29 server kernel: =======================
> Apr 11 10:13:29 server kernel: INFO: task imap:6882 blocked for more than
> 120 seconds.
> Apr 11 10:13:29 server kernel: "echo 0 >
> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> Apr 11 10:13:29 server kernel: imap          D 000124D0  1772  6882
> 14393          6999  6053 (NOTLB)
> Apr 11 10:13:29 server kernel: cf1d0ed0 00000082 632f379a 000124d0
> 000124cc 0000000e 00000000 00000009
> Apr 11 10:13:29 server kernel: f52e9000 632f3f0a 000124d0 00000770
> 00000000 f52e910c c17f6d00 cbe44740
> Apr 11 10:13:29 server kernel: f75d3e68 00000000 00000000 cf1d0ecc
> c041ec40 00000000 cf1d0ed8 f75d3e50
> Apr 11 10:13:29 server kernel: Call Trace:
> Apr 11 10:13:29 server kernel: [<c041ec40>] __wake_up+0x2a/0x3d
> Apr 11 10:13:29 server kernel: [<c0436d23>] prepare_to_wait+0x24/0x46
> Apr 11 10:13:31 server kernel: [<f885f1e2>] log_wait_commit+0x80/0xc7 [jbd]
> Apr 11 10:13:31 server kernel: [<c0436bd7>]
> autoremove_wake_function+0x0/0x2d
> Apr 11 10:13:31 server kernel: [<f885a661>] journal_stop+0x195/0x1ba [jbd]
> Apr 11 10:13:31 server kernel: [<c0495caa>]
> __writeback_single_inode+0x197/0x2a3
> Apr 11 10:13:31 server kernel: [<c045ee18>] do_writepages+0x2b/0x32
> Apr 11 10:13:31 server kernel: [<c045a96c>]
> __filemap_fdatawrite_range+0x66/0x72
> Apr 11 10:13:31 server kernel: [<c0496346>] sync_inode+0x19/0x24
> Apr 11 10:13:31 server kernel: [<f8983009>] ext3_sync_file+0xb1/0xdc [ext3]
> Apr 11 10:13:31 server kernel: [<c047956c>] do_fsync+0x41/0x83
> Apr 11 10:13:31 server kernel: [<c04795cb>] __do_fsync+0x1d/0x2b
> Apr 11 10:13:31 server kernel: [<c0404f4b>] syscall_call+0x7/0xb
> Apr 11 10:13:31 server kernel: =======================
>
> I have found:
> http://www.nico.schottelius.org/blog/reboot-linux-if-task-blocked-for-more-than-n-seconds/, so I think that the processes took all resources of the machine.
>
> What should be done to troubleshoot the situation?
> I have information that probably imap:5855 and imap:6882 caused the
> problem. How the output:
>
> Apr 11 10:13:29 server kernel: imap          D 000124D0  1772  6882
> 14393          6999  6053 (NOTLB)
> Apr 11 10:13:29 server kernel: cf1d0ed0 00000082 632f379a 000124d0
> 000124cc 0000000e 00000000 00000009
> Apr 11 10:13:29 server kernel: f52e9000 632f3f0a 000124d0 00000770
> 00000000 f52e910c c17f6d00 cbe44740
> Apr 11 10:13:29 server kernel: f75d3e68 00000000 00000000 cf1d0ecc
> c041ec40 00000000 cf1d0ed8 f75d3e50
> Apr 11 10:13:29 server kernel: Call Trace:
> Apr 11 10:13:29 server kernel: [<c041ec40>] __wake_up+0x2a/0x3d
> Apr 11 10:13:29 server kernel: [<c0436d23>] prepare_to_wait+0x24/0x46
> Apr 11 10:13:31 server kernel: [<f885f1e2>] log_wait_commit+0x80/0xc7 [jbd]
> Apr 11 10:13:31 server kernel: [<c0436bd7>]
> autoremove_wake_function+0x0/0x2d
> Apr 11 10:13:31 server kernel: [<f885a661>] journal_stop+0x195/0x1ba [jbd]
> Apr 11 10:13:31 server kernel: [<c0495caa>]
> __writeback_single_inode+0x197/0x2a3
> Apr 11 10:13:31 server kernel: [<c045ee18>] do_writepages+0x2b/0x32
> Apr 11 10:13:31 server kernel: [<c045a96c>]
> __filemap_fdatawrite_range+0x66/0x72
> Apr 11 10:13:31 server kernel: [<c0496346>] sync_inode+0x19/0x24
> Apr 11 10:13:31 server kernel: [<f8983009>] ext3_sync_file+0xb1/0xdc [ext3]
> Apr 11 10:13:31 server kernel: [<c047956c>] do_fsync+0x41/0x83
> Apr 11 10:13:31 server kernel: [<c04795cb>] __do_fsync+0x1d/0x2b
> Apr 11 10:13:31 server kernel: [<c0404f4b>] syscall_call+0x7/0xb
> should be read? Any howto?
>
> Best regards,
> Rafal Radecki.
>
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos



[Index of Archives]     [CentOS]     [CentOS Announce]     [CentOS Development]     [CentOS ARM Devel]     [CentOS Docs]     [CentOS Virtualization]     [Carrier Grade Linux]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Linux USB]
  Powered by Linux