Hello,
I am running a server with heavy workload on a XFS mount:
/dev/mapper/v2-d1 on /d/d1 type xfs (rw,nodev,noatime,inode64,allocsize=16m)
2.6.32-424.el6.x86_64 #1 SMP Mon Oct 14 20:11:50 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
We get errors in log indicating processes are blocked for more than 120 seconds.
1) Is this expected during heavy workload?
2) What would be the impact on the processes? Are they basically hung in userspace waiting for IO?
3) Is there anything we tune here?
Below is the output.
Cheers,
Ivan Novick
INFO: task flush-253:1:6882 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-253:1 D 0000000000000000 0 6882 2 0x00000000
ffff881011fa5690 0000000000000046 0000000000000000 ffff881013cba040
0000000000000001 ffff88012a447d40 ffff881013cba040 ffff881016eb4830
ffff881013cba5f8 ffff881011fa5fd8 000000000000fbc8 ffff881013cba5f8
Call Trace:
[<ffffffff815289d3>] io_schedule+0x73/0xc0
[<ffffffff81267d18>] get_request_wait+0x108/0x1d0
[<ffffffff8109b4a0>] ? autoremove_wake_function+0x0/0x40
[<ffffffff812618ce>] ? elv_merge+0x17e/0x1c0
[<ffffffff81267e79>] blk_queue_bio+0x99/0x620
[<ffffffff81266f00>] generic_make_request+0x240/0x5a0
[<ffffffffa01c3f97>] ? dm_merge_bvec+0xc7/0x100 [dm_mod]
[<ffffffff812672d0>] submit_bio+0x70/0x120
[<ffffffffa03b9c53>] xfs_submit_ioend_bio+0x33/0x40 [xfs]
[<ffffffffa03b9d2c>] xfs_submit_ioend+0xcc/0x140 [xfs]
[<ffffffffa03ba678>] xfs_vm_writepage+0x308/0x590 [xfs]
[<ffffffff81133967>] __writepage+0x17/0x40
[<ffffffff81134c2d>] write_cache_pages+0x1fd/0x4c0
[<ffffffff8100bdae>] ? call_function_single_interrupt+0xe/0x20
[<ffffffff81133950>] ? __writepage+0x0/0x40
[<ffffffff81134f14>] generic_writepages+0x24/0x30
[<ffffffffa03b98ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
[<ffffffff81134f41>] do_writepages+0x21/0x40
[<ffffffff811b52ad>] writeback_single_inode+0xdd/0x290
[<ffffffff811b56ad>] writeback_sb_inodes+0xbd/0x170
[<ffffffff811b580b>] writeback_inodes_wb+0xab/0x1b0
[<ffffffff811b5c03>] wb_writeback+0x2f3/0x410
[<ffffffff81528240>] ? thread_return+0x4e/0x76e
[<ffffffff81084fb2>] ? del_timer_sync+0x22/0x30
[<ffffffff811b5ec5>] wb_do_writeback+0x1a5/0x240
[<ffffffff811b5fc3>] bdi_writeback_task+0x63/0x1b0
[<ffffffff8109b327>] ? bit_waitqueue+0x17/0xd0
[<ffffffff81143c40>] ? bdi_start_fn+0x0/0x100
[<ffffffff81143cc6>] bdi_start_fn+0x86/0x100
[<ffffffff81143c40>] ? bdi_start_fn+0x0/0x100
[<ffffffff8109b0f6>] kthread+0x96/0xa0
[<ffffffff8100c20a>] child_rip+0xa/0x20
[<ffffffff8109b060>] ? kthread+0x0/0xa0
[<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task ivanserver:6969 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000000 0 6969 6947 0x00000000
ffff881f4ef179d8 0000000000000082 0000000000000000 000000000000000f
0000000000000001 ffff8800622143c0 0000000000000001 ffff880062216840
ffff881f4eeae638 ffff881f4ef17fd8 000000000000fbc8 ffff881f4eeae638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8100bdae>] ? call_function_single_interrupt+0xe/0x20
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8118a5b0>] ? fget+0x30/0x80
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task ivanserver:6972 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000008 0 6972 6948 0x00000000
ffff88100f1219d8 0000000000000086 0000000000000000 000000000000000f
0000000000000001 ffff8810788143c0 0000000000000001 ffff881078816840
ffff88100fd62638 ffff88100f121fd8 000000000000fbc8 ffff88100fd62638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task ivanserver:6985 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000000 0 6985 6963 0x00000000
ffff8810103ad9d8 0000000000000082 0000000000000000 000000000000000f
0000000000000001 ffff8800622143c0 0000000000000001 ffff880062216840
ffff88100e01e638 ffff8810103adfd8 000000000000fbc8 ffff88100e01e638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task ivanserver:6989 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000008 0 6989 6981 0x00000000
ffff8810101279d8 0000000000000086 0000000000000000 000000000000000f
0000000000000001 ffff8810788143c0 0000000000000001 ffff881078816840
ffff8810252a4638 ffff881010127fd8 000000000000fbc8 ffff8810252a4638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
usxxnovicim1:Downloads novici$ cat dmesg.txt
INFO: task flush-253:1:6882 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-253:1 D 0000000000000000 0 6882 2 0x00000000
ffff881011fa5690 0000000000000046 0000000000000000 ffff881013cba040
0000000000000001 ffff88012a447d40 ffff881013cba040 ffff881016eb4830
ffff881013cba5f8 ffff881011fa5fd8 000000000000fbc8 ffff881013cba5f8
Call Trace:
[<ffffffff815289d3>] io_schedule+0x73/0xc0
[<ffffffff81267d18>] get_request_wait+0x108/0x1d0
[<ffffffff8109b4a0>] ? autoremove_wake_function+0x0/0x40
[<ffffffff812618ce>] ? elv_merge+0x17e/0x1c0
[<ffffffff81267e79>] blk_queue_bio+0x99/0x620
[<ffffffff81266f00>] generic_make_request+0x240/0x5a0
[<ffffffffa01c3f97>] ? dm_merge_bvec+0xc7/0x100 [dm_mod]
[<ffffffff812672d0>] submit_bio+0x70/0x120
[<ffffffffa03b9c53>] xfs_submit_ioend_bio+0x33/0x40 [xfs]
[<ffffffffa03b9d2c>] xfs_submit_ioend+0xcc/0x140 [xfs]
[<ffffffffa03ba678>] xfs_vm_writepage+0x308/0x590 [xfs]
[<ffffffff81133967>] __writepage+0x17/0x40
[<ffffffff81134c2d>] write_cache_pages+0x1fd/0x4c0
[<ffffffff8100bdae>] ? call_function_single_interrupt+0xe/0x20
[<ffffffff81133950>] ? __writepage+0x0/0x40
[<ffffffff81134f14>] generic_writepages+0x24/0x30
[<ffffffffa03b98ad>] xfs_vm_writepages+0x5d/0x80 [xfs]
[<ffffffff81134f41>] do_writepages+0x21/0x40
[<ffffffff811b52ad>] writeback_single_inode+0xdd/0x290
[<ffffffff811b56ad>] writeback_sb_inodes+0xbd/0x170
[<ffffffff811b580b>] writeback_inodes_wb+0xab/0x1b0
[<ffffffff811b5c03>] wb_writeback+0x2f3/0x410
[<ffffffff81528240>] ? thread_return+0x4e/0x76e
[<ffffffff81084fb2>] ? del_timer_sync+0x22/0x30
[<ffffffff811b5ec5>] wb_do_writeback+0x1a5/0x240
[<ffffffff811b5fc3>] bdi_writeback_task+0x63/0x1b0
[<ffffffff8109b327>] ? bit_waitqueue+0x17/0xd0
[<ffffffff81143c40>] ? bdi_start_fn+0x0/0x100
[<ffffffff81143cc6>] bdi_start_fn+0x86/0x100
[<ffffffff81143c40>] ? bdi_start_fn+0x0/0x100
[<ffffffff8109b0f6>] kthread+0x96/0xa0
[<ffffffff8100c20a>] child_rip+0xa/0x20
[<ffffffff8109b060>] ? kthread+0x0/0xa0
[<ffffffff8100c200>] ? child_rip+0x0/0x20
INFO: task ivanserver:6969 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000000 0 6969 6947 0x00000000
ffff881f4ef179d8 0000000000000082 0000000000000000 000000000000000f
0000000000000001 ffff8800622143c0 0000000000000001 ffff880062216840
ffff881f4eeae638 ffff881f4ef17fd8 000000000000fbc8 ffff881f4eeae638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8100bdae>] ? call_function_single_interrupt+0xe/0x20
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8118a5b0>] ? fget+0x30/0x80
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task ivanserver:6972 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000008 0 6972 6948 0x00000000
ffff88100f1219d8 0000000000000086 0000000000000000 000000000000000f
0000000000000001 ffff8810788143c0 0000000000000001 ffff881078816840
ffff88100fd62638 ffff88100f121fd8 000000000000fbc8 ffff88100fd62638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task ivanserver:6985 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000000 0 6985 6963 0x00000000
ffff8810103ad9d8 0000000000000082 0000000000000000 000000000000000f
0000000000000001 ffff8800622143c0 0000000000000001 ffff880062216840
ffff88100e01e638 ffff8810103adfd8 000000000000fbc8 ffff88100e01e638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
INFO: task ivanserver:6989 blocked for more than 120 seconds.
Tainted: P --------------- 2.6.32-424.el6.x86_64 #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
ivanserver D 0000000000000008 0 6989 6981 0x00000000
ffff8810101279d8 0000000000000086 0000000000000000 000000000000000f
0000000000000001 ffff8810788143c0 0000000000000001 ffff881078816840
ffff8810252a4638 ffff881010127fd8 000000000000fbc8 ffff8810252a4638
Call Trace:
[<ffffffff815290e5>] schedule_timeout+0x215/0x2e0
[<ffffffff8152a0e2>] __down+0x72/0xb0
[<ffffffffa03bcaa5>] ? _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffff810a1591>] down+0x41/0x50
[<ffffffffa03bc911>] xfs_buf_lock+0x51/0x100 [xfs]
[<ffffffffa03bcaa5>] _xfs_buf_find+0xe5/0x230 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bd266>] ? _xfs_buf_read+0x46/0x60 [xfs]
[<ffffffffa03bcc24>] xfs_buf_get+0x34/0x1b0 [xfs]
[<ffffffffa03bd2ac>] xfs_buf_read+0x2c/0x100 [xfs]
[<ffffffffa03b2c37>] xfs_trans_read_buf+0x197/0x410 [xfs]
[<ffffffffa0385af9>] xfs_da_do_buf+0x279/0x770 [xfs]
[<ffffffffa03babcf>] ? xfs_buf_lru_add+0x6f/0x80 [xfs]
[<ffffffffa03b3288>] ? xfs_trans_brelse+0xe8/0x130 [xfs]
[<ffffffffa038606a>] xfs_da_read_buf+0x2a/0x30 [xfs]
[<ffffffffa038c9e6>] ? xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffffa038c9e6>] xfs_dir2_leaf_getdents+0x266/0x780 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03891e9>] xfs_readdir+0xf9/0x130 [xfs]
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffff8119f020>] ? filldir+0x0/0xe0
[<ffffffffa03be619>] xfs_file_readdir+0x39/0x50 [xfs]
[<ffffffff8119f2a0>] vfs_readdir+0xc0/0xe0
[<ffffffff8119f429>] sys_getdents+0x89/0xf0
[<ffffffff8100bfbb>] ? device_not_available+0x1b/0x20
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
_______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
