I have an HP DL385 running RHEL5.5 that's close to going into production. However, we've experienced a couple of lockups in the past couple weeks that has management concerned. I've been doing some research but I haven't found any answer to the issue. I did see something about a possible issue with the cciss driver so I decided to run bonnie++ against one of the internal LUNs and sure enough I can duplocate the issue within minutes. What is going on here? Is there a fix for this issue? Is anyone working on a solution??? >Jun 2 12:59:46 hidden kernel: INFO: task kswapd0:563 blocked for more >than >120 seconds. >Jun 2 12:59:46 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 12:59:46 hidden kernel: kswapd0 D ffff810021004420 0 >563 >185 564 560 (L-TLB) >Jun 2 12:59:46 hidden kernel: ffff81042efd9b00 0000000000000046 >0000000000000002 0000000000000010 >Jun 2 12:59:46 hidden kernel: 0000000000008f3f 000000000000000a >ffff81042f5950c0 ffffffff80311b60 >Jun 2 12:59:46 hidden kernel: 0002c883e50c78ed 00000000002f2877 >ffff81042f5952a8 000000000000003d >Jun 2 12:59:46 hidden kernel: Call Trace: >Jun 2 12:59:46 hidden kernel: [<ffffffff88036d8a>] >:jbd:log_wait_commit+0xa3/0xf5 >Jun 2 12:59:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 12:59:46 hidden kernel: [<ffffffff8002781a>] >try_to_free_buffers+0x60/0xb9 >Jun 2 12:59:46 hidden kernel: [<ffffffff880332da>] >:jbd:journal_try_to_free_buffers+0x19d/0x1c0 >Jun 2 12:59:46 hidden kernel: [<ffffffff800cd369>] >shrink_inactive_list+0x511/0x8d8 >Jun 2 12:59:46 hidden kernel: [<ffffffff8004817a>] >__pagevec_release+0x19/0x22 >Jun 2 12:59:48 hidden kernel: [<ffffffff800ccd37>] >shrink_active_list+0x4b4/0x4c4 >Jun 2 12:59:49 hidden kernel: [<ffffffff8001327d>] >shrink_zone+0x127/0x18d >Jun 2 12:59:50 hidden kernel: [<ffffffff80057e54>] kswapd+0x33d/0x495 >Jun 2 12:59:50 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 12:59:51 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 12:59:51 hidden kernel: [<ffffffff80057b17>] kswapd+0x0/0x495 >Jun 2 12:59:51 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 12:59:51 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132 >Jun 2 12:59:52 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 >Jun 2 12:59:53 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 12:59:54 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132 >Jun 2 12:59:54 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 >Jun 2 12:59:55 hidden kernel: >Jun 2 12:59:56 hidden kernel: INFO: task kjournald:6071 blocked for >more >than 120 seconds. >Jun 2 12:59:56 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 12:59:57 hidden kernel: kjournald D ffff81002100caa0 0 >6071 >185 6077 5264 (L-TLB) >Jun 2 12:59:58 hidden kernel: ffff810427f4bcf0 0000000000000046 >000000002dd9c000 ffff81042ff15950 >Jun 2 12:59:58 hidden kernel: 00000001ffffffff 0000000000000009 >ffff810420322080 ffff81042ff18100 >Jun 2 12:59:59 hidden kernel: 0002c889276dcdab 00000000071ac130 >ffff810420322268 0000000100000086 >Jun 2 13:00:00 hidden kernel: Call Trace: >Jun 2 13:00:02 hidden kernel: [<ffffffff8006ec4e>] >do_gettimeofday+0x40/0x90 >Jun 2 13:00:02 hidden kernel: [<ffffffff800155a2>] >sync_buffer+0x0/0x3f >Jun 2 13:00:03 hidden kernel: [<ffffffff800637ca>] >io_schedule+0x3f/0x67 >Jun 2 13:00:04 hidden kernel: [<ffffffff800155dd>] >sync_buffer+0x3b/0x3f >Jun 2 13:00:05 hidden kernel: [<ffffffff800639f6>] >__wait_on_bit+0x40/0x6e >Jun 2 13:00:06 hidden kernel: [<ffffffff800155a2>] >sync_buffer+0x0/0x3f >Jun 2 13:00:06 hidden kernel: [<ffffffff80063a90>] >out_of_line_wait_on_bit+0x6c/0x78 >Jun 2 13:00:07 hidden kernel: [<ffffffff800a2921>] >wake_bit_function+0x0/0x23 >Jun 2 13:00:08 hidden kernel: [<ffffffff880339a5>] >:jbd:journal_commit_transaction+0x543/0x1066 >Jun 2 13:00:10 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:00:12 hidden kernel: [<ffffffff8004b3fb>] >try_to_del_timer_sync+0x7f/0x88 >Jun 2 13:00:13 hidden kernel: [<ffffffff880375d3>] >:jbd:kjournald+0xc1/0x213 >Jun 2 13:00:13 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:00:13 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:00:15 hidden kernel: [<ffffffff88037512>] >:jbd:kjournald+0x0/0x213 >Jun 2 13:00:15 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:00:17 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132 >Jun 2 13:00:18 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 >Jun 2 13:00:19 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:00:19 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132 >Jun 2 13:00:20 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 >Jun 2 13:00:22 hidden kernel: >Jun 2 13:01:46 hidden kernel: INFO: task kswapd0:563 blocked for more >than >120 seconds. >Jun 2 13:01:46 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 13:01:46 hidden kernel: kswapd0 D ffff810021004420 0 >563 >185 564 560 (L-TLB) >Jun 2 13:01:46 hidden kernel: ffff81042efd9b00 0000000000000046 >0000000000000002 0000000000000010 >Jun 2 13:01:46 hidden kernel: 0000000000008f3f 000000000000000a >ffff81042f5950c0 ffffffff80311b60 >Jun 2 13:01:46 hidden kernel: 0002c883e50c78ed 00000000002f2877 >ffff81042f5952a8 000000000000003d >Jun 2 13:01:46 hidden kernel: Call Trace: >Jun 2 13:01:46 hidden kernel: [<ffffffff88036d8a>] >:jbd:log_wait_commit+0xa3/0xf5 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:01:46 hidden kernel: [<ffffffff8002781a>] >try_to_free_buffers+0x60/0xb9 >Jun 2 13:01:46 hidden kernel: [<ffffffff880332da>] >:jbd:journal_try_to_free_buffers+0x19d/0x1c0 >Jun 2 13:01:46 hidden kernel: [<ffffffff800cd369>] >shrink_inactive_list+0x511/0x8d8 >Jun 2 13:01:46 hidden kernel: [<ffffffff8004817a>] >__pagevec_release+0x19/0x22 >Jun 2 13:01:46 hidden kernel: [<ffffffff800ccd37>] >shrink_active_list+0x4b4/0x4c4 >Jun 2 13:01:46 hidden kernel: [<ffffffff8001327d>] >shrink_zone+0x127/0x18d >Jun 2 13:01:46 hidden kernel: [<ffffffff80057e54>] kswapd+0x33d/0x495 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:01:46 hidden kernel: [<ffffffff80057b17>] kswapd+0x0/0x495 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:01:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132 >Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:01:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132 >Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 >Jun 2 13:01:46 hidden kernel: >Jun 2 13:01:46 hidden kernel: INFO: task kjournald:6071 blocked for >more >than 120 seconds. >Jun 2 13:01:46 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 13:01:46 hidden kernel: kjournald D ffff81002100caa0 0 >6071 >185 6077 5264 (L-TLB) >Jun 2 13:01:46 hidden kernel: ffff810427f4bcf0 0000000000000046 >000000002dd9c000 ffff81042ff15950 >Jun 2 13:01:46 hidden kernel: 00000001ffffffff 0000000000000009 >ffff810420322080 ffff81042ff18100 >Jun 2 13:01:46 hidden kernel: 0002c889276dcdab 00000000071ac130 >ffff810420322268 0000000100000086 >Jun 2 13:01:46 hidden kernel: Call Trace: >Jun 2 13:01:46 hidden kernel: [<ffffffff8006ec4e>] >do_gettimeofday+0x40/0x90 >Jun 2 13:01:46 hidden kernel: [<ffffffff800155a2>] >sync_buffer+0x0/0x3f >Jun 2 13:01:46 hidden kernel: [<ffffffff800637ca>] >io_schedule+0x3f/0x67 >Jun 2 13:01:46 hidden kernel: [<ffffffff800155dd>] >sync_buffer+0x3b/0x3f >Jun 2 13:01:46 hidden kernel: [<ffffffff800639f6>] >__wait_on_bit+0x40/0x6e >Jun 2 13:01:46 hidden kernel: [<ffffffff800155a2>] >sync_buffer+0x0/0x3f >Jun 2 13:01:46 hidden kernel: [<ffffffff80063a90>] >out_of_line_wait_on_bit+0x6c/0x78 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a2921>] >wake_bit_function+0x0/0x23 >Jun 2 13:01:46 hidden kernel: [<ffffffff880339a5>] >:jbd:journal_commit_transaction+0x543/0x1066 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:01:46 hidden kernel: [<ffffffff8004b3fb>] >try_to_del_timer_sync+0x7f/0x88 >Jun 2 13:01:46 hidden kernel: [<ffffffff880375d3>] >:jbd:kjournald+0xc1/0x213 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:01:46 hidden kernel: [<ffffffff88037512>] >:jbd:kjournald+0x0/0x213 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:01:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132 >Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 >Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:01:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132 >Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 >Jun 2 13:01:46 hidden kernel: >Jun 2 13:03:46 hidden kernel: INFO: task kswapd0:563 blocked for more >than >120 seconds. >Jun 2 13:03:46 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 13:03:46 hidden kernel: kswapd0 D ffff810021004420 0 >563 >185 564 560 (L-TLB) >Jun 2 13:03:46 hidden kernel: ffff81042efd9b00 0000000000000046 >0000000000000002 0000000000000010 >Jun 2 13:03:46 hidden kernel: 0000000000008f3f 000000000000000a >ffff81042f5950c0 ffffffff80311b60 >Jun 2 13:03:46 hidden kernel: 0002c883e50c78ed 00000000002f2877 >ffff81042f5952a8 000000000000003d >Jun 2 13:03:46 hidden kernel: Call Trace: >Jun 2 13:03:46 hidden kernel: [<ffffffff88036d8a>] >:jbd:log_wait_commit+0xa3/0xf5 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:03:46 hidden kernel: [<ffffffff8002781a>] >try_to_free_buffers+0x60/0xb9 >Jun 2 13:03:46 hidden kernel: [<ffffffff880332da>] >:jbd:journal_try_to_free_buffers+0x19d/0x1c0 >Jun 2 13:03:46 hidden kernel: [<ffffffff800cd369>] >shrink_inactive_list+0x511/0x8d8 >Jun 2 13:03:46 hidden kernel: [<ffffffff8004817a>] >__pagevec_release+0x19/0x22 >Jun 2 13:03:46 hidden kernel: [<ffffffff800ccd37>] >shrink_active_list+0x4b4/0x4c4 >Jun 2 13:03:46 hidden kernel: [<ffffffff8001327d>] >shrink_zone+0x127/0x18d >Jun 2 13:03:46 hidden kernel: [<ffffffff80057e54>] kswapd+0x33d/0x495 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:03:46 hidden kernel: [<ffffffff80057b17>] kswapd+0x0/0x495 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:03:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132 >Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:03:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132 >Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 >Jun 2 13:03:46 hidden kernel: >Jun 2 13:03:46 hidden kernel: INFO: task kjournald:6071 blocked for >more >than 120 seconds. >Jun 2 13:03:46 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 13:03:46 hidden kernel: kjournald D ffff81002100caa0 0 >6071 >185 6077 5264 (L-TLB) >Jun 2 13:03:46 hidden kernel: ffff810427f4bcf0 0000000000000046 >000000002dd9c000 ffff81042ff15950 >Jun 2 13:03:46 hidden kernel: 00000001ffffffff 0000000000000009 >ffff810420322080 ffff81042ff18100 >Jun 2 13:03:46 hidden kernel: 0002c889276dcdab 00000000071ac130 >ffff810420322268 0000000100000086 >Jun 2 13:03:46 hidden kernel: Call Trace: >Jun 2 13:03:46 hidden kernel: [<ffffffff8006ec4e>] >do_gettimeofday+0x40/0x90 >Jun 2 13:03:46 hidden kernel: [<ffffffff800155a2>] >sync_buffer+0x0/0x3f >Jun 2 13:03:46 hidden kernel: [<ffffffff800637ca>] >io_schedule+0x3f/0x67 >Jun 2 13:03:46 hidden kernel: [<ffffffff800155dd>] >sync_buffer+0x3b/0x3f >Jun 2 13:03:46 hidden kernel: [<ffffffff800639f6>] >__wait_on_bit+0x40/0x6e >Jun 2 13:03:46 hidden kernel: [<ffffffff800155a2>] >sync_buffer+0x0/0x3f >Jun 2 13:03:46 hidden kernel: [<ffffffff80063a90>] >out_of_line_wait_on_bit+0x6c/0x78 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a2921>] >wake_bit_function+0x0/0x23 >Jun 2 13:03:46 hidden kernel: [<ffffffff880339a5>] >:jbd:journal_commit_transaction+0x543/0x1066 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:03:46 hidden kernel: [<ffffffff8004b3fb>] >try_to_del_timer_sync+0x7f/0x88 >Jun 2 13:03:46 hidden kernel: [<ffffffff880375d3>] >:jbd:kjournald+0xc1/0x213 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:03:46 hidden kernel: [<ffffffff88037512>] >:jbd:kjournald+0x0/0x213 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:03:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132 >Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11 >Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>] >keventd_create_kthread+0x0/0xc4 >Jun 2 13:03:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132 >Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11 >Jun 2 13:03:46 hidden kernel: >Jun 2 13:03:46 hidden kernel: INFO: task oracle:20274 blocked for more >than >120 seconds. >Jun 2 13:03:46 hidden kernel: "echo 0 > >/proc/sys/kernel/hung_task_timeout_secs" disables this message. >Jun 2 13:03:46 hidden kernel: oracle D ffff810021025e20 0 >20274 >1 20276 20272 (NOTLB) >Jun 2 13:03:46 hidden kernel: ffff810270fedcf8 0000000000000086 >ffff810270fedce0 ffff810270fedca8 >Jun 2 13:03:46 hidden kernel: 0000000000000000 0000000000000009 >ffff81037f726820 ffff81042fe2a080 >Jun 2 13:03:46 hidden kernel: 0002c8c7206b57d6 000000000003d4c1 >ffff81037f726a08 0000000470fedd18 >Jun 2 13:03:46 hidden kernel: Call Trace: >Jun 2 13:03:46 hidden kernel: [<ffffffff88032002>] >:jbd:start_this_handle+0x2e5/0x36c >Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>] >autoremove_wake_function+0x0/0x2e >Jun 2 13:03:46 hidden kernel: [<ffffffff88032152>] >:jbd:journal_start+0xc9/0x100 >Jun 2 13:03:46 hidden kernel: [<ffffffff8805389c>] >:ext3:ext3_create+0x49/0x102 >Jun 2 13:03:46 hidden kernel: [<ffffffff8003a86a>] >vfs_create+0xe6/0x158 >Jun 2 13:03:46 hidden kernel: [<ffffffff8001b2d5>] >open_namei+0x19d/0x712 >Jun 2 13:03:46 hidden kernel: [<ffffffff80027660>] >do_filp_open+0x1c/0x38 >Jun 2 13:03:46 hidden kernel: [<ffffffff8001a061>] >do_sys_open+0x44/0xbe >Jun 2 13:03:46 hidden kernel: [<ffffffff8005d28d>] tracesys+0xd5/0xe0 >Jun 2 13:03:46 hidden kernel: -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -- redhat-list mailing list unsubscribe mailto:redhat-list-request@xxxxxxxxxx?subject=unsubscribe https://www.redhat.com/mailman/listinfo/redhat-list
