troubleshooting kernel crash?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



I'm getting kernel crashes on a new machine since it went into production 
some weeks ago. How can I troubleshoot this? I suspect a hardware problem, 
but I know too few about the kernel and kernel debugging on Linux to know 
how I can nail this down with debugging software.

Just for reference of what kind these oopses are I quote the last two. The 
symptom is that the machine stops responding but is pingable. After a 
reset all is well until the next crash in ten days or so.

Apr 29 15:05:07 nx05 kernel: c014802d
Apr 29 15:05:07 nx05 kernel: Modules linked in: nls_utf8 cifs smbfs 
ipt_REJECT ipt_limit ipt_state ipt_LOG iptable_filter ip_tables 
ip_conntrack_ftp ip_conntrack md5 ipv6 autofs4 sunrpc dm_mirror dm_mod 
button battery ac 8139too mii ext3 jbd ata_piix libata sd_mod scsi_mod
Apr 29 15:05:07 nx05 kernel: CPU:    0
Apr 29 15:05:07 nx05 kernel: EIP:    0060:[<c014802d>]    Not tainted VLI
Apr 29 15:05:07 nx05 kernel: EFLAGS: 00010006   (2.6.9-34.EL)
Apr 29 15:05:07 nx05 kernel: EIP is at find_get_page+0x73/0xdd
Apr 29 15:05:07 nx05 kernel: eax: 00000200   ebx: df002bf4   ecx: 00000200 
  edx: 00000200
Apr 29 15:05:07 nx05 kernel: esi: df002bf4   edi: 00000000   ebp: c1b64d1c 
  esp: c1b64cf0
Apr 29 15:05:07 nx05 kernel: ds: 007b   es: 007b   ss: 0068
Apr 29 15:05:07 nx05 kernel: Process httpd (pid: 24805, 
threadinfo=c1b64000 task=d308ad50)
Apr 29 15:05:07 nx05 kernel: Stack: 00000000 c016a705 0029005a 00000000 
00000001 00000000 00000000 df002b14
Apr 29 15:05:07 nx05 kernel:        00000000 0029005a 00000000 df002a80 
c016bdb7 00001000 00000000 df73b800
Apr 29 15:05:07 nx05 kernel:        0029005a 00000000 df002a80 c016bde6 
00001000 df73b800 cfc14bf4 00000000
Apr 29 15:05:07 nx05 kernel: Call Trace:
Apr 29 15:05:07 nx05 kernel:  [<c016a705>] 
__find_get_block_slow+0x4b/0x1c6
Apr 29 15:05:07 nx05 kernel:  [<c016bdb7>] __find_get_block+0x89/0xa5
Apr 29 15:05:07 nx05 kernel:  [<c016bde6>] __getblk+0x13/0x49
Apr 29 15:05:07 nx05 kernel:  [<e09298e3>] ext3_get_inode_loc+0x4f/0x223 
[ext3]
Apr 29 15:05:07 nx05 kernel:  [<e0929b45>] ext3_read_inode+0x38/0x309 
[ext3]
Apr 29 15:05:07 nx05 kernel:  [<c030fbf0>] __cond_resched+0x14/0x3b
Apr 29 15:05:07 nx05 kernel:  [<e092e1b7>] ext3_alloc_inode+0xf/0x46 
[ext3]
Apr 29 15:05:07 nx05 kernel:  [<c0185596>] alloc_inode+0xf6/0x17f
Apr 29 15:05:07 nx05 kernel:  [<c018662e>] get_new_inode_fast+0xa5/0x1e9
Apr 29 15:05:07 nx05 kernel:  [<e092b9a0>] ext3_lookup+0x55/0x87 [ext3]
Apr 29 15:05:07 nx05 kernel:  [<c0177d32>] real_lookup+0x73/0xde
Apr 29 15:05:07 nx05 kernel:  [<c0178062>] do_lookup+0x56/0x8f
Apr 29 15:05:07 nx05 kernel:  [<c0178ad4>] __link_path_walk+0xa39/0xd98
Apr 29 15:05:07 nx05 kernel:  [<c0178e74>] link_path_walk+0x41/0xb9
Apr 29 15:05:07 nx05 kernel:  [<c011e867>] 
autoremove_wake_function+0x0/0x2d
Apr 29 15:05:07 nx05 kernel:  [<c017916c>] path_lookup+0x104/0x135
Apr 29 15:05:07 nx05 kernel:  [<c01792b1>] __user_walk+0x21/0x51
Apr 29 15:05:07 nx05 kernel:  [<c01734b8>] vfs_stat+0x14/0x3a
Apr 29 15:05:07 nx05 kernel:  [<c011e867>] 
autoremove_wake_function+0x0/0x2d
Apr 29 15:05:07 nx05 kernel:  [<c0173ac1>] sys_stat64+0xf/0x23
Apr 29 15:05:07 nx05 kernel:  [<c0168b76>] vfs_read+0xda/0xe2
Apr 29 15:05:07 nx05 kernel:  [<c0168d65>] sys_read+0x3c/0x62
Apr 29 15:05:07 nx05 kernel:  [<c0311443>] syscall_call+0x7/0xb
Apr 29 15:05:07 nx05 kernel:  [<c031007b>] 
rwsem_down_read_failed+0x19f/0x204
Apr 29 15:05:07 nx05 kernel: Code: c0 e8 77 8d fd ff c7 43 14 01 00 00 00 
8d 43 04 c7 43 20 54 1f 32 c0 c7 43 24 0e 02 00 00 e8 1e c5 09 00 85 c0 89 
c1 74 0f 89 c2 <8b> 00 f6 c4 80 74 03 8b 51 0c ff 42 04 81 7b 10 3c 4b 24 
1d 74

Apr 20 03:55:31 nx05 kernel: c0167d59
Apr 20 03:55:31 nx05 kernel: Modules linked in: smbfs ipt_REJECT ipt_limit 
ipt_state ipt_LOG ip_conntrack_ftp ip_conntrack iptable_filter ip_tables 
parport_pc lp parport md5 ipv6 autofs4 sunrpc dm_mirror dm_mod button 
battery ac 8139too mii ext3 jbd ata_piix libata sd_mod scsi_mod
Apr 20 03:55:31 nx05 kernel: CPU:    0
Apr 20 03:55:31 nx05 kernel: EIP:    0060:[<c0167d59>]    Not tainted VLI
Apr 20 03:55:31 nx05 kernel: EFLAGS: 00010286   (2.6.9-34.EL)
Apr 20 03:55:31 nx05 kernel: EIP is at __dentry_open+0x62/0x16a
Apr 20 03:55:31 nx05 kernel: eax: a093ffa0   ebx: dfab0680   ecx: 0000000d 
  edx: dfe51100
Apr 20 03:55:31 nx05 kernel: esi: cd1dd78c   edi: dfe51100   ebp: 00000000 
  esp: dda19f30
Apr 20 03:55:31 nx05 kernel: ds: 007b   es: 007b   ss: 0068
Apr 20 03:55:31 nx05 kernel: Process mysqld (pid: 12922, 
threadinfo=dda19000 task=dee2edd0)
Apr 20 03:55:31 nx05 kernel: Stack: cae8a978 00000000 dfab0680 00008000 
00000000 c0167c96 dfab0680 caeac000
Apr 20 03:55:31 nx05 kernel:        cae8a978 dfe51100 dda19f58 dec82680 
dda19f88 00000101 00000001 00000000
Apr 20 03:55:31 nx05 kernel:        00001000 b0d4e780 dda19f80 c030fbf0 
caeac000 c01e67f2 00000000 00000039
Apr 20 03:55:31 nx05 kernel: Call Trace:
Apr 20 03:55:31 nx05 kernel:  [<c0167c96>] filp_open+0x5c/0x70
Apr 20 03:55:31 nx05 kernel:  [<c030fbf0>] __cond_resched+0x14/0x3b
Apr 20 03:55:31 nx05 kernel:  [<c01e67f2>] 
direct_strncpy_from_user+0x3e/0x5d
Apr 20 03:55:31 nx05 kernel:  [<c016819f>] sys_open+0x31/0x7d
Apr 20 03:55:31 nx05 kernel:  [<c0311443>] syscall_call+0x7/0xb
Apr 20 03:55:31 nx05 kernel:  [<c031007b>] 
rwsem_down_read_failed+0x19f/0x204
Apr 20 03:55:31 nx05 kernel: Code: dc 00 00 00 89 83 a0 00 00 00 8b 04 24 
89 7b 0c c7 43 24 00 00 00 00 89 43 08 c7 43 28 00 00 00 00 8b 86 d0 00 00 
00 85 c0 74 19 <8b> 00 85 c0 74 0b 83 38 02 74 0e ff 80 00 01 00 00 8b 86 
d0 00

Kai



[Index of Archives]     [CentOS]     [CentOS Announce]     [CentOS Development]     [CentOS ARM Devel]     [CentOS Docs]     [CentOS Virtualization]     [Carrier Grade Linux]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Linux USB]
  Powered by Linux