I have a machine with a recent install of centos
(2.6.32-220.2.1.el6.x86_64 kernel). It crashed 3 times this week, and
sent emails like the one below complaining about kernel taint. I've
gone back to a previous kernel to see if that helps, but otherwise I
don't know how to investigate this. What should I do?
Here's stuff from the log and one of the emails:
log:********************************
# grep -e '12 10:16' -e '10 14:46' -e '9 15:44' /var/log/messages
Jan 9 15:44:11 name kernel: ------------[ cut here ]------------
Jan 9 15:44:11 name kernel: WARNING: at kernel/sched.c:5914
thread_return+0x232/0x79d() (Not tainted)
Jan 9 15:44:11 name kernel: Hardware name: Precision WorkStation 490
Jan 9 15:44:11 name kernel: Modules linked in: fuse nfs lockd fscache
nfs_acl auth_rpcgss autofs4 sunrpc p4_clockmod freq_table
speedstep_lib ipv6 ppdev parport_pc parport sg microcode dcdbas
serio_raw i2c_i801 iTCO_wdt iTCO_vendor_support tg3 snd_hda_codec_idt
snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm
snd_timer snd soundcore snd_page_alloc i5000_edac edac_core i5k_amb
shpchp ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom firewire_ohci
firewire_core crc_itu_t ahci pata_acpi ata_generic ata_piix nouveau
ttm drm_kms_helper drm i2c_algo_bit i2c_core mxm_wmi wmi video output
dm_mirror dm_region_hash dm_log dm_mod [last unloaded: mperf]
Jan 9 15:44:11 name kernel: Pid: 17, comm: ksoftirqd/3 Not tainted
2.6.32-220.2.1.el6.x86_64 #1
Jan 9 15:44:11 name kernel: Call Trace:
Jan 9 15:44:11 name kernel: [<ffffffff81069997>] ?
warn_slowpath_common+0x87/0xc0
Jan 9 15:44:11 name kernel: [<ffffffff810699ea>] ? warn_slowpath_null+0x1a/0x20
Jan 9 15:44:11 name kernel: [<ffffffff814eccc5>] ? thread_return+0x232/0x79d
Jan 9 15:44:11 name kernel: [<ffffffff81071b35>] ? ksoftirqd+0xd5/0x110
Jan 9 15:44:11 name kernel: [<ffffffff81071a60>] ? ksoftirqd+0x0/0x110
Jan 9 15:44:11 name kernel: [<ffffffff810906a6>] ? kthread+0x96/0xa0
Jan 9 15:44:11 name kernel: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
Jan 9 15:44:11 name kernel: [<ffffffff81090610>] ? kthread+0x0/0xa0
Jan 9 15:44:11 name kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20
Jan 9 15:44:11 name kernel: ---[ end trace ce943d42785388df ]---
Jan 9 15:44:12 name abrtd: Directory
'oops-2012-01-09-15:44:12-1647-0' creation detected
Jan 9 15:44:12 name abrt-dump-oops: Reported 1 kernel oopses to Abrt
Jan 9 15:44:12 name abrtd: Can't open file
'/var/spool/abrt/oops-2012-01-09-15:44:12-1647-0/uid': No such file or
directory
Jan 9 15:44:20 name kernel: Bridge firewalling registered
Jan 9 15:44:32 name abrtd: Sending an email...
Jan 9 15:44:32 name abrtd: Email was sent to: root@localhost
Jan 9 15:44:32 name abrtd: New dump directory
/var/spool/abrt/oops-2012-01-09-15:44:12-1647-0, processing
Jan 10 14:46:44 name kernel: ------------[ cut here ]------------
Jan 10 14:46:44 name kernel: WARNING: at kernel/sched.c:5914
thread_return+0x232/0x79d() (Not tainted)
Jan 10 14:46:44 name kernel: Hardware name: Precision WorkStation 490
Jan 10 14:46:44 name kernel: Modules linked in: fuse nfs lockd fscache
nfs_acl auth_rpcgss autofs4 sunrpc p4_clockmod freq_table
speedstep_lib ipv6 ppdev parport_pc parport sg microcode dcdbas
serio_raw i2c_i801 iTCO_wdt iTCO_vendor_support tg3 snd_hda_codec_idt
snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm
snd_timer snd soundcore snd_page_alloc i5000_edac edac_core i5k_amb
shpchp ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom firewire_ohci
firewire_core crc_itu_t ahci pata_acpi ata_generic ata_piix nouveau
ttm drm_kms_helper drm i2c_algo_bit i2c_core mxm_wmi wmi video output
dm_mirror dm_region_hash dm_log dm_mod [last unloaded: mperf]
Jan 10 14:46:44 name kernel: Pid: 2304, comm: thunderbird-bin Not
tainted 2.6.32-220.2.1.el6.x86_64 #1
Jan 10 14:46:44 name kernel: Call Trace:
Jan 10 14:46:44 name kernel: [<ffffffff81069997>] ?
warn_slowpath_common+0x87/0xc0
Jan 10 14:46:44 name kernel: [<ffffffff810699ea>] ? warn_slowpath_null+0x1a/0x20
Jan 10 14:46:44 name kernel: [<ffffffff814eccc5>] ? thread_return+0x232/0x79d
Jan 10 14:46:44 name kernel: [<ffffffff81051dc6>] ? enqueue_task+0x66/0x80
Jan 10 14:46:44 name kernel: [<ffffffff814ee50d>] ?
schedule_hrtimeout_range+0x13d/0x160
Jan 10 14:46:44 name kernel: [<ffffffff81090d76>] ? add_wait_queue+0x46/0x60
Jan 10 14:46:44 name kernel: [<ffffffff8118b895>] ? __pollwait+0x75/0xf0
Jan 10 14:46:44 name kernel: [<ffffffff8118b079>] ?
poll_schedule_timeout+0x39/0x60
Jan 10 14:46:44 name kernel: [<ffffffff8118bdcb>] ? do_sys_poll+0x45b/0x520
Jan 10 14:46:44 name kernel: [<ffffffff8118b820>] ? __pollwait+0x0/0xf0
Jan 10 14:46:44 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 10 14:46:44 name kernel: [<ffffffff8141be4e>] ? sock_aio_write+0x15e/0x170
Jan 10 14:46:44 name kernel: [<ffffffff8141bcf0>] ? sock_aio_write+0x0/0x170
Jan 10 14:46:44 name kernel: [<ffffffff81175fbb>] ?
do_sync_readv_writev+0xfb/0x140
Jan 10 14:46:44 name kernel: [<ffffffff81090a10>] ?
autoremove_wake_function+0x0/0x40
Jan 10 14:46:44 name kernel: [<ffffffff81218def>] ?
selinux_file_permission+0xbf/0x150
Jan 10 14:46:44 name kernel: [<ffffffff811770e2>] ? do_readv_writev+0x162/0x1f0
Jan 10 14:46:44 name kernel: [<ffffffff8120c1d6>] ?
security_file_permission+0x16/0x20
Jan 10 14:46:44 name kernel: [<ffffffff8118c08c>] ? sys_poll+0x7c/0x110
Jan 10 14:46:44 name kernel: [<ffffffff8100b0f2>] ?
system_call_fastpath+0x16/0x1b
Jan 10 14:46:44 name kernel: ---[ end trace f9f851f2bb55fcd3 ]---
Jan 10 14:46:45 name abrtd: Directory
'oops-2012-01-10-14:46:45-1649-0' creation detected
Jan 10 14:46:45 name abrt-dump-oops: Reported 1 kernel oopses to Abrt
Jan 10 14:46:45 name abrtd: Can't open file
'/var/spool/abrt/oops-2012-01-10-14:46:45-1649-0/uid': No such file or
directory
Jan 10 14:46:50 name kernel: Bridge firewalling registered
Jan 12 10:16:39 name kernel: ------------[ cut here ]------------
Jan 12 10:16:39 name kernel: WARNING: at kernel/sched.c:5914
thread_return+0x232/0x79d() (Not tainted)
Jan 12 10:16:39 name kernel: Hardware name: Precision WorkStation 490
Jan 12 10:16:39 name kernel: Modules linked in: fuse nfs lockd fscache
nfs_acl auth_rpcgss autofs4 sunrpc p4_clockmod freq_table
speedstep_lib ipv6 ppdev parport_pc parport sg microcode dcdbas
serio_raw i2c_i801 iTCO_wdt iTCO_vendor_support tg3 snd_hda_codec_idt
snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm
snd_timer snd soundcore snd_page_alloc i5000_edac edac_core i5k_amb
shpchp ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom firewire_ohci
firewire_core crc_itu_t ahci pata_acpi ata_generic ata_piix nouveau
ttm drm_kms_helper drm i2c_algo_bit i2c_core mxm_wmi wmi video output
dm_mirror dm_region_hash dm_log dm_mod [last unloaded: mperf]
Jan 12 10:16:39 name kernel: Pid: 1734, comm: Xorg Not tainted
2.6.32-220.2.1.el6.x86_64 #1
Jan 12 10:16:39 name kernel: Call Trace:
Jan 12 10:16:39 name kernel: [<ffffffff81069997>] ?
warn_slowpath_common+0x87/0xc0
Jan 12 10:16:39 name kernel: [<ffffffff810699ea>] ? warn_slowpath_null+0x1a/0x20
Jan 12 10:16:39 name kernel: [<ffffffff814eccc5>] ? thread_return+0x232/0x79d
Jan 12 10:16:39 name kernel: [<ffffffff810958e3>] ?
__hrtimer_start_range_ns+0x1a3/0x460
Jan 12 10:16:39 name kernel: [<ffffffff81094dc1>] ? lock_hrtimer_base+0x31/0x60
Jan 12 10:16:39 name kernel: [<ffffffff814ee498>] ?
schedule_hrtimeout_range+0xc8/0x160
Jan 12 10:16:39 name kernel: [<ffffffff81094b70>] ? hrtimer_wakeup+0x0/0x30
Jan 12 10:16:39 name kernel: [<ffffffff81095bd4>] ?
hrtimer_start_range_ns+0x14/0x20
Jan 12 10:16:39 name kernel: [<ffffffff8118b079>] ?
poll_schedule_timeout+0x39/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b6e8>] ? do_select+0x578/0x6b0
Jan 12 10:16:39 name kernel: [<ffffffff8104d80d>] ? check_preempt_curr+0x6d/0x90
Jan 12 10:16:39 name kernel: [<ffffffff8118b820>] ? __pollwait+0x0/0xf0
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118b910>] ? pollwake+0x0/0x60
Jan 12 10:16:39 name kernel: [<ffffffff8118c30a>] ? core_sys_select+0x18a/0x2c0
Jan 12 10:16:39 name kernel: [<ffffffff81090a10>] ?
autoremove_wake_function+0x0/0x40
Jan 12 10:16:39 name kernel: [<ffffffff81277141>] ? __clear_user+0x21/0x70
Jan 12 10:16:39 name kernel: [<ffffffff811770e2>] ? do_readv_writev+0x162/0x1f0
Jan 12 10:16:39 name kernel: [<ffffffff81012b59>] ? read_tsc+0x9/0x20
Jan 12 10:16:39 name kernel: [<ffffffff8109b629>] ? ktime_get_ts+0xa9/0xe0
Jan 12 10:16:39 name kernel: [<ffffffff810d41b5>] ?
__audit_syscall_exit+0x265/0x290
Jan 12 10:16:39 name kernel: [<ffffffff8118c697>] ? sys_select+0x47/0x110
Jan 12 10:16:39 name kernel: [<ffffffff8100b0f2>] ?
system_call_fastpath+0x16/0x1b
Jan 12 10:16:39 name kernel: ---[ end trace 9c5478b05024d9ed ]---
Jan 12 10:16:40 name abrtd: Directory
'oops-2012-01-12-10:16:40-1660-0' creation detected
Jan 12 10:16:40 name abrt-dump-oops: Reported 1 kernel oopses to Abrt
Jan 12 10:16:40 name abrtd: Can't open file
'/var/spool/abrt/oops-2012-01-12-10:16:40-1660-0/uid': No such file or
directory
Jan 12 10:16:47 name kernel: Bridge firewalling registered
>From email: **************************************
Duplicate check
=====
Common information
=====
package
-----
kernel
architecture
-----
x86_64
kernel
-----
2.6.32-220.2.1.el6.x86_64
Additional information
=====
kernel_tainted_long
-----
Taint on warning.
kernel_tainted
-----
512
backtrace
-----
WARNING: at kernel/sched.c:5914 thread_return+0x232/0x79d() (Not tainted)
Hardware name: Precision WorkStation 490
Modules linked in: fuse nfs lockd fscache nfs_acl auth_rpcgss autofs4
sunrpc p4_clockmod freq_table speedstep_lib ipv6 ppdev parport_pc
parport sg microcode dcdbas serio_raw i2c_i801 iTCO_wdt
iTCO_vendor_support tg3 snd_hda_codec_idt snd_hda_intel snd_hda_codec
snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd soundcore
snd_page_alloc i5000_edac edac_core i5k_amb shpchp ext4 mbcache jbd2
sd_mod crc_t10dif sr_mod cdrom firewire_ohci firewire_core crc_itu_t
ahci pata_acpi ata_generic ata_piix nouveau ttm drm_kms_helper drm
i2c_algo_bit i2c_core mxm_wmi wmi video output dm_mirror
dm_region_hash dm_log dm_mod [last unloaded: mperf]
Pid: 1734, comm: Xorg Not tainted 2.6.32-220.2.1.el6.x86_64 #1
Call Trace:
[<ffffffff81069997>] ? warn_slowpath_common+0x87/0xc0
[<ffffffff810699ea>] ? warn_slowpath_null+0x1a/0x20
[<ffffffff814eccc5>] ? thread_return+0x232/0x79d
[<ffffffff810958e3>] ? __hrtimer_start_range_ns+0x1a3/0x460
[<ffffffff81094dc1>] ? lock_hrtimer_base+0x31/0x60
[<ffffffff814ee498>] ? schedule_hrtimeout_range+0xc8/0x160
[<ffffffff81094b70>] ? hrtimer_wakeup+0x0/0x30
[<ffffffff81095bd4>] ? hrtimer_start_range_ns+0x14/0x20
[<ffffffff8118b079>] ? poll_schedule_timeout+0x39/0x60
[<ffffffff8118b6e8>] ? do_select+0x578/0x6b0
[<ffffffff8104d80d>] ? check_preempt_curr+0x6d/0x90
[<ffffffff8118b820>] ? __pollwait+0x0/0xf0
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118b910>] ? pollwake+0x0/0x60
[<ffffffff8118c30a>] ? core_sys_select+0x18a/0x2c0
[<ffffffff81090a10>] ? autoremove_wake_function+0x0/0x40
[<ffffffff81277141>] ? __clear_user+0x21/0x70
[<ffffffff811770e2>] ? do_readv_writev+0x162/0x1f0
[<ffffffff81012b59>] ? read_tsc+0x9/0x20
[<ffffffff8109b629>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff810d41b5>] ? __audit_syscall_exit+0x265/0x290
[<ffffffff8118c697>] ? sys_select+0x47/0x110
[<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
hostname
-----
airy
component
-----
kernel
reason
-----
WARNING: at kernel/sched.c:5914 thread_return+0x232/0x79d() (Not tainted)
cmdline
-----
ro root=/dev/mapper/vg_iprctmp9-lv_root rd_LVM_LV=vg_iprctmp9/lv_root
rd_LVM_LV=vg_iprctmp9/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM
LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
crashkernel=128M rhgb quiet
kernel_tainted_short
-----
---------W
analyzer
-----
Kerneloops
time
-----
1326399400
os_release
-----
CentOS release 6.2 (Final)
--
Q: Why should this email be 5 sentences or less?
A: http://five.sentenc.es
IPRC-help FAQ: http://auana1.soest.hawaii.edu/wiki/index.php/Faq
--
Q: Why should this email be 5 sentences or less?
A: http://five.sentenc.es
IPRC-help FAQ: http://auana1.soest.hawaii.edu/wiki/index.php/Faq
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
