Re: crash tool not working

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




----- Original Message -----
> Hi List,
> 
> I am new to crash / kdump tool and failing some problem as mentioned
> below.
> 
> I am referring the Linux Kernel Crash Book (http://www.dedoimedo.com/)
> and URL http://www.dedoimedo.com/computers/crash.html.

Well, that's your first mistake...

The section in that book that has caused your confusion is this part:

| You can use the old way. Here's an example on CentOS 5.4:
|
| crash /boot/System.map-2.6.18-164.10.1.el5 /boot/vmlinuz-2.6.18-164.10.1.el5 vmcore
|
| Notice the use of vmlinuz kernel image, as opposed to vmlinux previously required.

Given CentOS 5.4 is a copy of RHEL5.4, I cannot even imagine where the
author got the impression that the /boot/vmlinuz-<release> file
could *ever* have been used by the crash utility.  The vmlinuz file is
a stripped and compressed output from the vmlinux file, and is completely
unsuitable for use by the crash utility.  His statement above is just 
complete fanatasy.  The crash utility needs an unstripped vmlinux file
containing the debuginfo sections for the base kernel.  He does subsequently
state that "alternatively" you can use vmlinux file:

| Alternatively, you can use only the debug information under /usr/lib/debug. The 
| information is extracted during the installation of kernel-debuginfo packages 
| matching the kernel that was running at the time of the kernel crash. 

But in reality -- that is the only file that can be used with the
crash utility, at least from a RHEL5/CentOS perspective.  I can't
speak for any other distribution.

> I am building modified kernel source (2.6.32 based) and added my
> modules (for study purpose). Building, installing and booting kernel
> is successful. I have enabled the options for kdump as mentioned in
> the book:
> 
> Enable Kexec system call:
> CONFIG_KEXEC=y
> 
> Enable kernel crash dumps:
> CONFIG_CRASH_DUMP=y
> 
> Optional: Disable Symmetric Multi-Processing (SMP) support
> CONFIG_SMP=y
> 
> Enable sysfs file system support:
> CONFIG_SYSFS=y
> 
> Enable /proc/vmcore support:
> CONFIG_PROC_VMCORE=y
> 
> Configure the kernel with debug info:
> CONFIG_DEBUG_INFO=y
> 
> Configure the start section for reserved RAM for the crash kernel:
> CONFIG_PHYSICAL_START=0x200000             (2MB)
> 
> Configure kdump kernel so it can be identified:
> CONFIG_LOCALVERSION="-crash"
> 
> 
> Kdump configuration /etc/sysconfig/kdump:
> KDUMP_KERNELVER=""
> 
> KDUMP_COMMANDLINE=""
> 
> KDUMP_COMMANDLINE_APPEND="maxcpus=1 "
> 
> KEXEC_OPTIONS=""
> 
> KDUMP_IMMEDIATE_REBOOT="yes"
> 
> KDUMP_TRANSFER=""
> 
> KDUMP_SAVEDIR="file:///var/crash"
> 
> KDUMP_KEEP_OLD_DUMPS="5"
> 
> KDUMP_FREE_DISK_SIZE="64"
> 
> KDUMP_VERBOSE="3"
> 
> KDUMP_DUMPLEVEL="0"
> 
> KDUMP_DUMPFORMAT="compressed"
> 
> There is no option KDUMP_DUMPDEV option
> There is no option KDUMP_RUNLEVEL
> 
> I booted successfully with this kernel and tried to crash it by
> module. After rebooting, I found that vmcore is generated under
> /var/crash/ But I am not able to analyze it with crash command.
> 
> linux:/home/adil # cat /proc/cmdline
> root=/dev/disk/by-id/ata-WDC_WD800BD-22LRA1_WD-WMAM9ZS19445-part1
> resume=/dev/disk/by-id/ata-WDC_WD800BD-22LRA1_WD-WMAM9ZS19445-part2
> splash=silent crashkernel=256M-:128M vga=0x31a
> linux:/home/adil #
> 
> linux:/home/adil # crash /boot/System.map-2.6.32.12-crash-crash
> /boot/vmlinuz-2.6.32.12-crash-crash
> 
> crash 5.0.1
> Copyright (C) 2002-2010  Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006  IBM Corporation
> Copyright (C) 1999-2006  Hewlett-Packard Co
> Copyright (C) 2005, 2006  Fujitsu Limited
> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
> Copyright (C) 2005  NEC Corporation
> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions.  Enter "help copying" to see the conditions.
> This program has absolutely no warranty.  Enter "help warranty" for
> details.
> 
> crash: /boot/vmlinuz-2.6.32.12-crash-crash: not a supported file format
> 
> Usage:
>   crash [-h [opt]][-v][-s][-i file][-d num] [-S] [mapfile] [namelist]
>   [dumpfile]
> 
> Enter "crash -h" for details.
> linux:/home/adil #

Like the error message indicates, the vmlinuz-2.6.32.12-crash-crash
file is not a supported file format.  Never has been, never will be...

> 
> The URL http://www.dedoimedo.com/computers/crash.html mentioned that
> "The newer versions of Kdump can work with compressed kernel images.
> Furthermore, they copy the System map file and the kernel image into
> the crash directory, making the use of crash utility somewhat
> simpler."

What has been supported since crash-5.1.3 is the usage of
vmlinux files that have been subsequently compressed with either
gzip or bzip2.  But again, the /boot/vmlinuz-<release> file is
a completely different animal.
 
> linux:/home/adil # ls -al /var/crash/2012-01-30-18\:08/
> total 1335536
> drwxr-xr-x 2 root root       4096 2012-01-30 18:13 .
> drwxr-xr-x 8 root root       4096 2012-01-31 12:17 ..
> -rw-r--r-- 1 root root        187 2012-01-30 18:13 README.txt
> -rw-r--r-- 1 root root    1716605 2012-01-30 18:13 System.map-2.6.32.12-0.7-default
> -rw------- 1 root root 1360732590 2012-01-30 18:13 vmcore
> -rw-r--r-- 1 root root    3774506 2012-01-30 18:13 vmlinux-2.6.32.12-0.7-default.gz
> linux:/home/adil # ls -al /var/crash/2012-01-31-12\:17/
> total 1343860
> drwxr-xr-x 2 root root       4096 2012-01-31 12:24 .
> drwxr-xr-x 8 root root       4096 2012-01-31 12:17 ..
> -rw-r--r-- 1 root root        187 2012-01-31 12:24 README.txt
> -rw------- 1 root root 1374748735 2012-01-31 12:24 vmcore
> linux:/home/adil #
> 
> linux:/home/adil # ls /boot/
> backup_mbr
> boot
> boot.readme
> config-2.6.32.12-0.7-default
> config-2.6.32.12-0.7-xen
> grub
> initrd
> initrd-2.6.32.12-0.7-default
> initrd-2.6.32.12-0.7-default-kdump
> initrd-2.6.32.12-0.7-xen
> initrd-2.6.32.12-crash-crash
> initrd-2.6.32.12-crash-crash-kdump
> initrd-xen
> message
> symsets-2.6.32.12-0.7-default.tar.gz
> symtypes-2.6.32.12-0.7-default.gz
> symvers-2.6.32.12-0.7-default.gz
> symvers-2.6.32.12-0.7-xen.gz
> System.map-2.6.32.12-0.7-default
> System.map-2.6.32.12-0.7-xen
> System.map-2.6.32.12-crash-crash
> vmlinux-2.6.32.12-0.7-xen.gz
> vmlinuz
> vmlinuz-2.6.32.12-0.7-default
> vmlinuz-2.6.32.12-0.7-xen
> vmlinuz-2.6.32.12-crash-crash
> vmlinuz-xen
> vmlinux-2.6.32.12-0.7-default.gz
> linux:/home/adil #
> 
> 
> Another observation is boot.kdump seems to on but manually start
> giving me error:
> 
> linux:/home/adil # chkconfig boot.kdump
> boot.kdump  on
> linux:/home/adil #
> 
> linux:/home/adil # /etc/init.d/boot.kdump start
> Loading kdump
> Regenerating kdump initrd ...
> Can't find kernel text map area from kcore
> Cannot load /boot/vmlinuz-2.6.32.12-crash-crash
> 
>                                                     failed
> linux:/home/adil #
> 
> Other Query: Following is not clear mentioned in the book under
> "section 11.2 Crash (capture) kernel":
> --------------
> This means that while your production kernels will most likely be
> named vmlinuz, the Kdump crash kernels need to be uncompressed,
> hence named vmlinux, or rather vmlinux-kdump.
> ---------------
> 
> Please help how to correctly setup and use crash on my machine.

With respect to the crash utility, if your kernel build procedure
has created a /usr/lib/debug/lib/modules/<kernel-version>/vmlinux,
then you don't need a System.map file, but rather:

 $ crash /usr/lib/debug/lib/modules/<kernel-version>/vmlinux vmcore

If you don't have the file above, and if whatever non-RHEL system you're
running actually puts the vmlinux-<version>.gz file in the /var/crash 
subdirectory with the vmcore, then use that file:
 
 $ crash vmlinux-<kernel-version>.gz vmcore

You may also need the System.map file found there as well.  I believe
Suse post-builds their debuginfo kernels instead of saving the vmlinux
file from when it was originally built.  Could be that their version
of kexec-tools does the copy of the extra files into /var/crash?
I really don't know -- in a RHEL system, only the vmcore files are
copied there.

Anyway, if you have neither of the above, then check your modified kernel
build tree.  In the top-level directory, there should be a "vmlinux" file,
which is the file that the crash utility requires.  This also presumes that
you built the kernel with -g, which would get turned on when the kernel
is configured with CONFIG_DEBUG_INFO.  However, it should be noted
that if your kernel build procedure built more than one kernel "flavor",
then the vmlinux file found there will be that of the last kernel that
was built -- which may not be the one you need. 

For example, RHEL5 x86_64 builds three x86_64 kernel rpms:
 
  kernel-2.6.18-232.el5.x86_64.rpm
  kernel-debug-2.6.18-232.el5.x86_64.rpm
  kernel-xen-2.6.18-232.el5.x86_64.rpm

The rpm files above contain the /boot/vmlinuz-<release> files.
  
Then, each kernel above has a debuginfo package associated
with it, which contains its original unstripped vmlinux file,
which gets put in /usr/lib/debug/lib/modules/<kernel-version>/vmlinux:

  kernel-debuginfo-2.6.18-232.el5.x86_64.rpm
  kernel-debug-debuginfo-2.6.18-232.el5.x86_64.rpm
  kernel-xen-debuginfo-2.6.18-232.el5.x86_64.rpm

And there is a "common" debuginfo file that is typically installed
with the kernel-specific file above:

  kernel-debuginfo-common-2.6.18-232.el5.x86_64.rpm

Again, I can't speak for the procedures used by any other distribution.

Also, do yourself a favor, and upgrade your crash utility version
from crash-5.0.1, which is two years old.  

And lastly, although you may get some responses, this is not the list for 
kdump-related questions.  That is found here:

  http://lists.infradead.org/mailman/listinfo/kexec

Dave Anderson

--
Crash-utility mailing list
Crash-utility@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/crash-utility


[Index of Archives]     [Fedora Development]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]     [Fedora Tools]

 

Powered by Linux