After getting around a few kdump kernel panic/hang, i finally was
able to complete a kdump vmcore with 2.6.27-rc5. The system under
testing was an IA64 with 128 cpu and 256G memory A4700 system.
The /proc/vmcore is:
a4700rac:/boot # ll /proc/vmcore
-r-------- 1 root root 263006257684 2008-09-10 14:45 /proc/vmcore
a4700rac:/boot # ls -lh /proc/vmcore
-r-------- 1 root root 245G 2008-09-10 14:44 /proc/vmcore
Time spent in saving the vmcore using cp was 7 min 17 sec:
a4700rac:/boot # date; cp /proc/vmcore /mnt/sda9/diskdump/vmcore-cp; date
Wed Sep 10 14:34:18 PDT 2008
Wed Sep 10 14:41:35 PDT 2008
Time spent with 'makedumpfile -c -d31' was 1 min 40 sec:
a4700rac:/boot # date; makedumpfile -c -d31 -x
/boot/vmlinux-2.6.27-rc5-default /proc/vmcore
/mnt/sda9/diskdump/vmcore-2.6.27-rc5-default; date
Wed Sep 10 14:31:56 PDT 2008
Can't distinguish the pgtable.
The kernel version is not supported.
The created dumpfile may be incomplete.
Copying data : [100 %]
The dumpfile is saved to /mnt/sda9/diskdump/vmcore-2.6.27-rc5-default.
makedumpfile Completed.
Wed Sep 10 14:33:36 PDT 2008
The fact that it took only 1 min 40 sec in running makedumpfile was
EXCELLENT and EXCITING!!! Remember last time i tested on a 256 cpu
1TB A4700? It took 18 hours to complete the makedumpfile. What an
improvement!
Hmmm, the reason it is only "almost there" was that crash failed
to analyze the output of makedumpfile. :( Crash was happy with
the vmcore saved with 'cp' command.
a4700rac:/var/tmp/jlan # crash -d 1 /boot/vmlinux-2.6.27-rc5-default
/mnt/sda9/diskdump/vmcore-2.6.27-rc5-default
crash 4.0-4.10
Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
crash: xc_core_elf_verify: not a xen ELF core file
diskdump_data:
flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
dfd: 3
ofp: 0
machine_type: 50 (EM_IA_64)
header: 6000000001142c70
signature: "KDUMP "
header_version: 1
utsname:
sysname:
nodename:
release:
version:
machine:
domainname:
timestamp:
tv_sec: 0
tv_usec: 0
status: 0 ()
block_size: 65536
sub_hdr_size: 1
bitmap_blocks: 2076
max_mapnr: 543813611
total_ram_blocks: 0
device_blocks: 0
written_blocks: 0
current_cpu: 0
nr_cpus: 1
tasks[nr_cpus]: 0
sub_header: 0 (n/a)
sub_header_kdump: 6000000001152c80
phys_base: 6044000000
dump_level: 31 (0x1f)
(DUMP_EXCLUDE_ZERO|DUMP_EXCLUDE_CACHE|DUMP_EXCLUDE_CACHE_PRI|DUMP_EXCLUDE_USER_DATA|DUMP_EXCLUDE_FREE)
data_offset: 81e0000
block_size: 65536
block_shift: 16
bitmap: 2000000000530010
bitmap_len: 136052736
dumpable_bitmap: 2000000008700010
byte: 0
bit: 0
compressed_page: 6000000001162c90
curbufptr: 0
page_cache_hdr[0]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 20000000109e0010
pg_hit_count: 0
page_cache_hdr[1]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 20000000109f0010
pg_hit_count: 0
page_cache_hdr[2]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a00010
pg_hit_count: 0
page_cache_hdr[3]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a10010
pg_hit_count: 0
page_cache_hdr[4]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a20010
pg_hit_count: 0
page_cache_hdr[5]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a30010
pg_hit_count: 0
page_cache_hdr[6]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a40010
pg_hit_count: 0
page_cache_hdr[7]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a50010
pg_hit_count: 0
page_cache_hdr[8]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a60010
pg_hit_count: 0
page_cache_hdr[9]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a70010
pg_hit_count: 0
page_cache_hdr[10]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a80010
pg_hit_count: 0
page_cache_hdr[11]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010a90010
pg_hit_count: 0
page_cache_hdr[12]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010aa0010
pg_hit_count: 0
page_cache_hdr[13]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010ab0010
pg_hit_count: 0
page_cache_hdr[14]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010ac0010
pg_hit_count: 0
page_cache_hdr[15]:
pg_flags: 0 ()
pg_addr: 0
pg_bufptr: 2000000010ad0010
pg_hit_count: 0
page_cache_buf: 20000000109e0010
evict_index: 0
evictions: 0
accesses: 0
cached_reads: 0
valid_pages: 20000000108d0010
compressed kdump: phys_start: 6044000000
gdb /boot/vmlinux-2.6.27-rc5-default
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "ia64-unknown-linux-gnu"...
crash: CONFIG_HZ: 250
crash: CONFIG_NR_CPUS: 512
verify_namelist:
/proc/version:
Linux version 2.6.27-rc5-default (jlan at jackhammer) (gcc version 4.1.2
20070115 (SUSE Linux)) #61 SMP Wed Sep 10 14:21:26 PDT 2008
utsname version: #61 SMP Wed Sep 10 14:21:26 PDT 2008
/boot/vmlinux-2.6.27-rc5-default:
Linux version 2.6.27-rc5-default (jlan at jackhammer) (gcc version 4.1.2
20070115 (SUSE Linux)) #61 SMP Wed Sep 10 14:21:26 PDT 2008
WARNING: Because this kernel was compiled with gcc version 4.1.2, certain
commands or command options may fail unless crash is invoked with
the "--readnow" command line option.
crash: get_cpus_online: online: 128
node_table[0]:
id: 0
pgdat: 0
size: 543813632
present: 73014444033
mem_map: 0
start_paddr: 0
start_mapnr: 0
NOTE: page_hash_table does not exist in this kernel
crash: page excluded: kernel virtual address: e000006003108e00 type:
"runqueues entry (per_cpu)"
a4700rac:/var/tmp/jlan #
