Hi Andy,
I've got a ELF kdump vmcore that was created in-house from a kernel configured
with CONFIG_RANDOMIZE_BASE. I thought I might be able to analyze it by applying
your earlier patch that introduced the --kaslr option. The kernel does not
have the offset registered in the vmcoreinfo, and so I'm trying to determine
the offset, but with no luck.
Earlier, Kees had mentioned this:
>> FWIW, the offset reported during a panic to dmesg is:
>> (unsigned long)&_text - __START_KERNEL
Where does it get reported during a panic exactly? Here's the oops trace, gotten
by running "strings" on the vmcore:
$ strings vmcore
... [ cut ] ...
SysRq : Trigger a crash
"BUG: unable to handle kernel NULL pointer dereference at (null)
"IP: [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b
PGD 3a067 PUD 2e067 PMD 0
Oops: 0002 [#1] PREEMPT SMP 0"
Modules linked in:
CPU: 0 PID: 1720 Comm: bash Not tainted 3.14.0-rc1+ #1130"
task: ffff88001d028000 ti: ffff88001c986000 task.ti: ffff88001c986000
RIP: 0010:[<ffffffff992bf6cf>] [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b
RSP: 0018:ffff88001c987e90 EFLAGS: 000100920"
RAX: 000000000000000f RBX: ffffffff9975ed50 RCX: 0000000000000000
RDX: ffff88001d028000 RSI: ffff88001cc0e338 RDI: 0000000000000063
RBP: ffff88001c987e90 R08: 0000000000000002 R09: 0000000000000000
R10: ffffffff994e9630 R11: 0000000000000000 R12: 0000000000000007
R13: 0000000000000246 R14: 0000000000000063 R15: 0000000000000000
FS: 00007f0ec2181740(0000) GS:ffff88001cc00000(0000) knlGS:00000000000000000"
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000001d36f000 CR4: 00000000000006f0
Stack:
ffff88001c987ec8 ffffffff992bfc88 0000000000000002 00007f0ec21870000"
0000000000000002 ffff88001c987f58 0000000000000000 ffff88001c987ee80"
ffffffff992c000c ffff88001c92acc0 00007f0ec2187000 ffff88001c987f080"
Call Trace:
[<ffffffff992bfc88>] __handle_sysrq+0x9b/0x133
[<ffffffff992c000c>] write_sysrq_trigger+0x2d/0x3e
[<ffffffff991681cb>] proc_reg_write+0x45/0x65
[<ffffffff9911897c>] vfs_write+0xbf/0x17c
[<ffffffff9911918f>] SyS_write+0x44/0x7a
[<ffffffff9949ad7d>] system_call_fastpath+0x1a/0x1f0"
Code: 4f 00 00 55 b8 01 00 00 00 48 89 e5 75 07 0f b6 05 b3 20 4f 00 83 e0 01 5d c3 55 c7 05 03 18 61 00 01 00 00 00 48 89 e5 0f ae f8 <c6> 04 25 00 00 00 00 01 5d c3 55 31 c0 c7 05 ba dc 46 00 07 00 0"
"RIP [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b
RSP <ffff88001c987e90>
CR2: 0000000000000000
ttySffffffff99000000 T _text
UUUU
UUUU
VMCOREINFO
OSRELEASE=3.14.0-rc1+
PAGESIZE=4096
SYMBOL(init_uts_ns)=ffffffff99713250
SYMBOL(node_online_map)=ffffffff997b0c68
SYMBOL(swapper_pg_dir)=ffffffff9970e000
SYMBOL(_stext)=ffffffff990001c8
SYMBOL(vmap_area_list)=ffffffff99745c20
SYMBOL(mem_map)=ffffffff9a1253a8
SYMBOL(contig_page_data)=ffffffff99790000
SYMBOL(mem_section)=ffffffff9a126000
LENGTH(mem_section)=2048
SIZE(mem_section)=16
OFFSET(mem_section.section_mem_map)=0
SIZE(page)=64
SIZE(pglist_data)=53248
SIZE(zone)=12288
SIZE(free_area)=88
SIZE(list_head)=16
SIZE(nodemask_t)=8
OFFSET(page.flags)=0
OFFSET(page._count)=28
OFFSET(page.mapping)=8
OFFSET(page.lru)=32
OFFSET(page._mapcount)=24
OFFSET(page.private)=48
OFFSET(pglist_data.node_zones)=0
OFFSET(pglist_data.nr_zones)=49240
OFFSET(pglist_data.node_start_pfn)=49304
OFFSET(pglist_data.node_spanned_pages)=49320
OFFSET(pglist_data.node_id)=49328
OFFSET(zone.free_area)=256
OFFSET(zone.vm_stat)=4280
OFFSET(zone.spanned_pages)=8232
OFFSET(free_area.free_list)=0
OFFSET(list_head.next)=0
OFFSET(list_head.prev)=8
OFFSET(vmap_area.va_start)=0
OFFSET(vmap_area.list)=48
LENGTH(zone.free_area)=11
SYMBOL(log_buf)=ffffffff9972d290
SYMBOL(log_buf_len)=ffffffff9972d288
SYMBOL(log_first_idx)=ffffffff9a11eb48
SYMBOL(log_next_idx)=ffffffff9a11eb38
SIZE(printk_log)=16
OFFSET(printk_log.ts_nsec)=0
OFFSET(printk_log.len)=8
OFFSET(printk_log.text_len)=10
OFFSET(printk_log.dict_len)=12
LENGTH(free_area.free_list)=5
NUMBER(NR_FREE_PAGES)=0
NUMBER(PG_lru)=5
NUMBER(PG_private)=11
NUMBER(PG_swapcache)=16
NUMBER(PG_slab)=7
NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128
SYMBOL(phys_base)=ffffffff99713010
SYMBOL(init_level4_pgt)=ffffffff9970e000
CRASHTIME=1391826079
OSRELEASE=3.14.0-rc1+
PAGESIZE=4096
SYMBOL(init_uts_ns)=ffffffff99713250
SYMBOL(node_online_map)=ffffffff997b0c68
SYMBOL(swapper_pg_dir)=ffffffff9970e000
SYMBOL(_stext)=ffffffff990001c8
SYMBOL(vmap_area_list)=ffffffff99745c20
SYMBOL(mem_map)=ffffffff9a1253a8
SYMBOL(contig_page_data)=ffffffff99790000
SYMBOL(mem_section)=ffffffff9a126000
LENGTH(mem_section)=2048
SIZE(mem_section)=16
OFFSET(mem_section.section_mem_map)=0
SIZE(page)=64
SIZE(pglist_data)=53248
SIZE(zone)=12288
SIZE(free_area)=88
SIZE(list_head)=16
SIZE(nodemask_t)=8
OFFSET(page.flags)=0
OFFSET(page._count)=28
OFFSET(page.mapping)=8
OFFSET(page.lru)=32
OFFSET(page._mapcount)=24
OFFSET(page.private)=48
OFFSET(pglist_data.node_zones)=0
OFFSET(pglist_data.nr_zones)=49240
OFFSET(pglist_data.node_start_pfn)=49304
OFFSET(pglist_data.node_spanned_pages)=49320
OFFSET(pglist_data.node_id)=49328
OFFSET(zone.free_area)=256
OFFSET(zone.vm_stat)=4280
OFFSET(zone.spanned_pages)=8232
OFFSET(free_area.free_list)=0
OFFSET(list_head.next)=0
OFFSET(list_head.prev)=8
OFFSET(vmap_area.va_start)=0
OFFSET(vmap_area.list)=48
LENGTH(zone.free_area)=11
SYMBOL(log_buf)=ffffffff9972d290
SYMBOL(log_buf_len)=ffffffff9972d288
SYMBOL(log_first_idx)=ffffffff9a11eb48
SYMBOL(log_next_idx)=ffffffff9a11eb38
SIZE(printk_log)=16
OFFSET(printk_log.ts_nsec)=0
OFFSET(printk_log.len)=8
OFFSET(printk_log.text_len)=10
OFFSET(printk_log.dict_len)=12
LENGTH(free_area.free_list)=5
NUMBER(NR_FREE_PAGES)=0
NUMBER(PG_lru)=5
NUMBER(PG_private)=11
NUMBER(PG_swapcache)=16
NUMBER(PG_slab)=7
NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128
SYMBOL(phys_base)=ffffffff99713010
SYMBOL(init_level4_pgt)=ffffffff9970e000
CRASHTIME=1391826079
...
Anyway, the /proc/kallsyms file of the crashing system was saved,
and it shows this:
ffffffff99000000 T _text
and if I subtract __START_KERNEL (ffffffff80000000) from that, I get
what I presume is the kaslr offset of 0x19000000. The vmcore core
header would seeminlgy confirm that:
$ readelf -a vmcore
... [ cut ] ...
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x0000000000001000 0x0000000000000000 0x0000000000000000
0x00000000000007f8 0x00000000000007f8 0
LOAD 0x0000000000002000 0xffffffff99000000 0x0000000019000000 <===
0x0000000001183000 0x0000000001183000 RWE 0
LOAD 0x0000000001185000 0xffff880000001000 0x0000000000001000
0x000000000009f000 0x000000000009f000 RWE 0
LOAD 0x0000000001224000 0xffff880000100000 0x0000000000100000
0x0000000010f00000 0x0000000010f00000 RWE 0
LOAD 0x0000000012124000 0xffff880019000000 0x0000000019000000
0x0000000005194000 0x0000000005194000 RWE 0
LOAD 0x00000000172b8000 0xffff88001e1c1000 0x000000001e1c1000
0x00000000017c0000 0x00000000017c0000 RWE 0
LOAD 0x0000000018a78000 0xffff88001f9e5000 0x000000001f9e5000
0x00000000005fb000 0x00000000005fb000 RWE 0
But if I try that value with your patch applied, it fails in the same manner
as if I don't use the --kaslr option at all:
$ crash --kaslr 0x19000000 vmlinux vmcore
crash 7.0.5rc12
Copyright (C) 2002-2014 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
WARNING: could not find MAGIC_START!
WARNING: cannot read linux_banner string
crash: vmlinux and vmcore do not match!
Usage:
crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
crash [OPTION]... [NAMELIST] (live system form)
Enter "crash -h" for details.
$
Any ideas? I can give you the vmlinux/vmcore/kallsyms triplet if you'd like.
Thanks,
Dave
--
Crash-utility mailing list
Crash-utility@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/crash-utility
