On Mon, Feb 10, 2014 at 2:11 PM, Dave Anderson <anderson@xxxxxxxxxx> wrote: > > Hi Andy, > > I've got a ELF kdump vmcore that was created in-house from a kernel configured > with CONFIG_RANDOMIZE_BASE. I thought I might be able to analyze it by applying > your earlier patch that introduced the --kaslr option. The kernel does not > have the offset registered in the vmcoreinfo, and so I'm trying to determine > the offset, but with no luck. > > Earlier, Kees had mentioned this: > >>> FWIW, the offset reported during a panic to dmesg is: >>> (unsigned long)&_text - __START_KERNEL > > Where does it get reported during a panic exactly? Here's the oops trace, gotten > by running "strings" on the vmcore: The commit that reports it is here: f32360ef6608434a032dc7ad262d45e9693c27f3 And maybe related, but 6723734cdff15211bb78aeea76ca847374bd93ae makes sure panic handlers are called before kmsg_dump. And "strings" doesn't show anything starting with "Kernel Offset:" ? Maybe it's not being saved in the kcore? -Kees > > $ strings vmcore > ... [ cut ] ... > SysRq : Trigger a crash > "BUG: unable to handle kernel NULL pointer dereference at (null) > "IP: [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b > PGD 3a067 PUD 2e067 PMD 0 > Oops: 0002 [#1] PREEMPT SMP 0" > Modules linked in: > CPU: 0 PID: 1720 Comm: bash Not tainted 3.14.0-rc1+ #1130" > task: ffff88001d028000 ti: ffff88001c986000 task.ti: ffff88001c986000 > RIP: 0010:[<ffffffff992bf6cf>] [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b > RSP: 0018:ffff88001c987e90 EFLAGS: 000100920" > RAX: 000000000000000f RBX: ffffffff9975ed50 RCX: 0000000000000000 > RDX: ffff88001d028000 RSI: ffff88001cc0e338 RDI: 0000000000000063 > RBP: ffff88001c987e90 R08: 0000000000000002 R09: 0000000000000000 > R10: ffffffff994e9630 R11: 0000000000000000 R12: 0000000000000007 > R13: 0000000000000246 R14: 0000000000000063 R15: 0000000000000000 > FS: 00007f0ec2181740(0000) GS:ffff88001cc00000(0000) knlGS:00000000000000000" > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 0000000000000000 CR3: 000000001d36f000 CR4: 00000000000006f0 > Stack: > ffff88001c987ec8 ffffffff992bfc88 0000000000000002 00007f0ec21870000" > 0000000000000002 ffff88001c987f58 0000000000000000 ffff88001c987ee80" > ffffffff992c000c ffff88001c92acc0 00007f0ec2187000 ffff88001c987f080" > Call Trace: > [<ffffffff992bfc88>] __handle_sysrq+0x9b/0x133 > [<ffffffff992c000c>] write_sysrq_trigger+0x2d/0x3e > [<ffffffff991681cb>] proc_reg_write+0x45/0x65 > [<ffffffff9911897c>] vfs_write+0xbf/0x17c > [<ffffffff9911918f>] SyS_write+0x44/0x7a > [<ffffffff9949ad7d>] system_call_fastpath+0x1a/0x1f0" > Code: 4f 00 00 55 b8 01 00 00 00 48 89 e5 75 07 0f b6 05 b3 20 4f 00 83 e0 01 5d c3 55 c7 05 03 18 61 00 01 00 00 00 48 89 e5 0f ae f8 <c6> 04 25 00 00 00 00 01 5d c3 55 31 c0 c7 05 ba dc 46 00 07 00 0" > "RIP [<ffffffff992bf6cf>] sysrq_handle_crash+0x11/0x1b > RSP <ffff88001c987e90> > CR2: 0000000000000000 > ttySffffffff99000000 T _text > > UUUU > UUUU > VMCOREINFO > OSRELEASE=3.14.0-rc1+ > PAGESIZE=4096 > SYMBOL(init_uts_ns)=ffffffff99713250 > SYMBOL(node_online_map)=ffffffff997b0c68 > SYMBOL(swapper_pg_dir)=ffffffff9970e000 > SYMBOL(_stext)=ffffffff990001c8 > SYMBOL(vmap_area_list)=ffffffff99745c20 > SYMBOL(mem_map)=ffffffff9a1253a8 > SYMBOL(contig_page_data)=ffffffff99790000 > SYMBOL(mem_section)=ffffffff9a126000 > LENGTH(mem_section)=2048 > SIZE(mem_section)=16 > OFFSET(mem_section.section_mem_map)=0 > SIZE(page)=64 > SIZE(pglist_data)=53248 > SIZE(zone)=12288 > SIZE(free_area)=88 > SIZE(list_head)=16 > SIZE(nodemask_t)=8 > OFFSET(page.flags)=0 > OFFSET(page._count)=28 > OFFSET(page.mapping)=8 > OFFSET(page.lru)=32 > OFFSET(page._mapcount)=24 > OFFSET(page.private)=48 > OFFSET(pglist_data.node_zones)=0 > OFFSET(pglist_data.nr_zones)=49240 > OFFSET(pglist_data.node_start_pfn)=49304 > OFFSET(pglist_data.node_spanned_pages)=49320 > OFFSET(pglist_data.node_id)=49328 > OFFSET(zone.free_area)=256 > OFFSET(zone.vm_stat)=4280 > OFFSET(zone.spanned_pages)=8232 > OFFSET(free_area.free_list)=0 > OFFSET(list_head.next)=0 > OFFSET(list_head.prev)=8 > OFFSET(vmap_area.va_start)=0 > OFFSET(vmap_area.list)=48 > LENGTH(zone.free_area)=11 > SYMBOL(log_buf)=ffffffff9972d290 > SYMBOL(log_buf_len)=ffffffff9972d288 > SYMBOL(log_first_idx)=ffffffff9a11eb48 > SYMBOL(log_next_idx)=ffffffff9a11eb38 > SIZE(printk_log)=16 > OFFSET(printk_log.ts_nsec)=0 > OFFSET(printk_log.len)=8 > OFFSET(printk_log.text_len)=10 > OFFSET(printk_log.dict_len)=12 > LENGTH(free_area.free_list)=5 > NUMBER(NR_FREE_PAGES)=0 > NUMBER(PG_lru)=5 > NUMBER(PG_private)=11 > NUMBER(PG_swapcache)=16 > NUMBER(PG_slab)=7 > NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128 > SYMBOL(phys_base)=ffffffff99713010 > SYMBOL(init_level4_pgt)=ffffffff9970e000 > CRASHTIME=1391826079 > OSRELEASE=3.14.0-rc1+ > PAGESIZE=4096 > SYMBOL(init_uts_ns)=ffffffff99713250 > SYMBOL(node_online_map)=ffffffff997b0c68 > SYMBOL(swapper_pg_dir)=ffffffff9970e000 > SYMBOL(_stext)=ffffffff990001c8 > SYMBOL(vmap_area_list)=ffffffff99745c20 > SYMBOL(mem_map)=ffffffff9a1253a8 > SYMBOL(contig_page_data)=ffffffff99790000 > SYMBOL(mem_section)=ffffffff9a126000 > LENGTH(mem_section)=2048 > SIZE(mem_section)=16 > OFFSET(mem_section.section_mem_map)=0 > SIZE(page)=64 > SIZE(pglist_data)=53248 > SIZE(zone)=12288 > SIZE(free_area)=88 > SIZE(list_head)=16 > SIZE(nodemask_t)=8 > OFFSET(page.flags)=0 > OFFSET(page._count)=28 > OFFSET(page.mapping)=8 > OFFSET(page.lru)=32 > OFFSET(page._mapcount)=24 > OFFSET(page.private)=48 > OFFSET(pglist_data.node_zones)=0 > OFFSET(pglist_data.nr_zones)=49240 > OFFSET(pglist_data.node_start_pfn)=49304 > OFFSET(pglist_data.node_spanned_pages)=49320 > OFFSET(pglist_data.node_id)=49328 > OFFSET(zone.free_area)=256 > OFFSET(zone.vm_stat)=4280 > OFFSET(zone.spanned_pages)=8232 > OFFSET(free_area.free_list)=0 > OFFSET(list_head.next)=0 > OFFSET(list_head.prev)=8 > OFFSET(vmap_area.va_start)=0 > OFFSET(vmap_area.list)=48 > LENGTH(zone.free_area)=11 > SYMBOL(log_buf)=ffffffff9972d290 > SYMBOL(log_buf_len)=ffffffff9972d288 > SYMBOL(log_first_idx)=ffffffff9a11eb48 > SYMBOL(log_next_idx)=ffffffff9a11eb38 > SIZE(printk_log)=16 > OFFSET(printk_log.ts_nsec)=0 > OFFSET(printk_log.len)=8 > OFFSET(printk_log.text_len)=10 > OFFSET(printk_log.dict_len)=12 > LENGTH(free_area.free_list)=5 > NUMBER(NR_FREE_PAGES)=0 > NUMBER(PG_lru)=5 > NUMBER(PG_private)=11 > NUMBER(PG_swapcache)=16 > NUMBER(PG_slab)=7 > NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128 > SYMBOL(phys_base)=ffffffff99713010 > SYMBOL(init_level4_pgt)=ffffffff9970e000 > CRASHTIME=1391826079 > ... > > Anyway, the /proc/kallsyms file of the crashing system was saved, > and it shows this: > > ffffffff99000000 T _text > > and if I subtract __START_KERNEL (ffffffff80000000) from that, I get > what I presume is the kaslr offset of 0x19000000. The vmcore core > header would seeminlgy confirm that: > > $ readelf -a vmcore > ... [ cut ] ... > Program Headers: > Type Offset VirtAddr PhysAddr > FileSiz MemSiz Flags Align > NOTE 0x0000000000001000 0x0000000000000000 0x0000000000000000 > 0x00000000000007f8 0x00000000000007f8 0 > LOAD 0x0000000000002000 0xffffffff99000000 0x0000000019000000 <=== > 0x0000000001183000 0x0000000001183000 RWE 0 > LOAD 0x0000000001185000 0xffff880000001000 0x0000000000001000 > 0x000000000009f000 0x000000000009f000 RWE 0 > LOAD 0x0000000001224000 0xffff880000100000 0x0000000000100000 > 0x0000000010f00000 0x0000000010f00000 RWE 0 > LOAD 0x0000000012124000 0xffff880019000000 0x0000000019000000 > 0x0000000005194000 0x0000000005194000 RWE 0 > LOAD 0x00000000172b8000 0xffff88001e1c1000 0x000000001e1c1000 > 0x00000000017c0000 0x00000000017c0000 RWE 0 > LOAD 0x0000000018a78000 0xffff88001f9e5000 0x000000001f9e5000 > 0x00000000005fb000 0x00000000005fb000 RWE 0 > > But if I try that value with your patch applied, it fails in the same manner > as if I don't use the --kaslr option at all: > > $ crash --kaslr 0x19000000 vmlinux vmcore > > crash 7.0.5rc12 > Copyright (C) 2002-2014 Red Hat, Inc. > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation > Copyright (C) 1999-2006 Hewlett-Packard Co > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. > Copyright (C) 2005, 2011 NEC Corporation > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. > This program is free software, covered by the GNU General Public License, > and you are welcome to change it and/or distribute copies of it under > certain conditions. Enter "help copying" to see the conditions. > This program has absolutely no warranty. Enter "help warranty" for details. > > GNU gdb (GDB) 7.6 > Copyright (C) 2013 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-unknown-linux-gnu"... > > WARNING: could not find MAGIC_START! > WARNING: cannot read linux_banner string > crash: vmlinux and vmcore do not match! > > Usage: > > crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form) > crash [OPTION]... [NAMELIST] (live system form) > > Enter "crash -h" for details. > $ > > Any ideas? I can give you the vmlinux/vmcore/kallsyms triplet if you'd like. > > Thanks, > Dave > > -- Kees Cook Chrome OS Security -- Crash-utility mailing list Crash-utility@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/crash-utility