Hello I am getting the following error when I use crash on arm64 platform (NXP LS1043A, with 4 A53 cores). crash: read error: kernel virtual address: ffff00007b616100 type: "memory section root table" Additional information is that when I boot the kernel with nokaslr in bootargs, the crash seems to work well. Have any one else seen similar issue? The kernel version is top of mainline with the below patch applied: - arm64/crash_core: Export TCR_EL1.T1SZ in vmcoreinfo (http://lists.infradead.org/pipermail/kexec/2019-November/023962.html) And the crash-utils is from here: https://github.com/crash-utility/crash.git commit 5e975dd8c817ea6aea35e1e15b83c378aee9c136 Author: Dave Anderson <anderson@xxxxxxxxxx> Date: Tue Dec 24 08:43:52 2019 -0500 When determining the ARM64 kernel's "vabits_actual" value by reading the new TCR_EL1.T1SZ vmcoreinfo entry, display its value during session initialization only when invoking crash with "-d1" or larger -d debug value. (anderson@xxxxxxxxxx) Logs of crash -d1 are below: nxa19049@lsv03080:~/data/ups/crash$ ./crash -d1 ../linux/vmlinux ../linux/vmcore crash 7.2.7++ Copyright (C) 2002-2019 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. vmcore_data: flags: c0 (KDUMP_LOCAL|KDUMP_ELF64) ndfd: 3 ofp: 7fa9998cf620 header_size: 8192 num_pt_load_segments: 4 pt_load_segment[0]: file_offset: 2000 phys_start: 82080000 phys_end: 839bc000 zero_fill: 0 pt_load_segment[1]: file_offset: 193e000 phys_start: 80000000 phys_end: d8000000 zero_fill: 0 pt_load_segment[2]: file_offset: 5993e000 phys_start: fb000000 phys_end: fb800000 zero_fill: 0 pt_load_segment[3]: file_offset: 5a13e000 phys_start: fbc00000 phys_end: fbe00000 zero_fill: 0 elf_header: 2724410 elf32: 0 notes32: 0 load32: 0 elf64: 2724410 notes64: 2724450 load64: 2724488 sect0_64: 0 nt_prstatus: 2725410 nt_prpsinfo: 0 nt_taskstruct: 0 task_struct: 0 arch_data1: (unused) arch_data2: (unused) switch_stack: 0 page_size: 4096 xen_kdump_data: (unused) num_prstatus_notes: 4 num_qemu_notes: 0 vmcoreinfo: 2725a98 size_vmcoreinfo: 1939 nt_prstatus_percpu: 0000000002725410 00000000027255ac 0000000002725748 00000000027258e4 nt_qemu_percpu: backup_src_start: 0 backup_src_size: 0 backup_offset: 0 Elf64_Ehdr: e_ident: \177ELF e_ident[EI_CLASS]: 2 (ELFCLASS64) e_ident[EI_DATA]: 1 (ELFDATA2LSB) e_ident[EI_VERSION]: 1 (EV_CURRENT) e_ident[EI_OSABI]: 0 (ELFOSABI_SYSV) e_ident[EI_ABIVERSION]: 0 e_type: 4 (ET_CORE) e_machine: 183 (EM_AARCH64) e_version: 1 (EV_CURRENT) e_entry: 0 e_phoff: 40 e_shoff: 0 e_flags: 0 e_ehsize: 40 e_phentsize: 38 e_phnum: 5 e_shentsize: 0 e_shnum: 0 e_shstrndx: 0 Elf64_Phdr: p_type: 4 (PT_NOTE) p_offset: 4096 (1000) p_vaddr: 0 p_paddr: 0 p_filesz: 3612 (e1c) p_memsz: 3612 (e1c) p_flags: 0 () p_align: 0 Elf64_Phdr: p_type: 1 (PT_LOAD) p_offset: 8192 (2000) p_vaddr: ffffd3ebfa480000 p_paddr: 82080000 p_filesz: 26460160 (193c000) p_memsz: 26460160 (193c000) p_flags: 7 (PF_X|PF_W|PF_R) p_align: 0 Elf64_Phdr: p_type: 1 (PT_LOAD) p_offset: 26468352 (193e000) p_vaddr: ffff000000000000 p_paddr: 80000000 p_filesz: 1476395008 (58000000) p_memsz: 1476395008 (58000000) p_flags: 7 (PF_X|PF_W|PF_R) p_align: 0 Elf64_Phdr: p_type: 1 (PT_LOAD) p_offset: 1502863360 (5993e000) p_vaddr: ffff00007b000000 p_paddr: fb000000 p_filesz: 8388608 (800000) p_memsz: 8388608 (800000) p_flags: 7 (PF_X|PF_W|PF_R) p_align: 0 Elf64_Phdr: p_type: 1 (PT_LOAD) p_offset: 1511251968 (5a13e000) p_vaddr: ffff00007bc00000 p_paddr: fbc00000 p_filesz: 2097152 (200000) p_memsz: 2097152 (200000) p_flags: 7 (PF_X|PF_W|PF_R) p_align: 0 Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 392 n_type: 1 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 X0: 00000000000000e0 X1: ffff00007b5c4d80 X2: ffff 2c147fdf5000 X3: 000000000007b688 X4: 0000000000000015 X5: 00ff ffffffffffff X6: 0000000035f70457 X7: 0000000000000000 X8: ffff 00007b5c3e3c X9: 00000000000041b3 X10: ffff00007b5c3e1c X11: 0000 0000000325d3 X12: 00000000000325d3 X13: 071c71c71c71c71c X14: 0000 000000000000 X15: 0000000000000000 X16: 0000000000000000 X17: 0000 000000000000 X18: 0000000000000000 X19: 0000001e69146f40 X20: ffff 000053175800 X21: 0000000000000001 X22: ffff00007b5c3b00 X23: 0000 000000000001 X24: 0000001e633ee5f0 X25: ffff00007b5c3b00 X26: ffff 000053175800 X27: 0000000000000001 X28: 0000000082c16344 X29: ffff d3ebfbb73e50 LR: ffffd3ebfadbbab0 SP: ffffd3ebfbb73e50 PC: ffff d3ebfadbbab8 PSTATE: 60000005 FPVALID: 00000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 00000000000000e0 ffff00007b5c4d80 ffff2c147fdf5000 000000000007b688 0000000000000015 00ffffffffffffff 0000000035f70457 0000000000000000 ffff00007b5c3e3c 00000000000041b3 ffff00007b5c3e1c 00000000000325d3 00000000000325d3 071c71c71c71c71c 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000001e69146f40 ffff000053175800 0000000000000001 ffff00007b5c3b00 0000000000000001 0000001e633ee5f0 ffff00007b5c3b00 ffff000053175800 0000000000000001 0000000082c16344 ffffd3ebfbb73e50 ffffd3ebfadbbab0 ffffd3ebfbb73e50 ffffd3ebfadbbab8 0000000060000005 0000000000000000 Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 392 n_type: 1 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 X0: 00000000000000e0 X1: ffff00007b5dad80 X2: ffff 2c147fe0b000 X3: 000000000007b688 X4: 0000000000000015 X5: 00ff ffffffffffff X6: 0000000035f70457 X7: 0000000000000000 X8: ffff 00007b5d9e3c X9: 0000000000000db6 X10: ffff00007b5d9e1c X11: 0000 000000085aa2 X12: 0000000000085aa2 X13: 071c71c71c71c71c X14: 0000 000000000000 X15: 0000000000000000 X16: 0000000000000000 X17: 0000 000000000000 X18: 0000000000000000 X19: 0000001e69147148 X20: ffff 000053176000 X21: 0000000000000001 X22: ffff00007b5d9b00 X23: 0000 000000000001 X24: 0000001e599e8230 X25: ffff00007b5d9b00 X26: ffff 000053176000 X27: 0000000000000001 X28: 0000000000000000 X29: ffff 800010083ec0 LR: ffffd3ebfadbbab0 SP: ffff800010083ec0 PC: ffff d3ebfadbbab8 PSTATE: 60000005 FPVALID: 00000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 00000000000000e0 ffff00007b5dad80 ffff2c147fe0b000 000000000007b688 0000000000000015 00ffffffffffffff 0000000035f70457 0000000000000000 ffff00007b5d9e3c 0000000000000db6 ffff00007b5d9e1c 0000000000085aa2 0000000000085aa2 071c71c71c71c71c 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000001e69147148 ffff000053176000 0000000000000001 ffff00007b5d9b00 0000000000000001 0000001e599e8230 ffff00007b5d9b00 ffff000053176000 0000000000000001 0000000000000000 ffff800010083ec0 ffffd3ebfadbbab0 ffff800010083ec0 ffffd3ebfadbbab8 0000000060000005 0000000000000000 Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 392 n_type: 1 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 0 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 X0: 00000000000000e0 X1: ffff00007b5f0d80 X2: ffff 2c147fe21000 X3: 000000000007b688 X4: 0000000000000015 X5: 00ff ffffffffffff X6: 0000000035f70457 X7: 0000000000000000 X8: ffff 00007b5efe3c X9: ffff80001008beb0 X10: ffff00007b5efe1c X11: 0000 00000000466e X12: 000000000000466e X13: 071c71c71c71c71c X14: 0000 000000000000 X15: 0000000000000000 X16: 0000000000000000 X17: 0000 000000000000 X18: 0000000000000000 X19: 0000001e69147148 X20: ffff 000053176800 X21: 0000000000000001 X22: ffff00007b5efb00 X23: 0000 000000000001 X24: 0000001e68f584b8 X25: ffff00007b5efb00 X26: ffff 000053176800 X27: 0000000000000001 X28: 0000000000000000 X29: ffff 80001008bec0 LR: ffffd3ebfadbbab0 SP: ffff80001008bec0 PC: ffff d3ebfadbbab8 PSTATE: 60000005 FPVALID: 00000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 00000000000000e0 ffff00007b5f0d80 ffff2c147fe21000 000000000007b688 0000000000000015 00ffffffffffffff 0000000035f70457 0000000000000000 ffff00007b5efe3c ffff80001008beb0 ffff00007b5efe1c 000000000000466e 000000000000466e 071c71c71c71c71c 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000001e69147148 ffff000053176800 0000000000000001 ffff00007b5efb00 0000000000000001 0000001e68f584b8 ffff00007b5efb00 ffff000053176800 0000000000000001 0000000000000000 ffff80001008bec0 ffffd3ebfadbbab0 ffff80001008bec0 ffffd3ebfadbbab8 0000000060000005 0000000000000000 Elf64_Nhdr: n_namesz: 5 ("CORE") n_descsz: 392 n_type: 1 (NT_PRSTATUS) si.signo: 0 si.code: 0 si.errno: 0 cursig: 0 sigpend: 0 sighold: 0 pid: 884 ppid: 0 pgrp: 0 sid:0 utime: 0.000000 stime: 0.000000 cutime: 0.000000 cstime: 0.000000 X0: ffff00004de23400 X1: 0000000000000000 X2: ffff 00004f58c600 X3: 0000000000000000 X4: ffffd3ebfbb954b8 X5: ffff 00004f58c600 X6: 0000000000000001 X7: ffff8000102fbbe0 X8: 0000 000000000146 X9: 00000000ffffffd0 X10: ffffd3ebfbb7a120 X11: 0000 000005f5e0ff X12: ffffd3ebfbb90000 X13: ffff8000102fb8f5 X14: ffff 8000902fb8e7 X15: ffffd3ebfbb798c8 X16: 0000000000000000 X17: 0000 000000000000 X18: ffffffffffffffff X19: ffffd3ebfbb79000 X20: ffff d3ebfbb95000 X21: 0000000000000000 X22: ffffd3ebfbd7a000 X23: ffff 8000102fbaf8 X24: 0000000000000004 X25: 0000000000000000 X26: ffff d3ebfbce4000 X27: 0000000000000000 X28: ffff00004f58c600 X29: ffff 8000102fbab0 LR: ffffd3ebfa58ded8 SP: ffff8000102fbab0 PC: ffff d3ebfa58df5c PSTATE: 60000085 FPVALID: 00000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000374 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 ffff00004de23400 0000000000000000 ffff00004f58c600 0000000000000000 ffffd3ebfbb954b8 ffff00004f58c600 0000000000000001 ffff8000102fbbe0 0000000000000146 00000000ffffffd0 ffffd3ebfbb7a120 0000000005f5e0ff ffffd3ebfbb90000 ffff8000102fb8f5 ffff8000902fb8e7 ffffd3ebfbb798c8 0000000000000000 0000000000000000 ffffffffffffffff ffffd3ebfbb79000 ffffd3ebfbb95000 0000000000000000 ffffd3ebfbd7a000 ffff8000102fbaf8 0000000000000004 0000000000000000 ffffd3ebfbce4000 0000000000000000 ffff00004f58c600 ffff8000102fbab0 ffffd3ebfa58ded8 ffff8000102fbab0 ffffd3ebfa58df5c 0000000060000085 0000000000000000 Elf64_Nhdr: n_namesz: 11 ("VMCOREINFO") n_descsz: 1939 n_type: 0 (unused) OSRELEASE=5.5.0-rc4-00068-gf8631e1 PAGESIZE=4096 SYMBOL(init_uts_ns)=ffffd3ebfbb81bf8 SYMBOL(node_online_map)=ffffd3ebfbb7a3e8 SYMBOL(swapper_pg_dir)=ffffd3ebfb67a000 SYMBOL(_stext)=ffffd3ebfa481000 SYMBOL(vmap_area_list)=ffffd3ebfbb9b258 SYMBOL(mem_section)=ffff00007b616100 LENGTH(mem_section)=1024 SIZE(mem_section)=16 OFFSET(mem_section.section_mem_map)=0 SIZE(page)=64 SIZE(pglist_data)=7936 SIZE(zone)=1664 SIZE(free_area)=104 SIZE(list_head)=16 SIZE(nodemask_t)=8 OFFSET(page.flags)=0 OFFSET(page._refcount)=52 OFFSET(page.mapping)=24 OFFSET(page.lru)=8 OFFSET(page._mapcount)=48 OFFSET(page.private)=40 OFFSET(page.compound_dtor)=16 OFFSET(page.compound_order)=17 OFFSET(page.compound_head)=8 OFFSET(pglist_data.node_zones)=0 OFFSET(pglist_data.nr_zones)=7200 OFFSET(pglist_data.node_start_pfn)=7208 OFFSET(pglist_data.node_spanned_pages)=7224 OFFSET(pglist_data.node_id)=7232 OFFSET(zone.free_area)=192 OFFSET(zone.vm_stat)=1472 OFFSET(zone.spanned_pages)=112 OFFSET(free_area.free_list)=0 OFFSET(list_head.next)=0 OFFSET(list_head.prev)=8 OFFSET(vmap_area.va_start)=0 OFFSET(vmap_area.list)=40 LENGTH(zone.free_area)=11 SYMBOL(log_buf)=ffffd3ebfbb90b40 SYMBOL(log_buf_len)=ffffd3ebfbb90b38 SYMBOL(log_first_idx)=ffffd3ebfbd5566c SYMBOL(clear_idx)=ffffd3ebfbd55678 SYMBOL(log_next_idx)=ffffd3ebfbd55668 SIZE(printk_log)=16 OFFSET(printk_log.ts_nsec)=0 OFFSET(printk_log.len)=8 OFFSET(printk_log.text_len)=10 OFFSET(printk_log.dict_len)=12 LENGTH(free_area.free_list)=6 NUMBER(NR_FREE_PAGES)=0 NUMBER(PG_lru)=4 NUMBER(PG_private)=13 NUMBER(PG_swapcache)=10 NUMBER(PG_swapbacked)=19 NUMBER(PG_slab)=9 NUMBER(PG_hwpoison)=22 NUMBER(PG_head_mask)=65536 NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129 NUMBER(HUGETLB_PAGE_DTOR)=2 NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257 NUMBER(VA_BITS)=48 NUMBER(kimage_voffset)=0xffffd3eb78400000 NUMBER(PHYS_OFFSET)=0xffffafb780000000 NUMBER(tcr_el1_t1sz)=0x10 KERNELOFFSET=53ebea400000 CRASHTIME=1517155176 readmem: read_kdump() KASLR: _stext from vmlinux: ffff800010081000 _stext from vmcoreinfo: ffffd3ebfa481000 relocate: 53ebea400000 (87998116MB) crash: pv_ops exists: ARCH_PVOPS vmcoreinfo : vabits_actual: 48 using ffffafb780000000 as phys_offset kimage_voffset: ffffd3eb78400000 phys_offset: ffffafb780000000 gdb ../linux/vmlinux GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=aarch64-elf-linux"... WARNING: kernel relocated [87998116MB]: patching 135244 gdb minimal_symbol values crash: CONFIG_NR_CPUS: 256 crash: CONFIG_HZ: 250 crash: # CONFIG_DEBUG_INFO_REDUCED is not set cpu_possible_mask: cpus: 0 1 2 3 cpu_present_mask: cpus: 0 1 2 3 cpu_online_mask: cpus: 0 1 2 3 cpu_active_mask: cpus: 0 1 2 3 xtime timespec.tv_sec: 5a6df368: Sun Jan 28 21:29:36 2018 utsname: sysname: Linux nodename: localhost release: 5.5.0-rc4-00068-gf8631e1 version: #18 SMP PREEMPT Fri Jan 3 19:16:28 IST 2020 machine: aarch64 domainname: (none) base kernel version: 5.5.0 verify_namelist: dumpfile /proc/version: Linux version 5.5.0-rc4-00068-gf8631e1 (mailto:nxa19049@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 7.3.0 (GCC)) #18 SMP PREEMPT Fri Jan 3 19:16:28 IST 2020 ../linux/vmlinux: Linux version 5.5.0-rc4-00068-gf8631e1 (mailto:nxa19049@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 7.3.0 (GCC)) #18 SMP PREEMPT Fri Jan 3 19:16:28 IST 2020 crash: get_cpus_present: present: 4 hypervisor: (undetermined) ikconfig: 2550 valid configs. SECTION_SIZE_BITS: 30 CONFIG_ARM64_VA_BITS: 48 VA_BITS_ACTUAL: 48 (calculated) VA_BITS: 48 PAGE_OFFSET: ffff000000000000 VA_START: ffff800000000000 modules: ffff800008000000 - ffff80000fffffff vmalloc: ffff800010000000 - fffffdffbffeffff kernel image: ffffd3ebfa480000 - ffffd3ebfbdbc000 vmemmap: fffffe0000000000 - ffffffffffffffff irq_stack_ptr: type: 1, TYPE_CODE_PTR target_typecode: 8, TYPE_CODE_INT target_length: 8 length: 8 crash: test msg 2crash: test msg 2 p=7b5b9050 cpunum=0crash: read error: kernel virtual address: ffff00007b5b9050 type: "IRQ stack pointer" crash: test msg 2 p=7b5cf050 cpunum=1crash: read error: kernel virtual address: ffff00007b5cf050 type: "IRQ stack pointer" crash: test msg 2 p=7b5e5050 cpunum=2crash: read error: kernel virtual address: ffff00007b5e5050 type: "IRQ stack pointer" crash: test msg 2 p=7b5fb050 cpunum=3crash: read error: kernel virtual address: ffff00007b5fb050 type: "IRQ stack pointer" crash: builtin stackframe.sp offset differs from kernel version crash: builtin stackframe.pc offset differs from kernel version kernel NR_CPUS: 256 PAGESIZE=4096 mem_section_size = 8192 NR_SECTION_ROOTS = 1024 NR_MEM_SECTIONS = 262144 SECTIONS_PER_ROOT = 256 SECTION_ROOT_MASK = 0xff PAGES_PER_SECTION = 262144 crash: read error: kernel virtual address: ffff00007b616100 type: "memory section root table" Regards Poonam -- Crash-utility mailing list Crash-utility@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/crash-utility
