Hi, Lianbo > > In addition, I just got a dump file generated by the "virsh dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors > > with or without this patch, can you also double check? Maybe it is a similar issue. > > > Ok, let me check further. > and can you tell me what kernel versin and linux distribution you are used? > > > [root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep vabits_actual=48 I used ubuntu-18.04-arm64 to reproduce your issue,kernel version is:4.15.18, in this version,arm64 still not support vabits_actual(5383cc6efed1 arm64: mm: Introduce vabits_actual),so we can't use "--machdep vabits_actual=48", after remove "--machdep vabits_actual=48",crash-utility can work fine. qianli zhao <zhaoqianligood@xxxxxxxxx> 于2022年6月30日周四 13:41写道： > > Hi, Lianbo > > > > > On Wed, Jun 29, 2022 at 2:40 PM qianli zhao <zhaoqianligood@xxxxxxxxx> wrote: > >> > >> Hi,Lianbo > >> Thank you for your feedback. > >> > >> My kernel version is 5.10.59, and this issue is happened when the > >> > >> ramdumps without vmcoreinfo. > > > > > > Thank you for the explanation. > > > > It is important to point this(*without vmcoreinfo*) out in the patch log. That can help understand this issue. Can you add it to the patch log? > sure, will update in next patchset > > > In addition, I just got a dump file generated by the "virsh dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors > > with or without this patch, can you also double check? Maybe it is a similar issue. > > > Ok, let me check further. > and can you tell me what kernel versin and linux distribution you are used? > > > [root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep vabits_actual=48 > > > > lijiang <lijiang@xxxxxxxxxx> 于2022年6月29日周三 17:31写道： > > > > On Wed, Jun 29, 2022 at 2:40 PM qianli zhao <zhaoqianligood@xxxxxxxxx> wrote: > >> > >> Hi,Lianbo > >> Thank you for your feedback. > >> > >> My kernel version is 5.10.59, and this issue is happened when the > >> > >> ramdumps without vmcoreinfo. > > > > > > Thank you for the explanation. > > > > It is important to point this(*without vmcoreinfo*) out in the patch log. That can help understand this issue. Can you add it to the patch log? > > > > In addition, I just got a dump file generated by the "virsh dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors > > with or without this patch, can you also double check? Maybe it is a similar issue. > > > > [root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep vabits_actual=48 > > > > crash 8.0.1++ > > Copyright (C) 2002-2022 Red Hat, Inc. > > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation > > Copyright (C) 1999-2006 Hewlett-Packard Co > > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited > > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. > > Copyright (C) 2005, 2011, 2020-2022 NEC Corporation > > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. > > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. > > Copyright (C) 2015, 2021 VMware, Inc. > > This program is free software, covered by the GNU General Public License, > > and you are welcome to change it and/or distribute copies of it under > > certain conditions. Enter "help copying" to see the conditions. > > This program has absolutely no warranty. Enter "help warranty" for details. > > > > NOTE: setting vabits_actual to: 48 > > > > WARNING: kimage_voffset cannot be determined from the dumpfile. > > Try using the command line option: --machdep kimage_voffset=<addr> > > GNU gdb (GDB) 10.2 > > Copyright (C) 2021 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. > > Type "show copying" and "show warranty" for details. > > This GDB was configured as "aarch64-unknown-linux-gnu". > > Type "show configuration" for configuration details. > > Find the GDB manual and other documentation resources online at: > > <http://www.gnu.org/software/gdb/documentation/>. > > > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > > > crash: read error: kernel virtual address: ffff8000119cee00 type: "possible" > > WARNING: cannot read cpu_possible_map > > crash: read error: kernel virtual address: ffff8000119cf208 type: "present" > > WARNING: cannot read cpu_present_map > > crash: read error: kernel virtual address: ffff8000119cec00 type: "online" > > WARNING: cannot read cpu_online_map > > crash: read error: kernel virtual address: ffff8000119cf408 type: "active" > > WARNING: cannot read cpu_active_map > > crash: read error: kernel virtual address: ffff8000120c97a8 type: "shadow_timekeeper xtime_sec" > > crash: read error: kernel virtual address: ffff8000119e1668 type: "init_uts_ns" > > crash: vmlinux and dump.core do not match! > > > > Usage: > > > > crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form) > > crash [OPTION]... [NAMELIST] (live system form) > > > > Enter "crash -h" for details. > > > > Thanks. > > Lianbo > > > >> I guess your scene may have vmcore,the following logic helps to > >> correctly locate MODULES/VMALLOC ranges with right kernel version. > >> > >> 279 ms->kimage_end = (sp ? sp->value : 0); > >> 280 > >> 281 if (ms->struct_page_size && (r = > >> arm64_get_va_range(ms))) {------------->//If there is a vmcore, it > >> will go to the following codes > >> 282 /* We can get all the > >> MODULES/VMALLOC/VMEMMAP ranges now.*/ > >> 283 ms->modules_vaddr = r->modules_vaddr; > >> 284 ms->modules_end = > >> r->modules_end - 1; > >> 285 ms->vmalloc_start_addr = > >> r->vmalloc_start_addr; > >> 286 ms->vmalloc_end = > >> r->vmalloc_end - 1; > >> 287 ms->vmemmap_vaddr = r->vmemmap_vaddr; > >> 288 ms->vmemmap_end = > >> r->vmemmap_end - 1; > >> 289 } else if (ms->VA_BITS_ACTUAL) > >> {------------>//unable to locate the version and other information > >> through vmcore > >> } > >> > >> lijiang <lijiang@xxxxxxxxxx> 于2022年6月29日周三 14:21写道： > >> > > >> > Hi, Qianli > >> > On Tue, Jun 28, 2022 at 9:55 AM lijiang <lijiang@xxxxxxxxxx> wrote: > >> >> > >> >> Hi, Kazu and Qianli > >> >> > >> >> On Tue, Jun 28, 2022 at 9:17 AM HAGIO KAZUHITO(萩尾 一仁) <k-hagio-ab@xxxxxxx> wrote: > >> >>> > >> >>> Hi Qianli, > >> >>> > >> >>> thanks for the patch and explanation. I was off. > >> >>> > >> >>> On 2022/06/27 11:24, qianli zhao wrote: > >> >>> > Hi,Kazu > >> >>> > > >> >>> > Would you like to help review this patch? > >> >>> > >> >>> Sure, I think I can review it this week. > >> >>> > >> >>> Lianbo, can you possibly reproduce and test this? > >> >> > >> >> > >> >> OK, I will have a look and give feedback later. > >> > > >> > > >> > Could you please point out the kernel version? I tried it with the latest kernel and did not reproduce this issue when disabling the kaslr feature(# CONFIG_RANDOMIZE_BASE is not set or nokaslr) > >> > > >> > crash > help -m > >> > .. > >> > vmalloc_start_addr: ffff800008000000 > >> > vmalloc_end: fffffbffefffffff > >> > modules_vaddr: ffff800000000000 > >> > modules_end: ffff800007ffffff > >> > vmemmap_vaddr: fffffc0000000000 > >> > vmemmap_end: fffffdffffffffff > >> > ... > >> > > >> > And the following information is dumped from the kernel(commit: 941e3e791269) > >> > ... > >> > [ 0.000000] Virtual kernel memory layout: > >> > [ 0.000000] modules : 0xffff800000000000 - 0xffff800008000000 ( 128 MB) > >> > [ 0.000000] vmalloc : 0xffff800008000000 - 0xfffffbfff0000000 (126975 GB) > >> > ... > >> > [ 0.000000] vmemmap : 0xfffffc0000000000 - 0xfffffe0000000000 ( 2048 GB maximum) > >> > [ 0.000000] 0xfffffc000000bc00 - 0xfffffc023df40000 ( 9183 MB actual) > >> > > >> > I'm wondering if that can be only reproduced on the old kernel, right? Or did I miss anything else? > >> > > >> > Thanks. > >> > Lianbo > >> > > >> >> > >> >>> > >> >>> Kazu > >> >>> > >> >>> > > >> >>> > qianli zhao <zhaoqianligood@xxxxxxxxx> 于2022年6月24日周五 10:56写道： > >> >>> > > >> >>> >> > >> >>> >> Hi,all > >> >>> >> > >> >>> >> Here's some explanation for this patch > >> >>> >> > >> >>> >> Without patch: > >> >>> >> Consider the following scenario > >> >>> >> ->arm64_init(PRE_GDB) > >> >>> >> case PRE_GDB: > >> >>> >> ... > >> >>> >> 292 } else if (ms->VA_BITS_ACTUAL) { > >> >>> >> 293 ms->modules_vaddr = > >> >>> >> (st->_stext_vmlinux & TEXT_OFFSET_MASK) - > >> >>> >> ARM64_MODULES_VSIZE;-->//ms->modules_vaddr=0xfffffffff8000000 > >> >>> >> 294 ms->modules_end = > >> >>> >> ms->modules_vaddr + ARM64_MODULES_VSIZE > >> >>> >> -1;--->//ms->modules_end=0xffffffffffffffff > >> >>> >> 295 ms->vmalloc_start_addr = > >> >>> >> ms->modules_end + 1;--->//ms->vmalloc_start_addr=0 > >> >>> >> 296 } else { > >> >>> >> .... > >> >>> >> } > >> >>> >> arm64_calc_kimage_voffset(); > >> >>> >> ..... > >> >>> >> > >> >>> >> Since arm64_calc_kimage_voffset() depends on vmalloc_start_addr, > >> >>> >> kimage_voffset cannot be calculated correctly. > >> >>> >> > >> >>> >> st->_stext_vmlinux can be initialized in numeric_forward(),just set > >> >>> >> st->_stext_vmlinux to UNINITIALIZED. > >> >>> >> > >> >>> >> ============ > >> >>> >> log as below: > >> >>> >> > >> >>> >> $ ~/crash/crash/crash vmlinux DDRCS0.bin@0x80000000 --machdep vabits_actual=48 > >> >>> >> > >> >>> >> crash 8.0.1++ > >> >>> >> Copyright (C) 2002-2022 Red Hat, Inc. > >> >>> >> Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation > >> >>> >> Copyright (C) 1999-2006 Hewlett-Packard Co > >> >>> >> Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited > >> >>> >> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. > >> >>> >> Copyright (C) 2005, 2011, 2020-2022 NEC Corporation > >> >>> >> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. > >> >>> >> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. > >> >>> >> Copyright (C) 2015, 2021 VMware, Inc. > >> >>> >> This program is free software, covered by the GNU General Public License, > >> >>> >> and you are welcome to change it and/or distribute copies of it under > >> >>> >> certain conditions. Enter "help copying" to see the conditions. > >> >>> >> This program has absolutely no warranty. Enter "help warranty" for details. > >> >>> >> > >> >>> >> NOTE: setting vabits_actual to: 48 > >> >>> >> > >> >>> >> WARNING: kimage_voffset cannot be determined from the dumpfile. > >> >>> >> Try using the command line option: --machdep kimage_voffset=<addr> > >> >>> >> GNU gdb (GDB) 10.2 > >> >>> >> Copyright (C) 2021 Free Software Foundation, Inc. > >> >>> >> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > >> >>> >> This is free software: you are free to change and redistribute it. > >> >>> >> There is NO WARRANTY, to the extent permitted by law. > >> >>> >> Type "show copying" and "show warranty" for details. > >> >>> >> This GDB was configured as "--host=x86_64-pc-linux-gnu > >> >>> >> --target=aarch64-elf-linux". > >> >>> >> Type "show configuration" for configuration details. > >> >>> >> Find the GDB manual and other documentation resources online at: > >> >>> >> <http://www.gnu.org/software/gdb/documentation/>. > >> >>> >> > >> >>> >> For help, type "help". > >> >>> >> Type "apropos word" to search for commands related to "word"... > >> >>> >> > >> >>> >> crash: read error: kernel virtual address: ffff80001083d4a0 type: > >> >>> >> "kernel_config_data" > >> >>> >> WARNING: cannot read kernel_config_data > >> >>> >> crash: read error: kernel virtual address: ffff80001170e798 type: "possible" > >> >>> >> WARNING: cannot read cpu_possible_map > >> >>> >> crash: read error: kernel virtual address: ffff80001170e7a8 type: "present" > >> >>> >> WARNING: cannot read cpu_present_map > >> >>> >> crash: read error: kernel virtual address: ffff80001170e788 type: "online" > >> >>> >> WARNING: cannot read cpu_online_map > >> >>> >> crash: read error: kernel virtual address: ffff80001170e7c0 type: "active" > >> >>> >> WARNING: cannot read cpu_active_map > >> >>> >> crash: read error: kernel virtual address: ffff8000122e00f0 type: > >> >>> >> "shadow_timekeeper xtime_sec" > >> >>> >> crash: read error: kernel virtual address: ffff80001171dc04 type: "init_uts_ns" > >> >>> >> crash: vmlinux and /var/tmp/ramdump_elf_m2ivkg do not match! > >> >>> >> > >> >>> >> Usage: > >> >>> >> > >> >>> >> crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form) > >> >>> >> crash [OPTION]... [NAMELIST] (live system form) > >> >>> >> > >> >>> >> Enter "crash -h" for details. > >> >>> >> > >> >>> >> Qianli Zhao <zhaoqianligood@xxxxxxxxx> 于2022年6月24日周五 00:14写道： > >> >>> >>> > >> >>> >>> From: Qianli Zhao <qianli.zhao@xxxxxxxxxx> > >> >>> >>> > >> >>> >>> Setting st->_stext_vmlinux to UNINITIALIZED to search for "_stext" from the vmlinux > >> >>> >>> Without the patch, if we do not enable kaslr, will get the wrong > >> >>> >>> MODULES/VMALLOC ranges, cause parsing dump failure > >> >>> >>> > >> >>> >>> Signed-off-by: Qianli Zhao <qianli.zhao@xxxxxxxxxx> > >> >>> >>> --- > >> >>> >>> arm64.c | 3 +++ > >> >>> >>> 1 file changed, 3 insertions(+) > >> >>> >>> > >> >>> >>> diff --git a/arm64.c b/arm64.c > >> >>> >>> index 0f615cf..4458a66 100644 > >> >>> >>> --- a/arm64.c > >> >>> >>> +++ b/arm64.c > >> >>> >>> @@ -149,6 +149,9 @@ arm64_init(int when) > >> >>> >>> > >> >>> >>> ms = machdep->machspec; > >> >>> >>> > >> >>> >>> + if (ms->VA_BITS_ACTUAL) > >> >>> >>> + st->_stext_vmlinux = UNINITIALIZED; > >> >>> >>> + > >> >>> >>> if (!ms->kimage_voffset && STREQ(pc->live_memsrc, "/dev/crash")) > >> >>> >>> ioctl(pc->mfd, DEV_CRASH_ARCH_DATA, &ms->kimage_voffset); > >> >>> >>> > >> >>> >>> -- > >> >>> >>> 2.17.1 > >> >>> >>> > >> -- Crash-utility mailing list Crash-utility@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/crash-utility Contribution Guidelines: https://github.com/crash-utility/crash/wiki
