Re: [PATCH] arm64: Fix for st->_stext_vmlinux not initialized when set VA_BITS_ACTUAL

[lijiang <lijiang@xxxxxxxxxx>]

On Wed, Jun 29, 2022 at 2:40 PM qianli zhao <zhaoqianligood@xxxxxxxxx> wrote:

Hi,Lianbo
Thank you for your feedback.

My kernel version is 5.10.59, and this issue is happened when the

ramdumps without vmcoreinfo.

Thank you for the explanation.

It is important to point this(*without vmcoreinfo*) out in the patch log. That can help understand this issue. Can you add it to the patch log?

In addition, I just got a dump file generated by the "virsh dump" command, the dumpfile doesn't have the vmcoreinfo, and get the same errors 

with or without this patch, can you also double check? Maybe it is a similar issue.

[root@hpe-apollo-cn99xx-18 home]# /home/crash/crash vmlinux dump.core --machdep vabits_actual=48

crash 8.0.1++
Copyright (C) 2002-2022  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011, 2020-2022  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
Copyright (C) 2015, 2021  VMware, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
NOTE: setting vabits_actual to: 48

WARNING: kimage_voffset cannot be determined from the dumpfile.
       Try using the command line option: --machdep kimage_voffset=<addr>
GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu".
Type "show configuration" for configuration details.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...

crash: read error: kernel virtual address: ffff8000119cee00  type: "possible"
WARNING: cannot read cpu_possible_map
crash: read error: kernel virtual address: ffff8000119cf208  type: "present"
WARNING: cannot read cpu_present_map
crash: read error: kernel virtual address: ffff8000119cec00  type: "online"
WARNING: cannot read cpu_online_map
crash: read error: kernel virtual address: ffff8000119cf408  type: "active"
WARNING: cannot read cpu_active_map
crash: read error: kernel virtual address: ffff8000120c97a8  type: "shadow_timekeeper xtime_sec"
crash: read error: kernel virtual address: ffff8000119e1668  type: "init_uts_ns"
crash: vmlinux and dump.core do not match!

Usage:

  crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form)
  crash [OPTION]... [NAMELIST]             (live system form)

Enter "crash -h" for details.

Thanks.

Lianbo

I guess your scene may have vmcore,the following logic helps to
correctly locate MODULES/VMALLOC ranges with right kernel version.

 279                         ms->kimage_end = (sp ? sp->value : 0);
 280
 281                         if (ms->struct_page_size && (r =
arm64_get_va_range(ms))) {------------->//If there is a vmcore, it
will go to the following codes
 282                                 /* We can get all the
MODULES/VMALLOC/VMEMMAP ranges now.*/
 283                                 ms->modules_vaddr       = r->modules_vaddr;
 284                                 ms->modules_end         =
r->modules_end - 1;
 285                                 ms->vmalloc_start_addr  =
r->vmalloc_start_addr;
 286                                 ms->vmalloc_end         =
r->vmalloc_end - 1;
 287                                 ms->vmemmap_vaddr       = r->vmemmap_vaddr;
 288                                 ms->vmemmap_end         =
r->vmemmap_end - 1;
 289                         } else if (ms->VA_BITS_ACTUAL)
{------------>//unable to locate the version and other information
through vmcore
}

lijiang <lijiang@xxxxxxxxxx> 于2022年6月29日周三 14:21写道：
>
> Hi, Qianli
> On Tue, Jun 28, 2022 at 9:55 AM lijiang <lijiang@xxxxxxxxxx> wrote:
>>
>> Hi, Kazu and Qianli
>>
>> On Tue, Jun 28, 2022 at 9:17 AM HAGIO KAZUHITO(萩尾　一仁) <k-hagio-ab@xxxxxxx> wrote:
>>>
>>> Hi Qianli,
>>>
>>> thanks for the patch and explanation.  I was off.
>>>
>>> On 2022/06/27 11:24, qianli zhao wrote:
>>> > Hi,Kazu
>>> >
>>> > Would you like to help review this patch?
>>>
>>> Sure, I think I can review it this week.
>>>
>>> Lianbo, can you possibly reproduce and test this?
>>
>>
>> OK, I will have a look and give feedback later.
>
>
> Could you please point out the kernel version? I tried it with the latest kernel and did not reproduce this issue when disabling the kaslr feature(# CONFIG_RANDOMIZE_BASE is not set  or nokaslr)
>
> crash > help -m
> ..
>     vmalloc_start_addr: ffff800008000000
>            vmalloc_end: fffffbffefffffff
>          modules_vaddr: ffff800000000000
>            modules_end: ffff800007ffffff
>          vmemmap_vaddr: fffffc0000000000
>            vmemmap_end: fffffdffffffffff
> ...
>
> And the following information is dumped from the kernel(commit: 941e3e791269)
> ...
> [    0.000000] Virtual kernel memory layout:
> [    0.000000]     modules : 0xffff800000000000 - 0xffff800008000000   (   128 MB)
> [    0.000000]     vmalloc : 0xffff800008000000 - 0xfffffbfff0000000   (126975 GB)
> ...
> [    0.000000]     vmemmap : 0xfffffc0000000000 - 0xfffffe0000000000   (  2048 GB maximum)
> [    0.000000]               0xfffffc000000bc00 - 0xfffffc023df40000   (  9183 MB actual)
>
> I'm wondering if that can be only reproduced on the old kernel, right? Or did I miss anything else?
>
> Thanks.
> Lianbo
>
>>
>>>
>>> Kazu
>>>
>>> >
>>> > qianli zhao <zhaoqianligood@xxxxxxxxx> 于2022年6月24日周五 10:56写道：
>>> >
>>> >>
>>> >> Hi,all
>>> >>
>>> >> Here's some explanation for this patch
>>> >>
>>> >> Without patch:
>>> >> Consider the following scenario
>>> >> ->arm64_init(PRE_GDB)
>>> >> case PRE_GDB:
>>> >> ...
>>> >>   292                         } else if (ms->VA_BITS_ACTUAL) {
>>> >>   293                                 ms->modules_vaddr =
>>> >> (st->_stext_vmlinux & TEXT_OFFSET_MASK) -
>>> >> ARM64_MODULES_VSIZE;-->//ms->modules_vaddr=0xfffffffff8000000
>>> >>   294                                 ms->modules_end =
>>> >> ms->modules_vaddr + ARM64_MODULES_VSIZE
>>> >> -1;--->//ms->modules_end=0xffffffffffffffff
>>> >>   295                                 ms->vmalloc_start_addr =
>>> >> ms->modules_end + 1;--->//ms->vmalloc_start_addr=0
>>> >> 296                         } else {
>>> >>                                 ....
>>> >>                                 }
>>> >>                                 arm64_calc_kimage_voffset();
>>> >> .....
>>> >>
>>> >> Since arm64_calc_kimage_voffset() depends on vmalloc_start_addr,
>>> >> kimage_voffset cannot be calculated correctly.
>>> >>
>>> >> st->_stext_vmlinux can be initialized in numeric_forward(),just set
>>> >> st->_stext_vmlinux to UNINITIALIZED.
>>> >>
>>> >> ============
>>> >> log as below:
>>> >>
>>> >> $ ~/crash/crash/crash vmlinux DDRCS0.bin@0x80000000 --machdep vabits_actual=48
>>> >>
>>> >> crash 8.0.1++
>>> >> Copyright (C) 2002-2022  Red Hat, Inc.
>>> >> Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
>>> >> Copyright (C) 1999-2006  Hewlett-Packard Co
>>> >> Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
>>> >> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
>>> >> Copyright (C) 2005, 2011, 2020-2022  NEC Corporation
>>> >> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
>>> >> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
>>> >> Copyright (C) 2015, 2021  VMware, Inc.
>>> >> This program is free software, covered by the GNU General Public License,
>>> >> and you are welcome to change it and/or distribute copies of it under
>>> >> certain conditions.  Enter "help copying" to see the conditions.
>>> >> This program has absolutely no warranty.  Enter "help warranty" for details.
>>> >>
>>> >> NOTE: setting vabits_actual to: 48
>>> >>
>>> >> WARNING: kimage_voffset cannot be determined from the dumpfile.
>>> >>         Try using the command line option: --machdep kimage_voffset=<addr>
>>> >> GNU gdb (GDB) 10.2
>>> >> Copyright (C) 2021 Free Software Foundation, Inc.
>>> >> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>>> >> This is free software: you are free to change and redistribute it.
>>> >> There is NO WARRANTY, to the extent permitted by law.
>>> >> Type "show copying" and "show warranty" for details.
>>> >> This GDB was configured as "--host=x86_64-pc-linux-gnu
>>> >> --target=aarch64-elf-linux".
>>> >> Type "show configuration" for configuration details.
>>> >> Find the GDB manual and other documentation resources online at:
>>> >>      <http://www.gnu.org/software/gdb/documentation/>.
>>> >>
>>> >> For help, type "help".
>>> >> Type "apropos word" to search for commands related to "word"...
>>> >>
>>> >> crash: read error: kernel virtual address: ffff80001083d4a0  type:
>>> >> "kernel_config_data"
>>> >> WARNING: cannot read kernel_config_data
>>> >> crash: read error: kernel virtual address: ffff80001170e798  type: "possible"
>>> >> WARNING: cannot read cpu_possible_map
>>> >> crash: read error: kernel virtual address: ffff80001170e7a8  type: "present"
>>> >> WARNING: cannot read cpu_present_map
>>> >> crash: read error: kernel virtual address: ffff80001170e788  type: "online"
>>> >> WARNING: cannot read cpu_online_map
>>> >> crash: read error: kernel virtual address: ffff80001170e7c0  type: "active"
>>> >> WARNING: cannot read cpu_active_map
>>> >> crash: read error: kernel virtual address: ffff8000122e00f0  type:
>>> >> "shadow_timekeeper xtime_sec"
>>> >> crash: read error: kernel virtual address: ffff80001171dc04  type: "init_uts_ns"
>>> >> crash: vmlinux and /var/tmp/ramdump_elf_m2ivkg do not match!
>>> >>
>>> >> Usage:
>>> >>
>>> >>    crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS]     (dumpfile form)
>>> >>    crash [OPTION]... [NAMELIST]                          (live system form)
>>> >>
>>> >> Enter "crash -h" for details.
>>> >>
>>> >> Qianli Zhao <zhaoqianligood@xxxxxxxxx> 于2022年6月24日周五 00:14写道：
>>> >>>
>>> >>> From: Qianli Zhao <qianli.zhao@xxxxxxxxxx>
>>> >>>
>>> >>> Setting st->_stext_vmlinux to UNINITIALIZED to search for "_stext" from the vmlinux
>>> >>> Without the patch, if we do not enable kaslr, will get the wrong
>>> >>> MODULES/VMALLOC ranges, cause parsing dump failure
>>> >>>
>>> >>> Signed-off-by: Qianli Zhao <qianli.zhao@xxxxxxxxxx>
>>> >>> ---
>>> >>>   arm64.c | 3 +++
>>> >>>   1 file changed, 3 insertions(+)
>>> >>>
>>> >>> diff --git a/arm64.c b/arm64.c
>>> >>> index 0f615cf..4458a66 100644
>>> >>> --- a/arm64.c
>>> >>> +++ b/arm64.c
>>> >>> @@ -149,6 +149,9 @@ arm64_init(int when)
>>> >>>
>>> >>>                  ms = machdep->machspec;
>>> >>>
>>> >>> +               if (ms->VA_BITS_ACTUAL)
>>> >>> +                       st->_stext_vmlinux = UNINITIALIZED;
>>> >>> +
>>> >>>                  if (!ms->kimage_voffset && STREQ(pc->live_memsrc, "/dev/crash"))
>>> >>>                          ioctl(pc->mfd, DEV_CRASH_ARCH_DATA, &ms->kimage_voffset);
>>> >>>
>>> >>> --
>>> >>> 2.17.1
>>> >>>

--
Crash-utility mailing list
Crash-utility@xxxxxxxxxx
https://listman.redhat.com/mailman/listinfo/crash-utility
Contribution Guidelines: https://github.com/crash-utility/crash/wiki