Hi Jose,
On Fri, Aug 2, 2024 at 11:50 PM Jose Fernandez <jose.fernandez@xxxxxxxxx> wrote:
>
> On 24/08/02 06:54PM, Tao Liu wrote:
> > Hi Jose,
> >
> > Thanks for reporting the issue. It looks like a known issue which we
> > are currently dealing with.
> >
> > On Fri, Aug 2, 2024 at 6:11 PM Jose Fernandez <jose.fernandez@xxxxxxxxx> wrote:
> > >
> > > I'm trying to get crash to work on Arch Linux but I keep running into this error:
> > >
> > > crash: seek error: kernel virtual address: ffffffff82b195c0 type: "page_offset_base"
> > >
> > It looks like some kaslr issue to me, could you please execute the
> > following cmd and show me the results?
> >
> > $ nm vmlinux | grep kaslr_regions
> >
> > If there is no output, aka no such symbol is found, then you hit a
> > known kaslr issue.
>
> Hi Tao, here is the output for the CachyOS kernel:
>
> nm /usr/src/debug/linux-cachyos/vmlinux | grep kaslr
> ffffffff820af590 T init_trampoline_kaslr
> ffffffff82014e60 T kaslr_get_random_long
> ffffffff83a9d940 d kaslr_regions
> ffffffff820af580 T __pfx_init_trampoline_kaslr
> ffffffff82014e50 T __pfx_kaslr_get_random_long
>
> And here is the output for the upstream 6.11RC I built myself:
> nm vmlinux | grep kaslr
> ffffffff810ae060 T init_trampoline_kaslr
> ffffffff81deb600 T kaslr_get_random_long
> ffffffff83871940 d kaslr_regions
> ffffffff810ae050 T __pfx_init_trampoline_kaslr
> ffffffff81deb5f0 T __pfx_kaslr_get_random_long
>
Thanks for the info, looks like a different issue than which I'm
handling. I'm new to archlinux, how can I install the
"/usr/src/debug/linux-cachyos/vmlinux" in CachyOS? I'd like to have a
debug on crash.
Thanks,
Tao Liu
> Thanks,
> Jose
>
> >
> > Thanks,
> > Tao Liu
> >
> > > I've the upstream 6.11RC and the latest CachyOS kernel (6.10.2). I boot into
> > > the target kernel and then run crash with the respective vmlinux file. I'm
> > > simply trying to load /proc/kcore with this command:
> > >
> > > crash /usr/src/debug/linux-cachyos/vmlinux /proc/kcore -d10
> > >
> > > Here is the full output:
> > >
> > > crash 8.0.5
> > > Copyright (C) 2002-2024 Red Hat, Inc.
> > > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
> > > Copyright (C) 1999-2006 Hewlett-Packard Co
> > > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
> > > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > > Copyright (C) 2005, 2011, 2020-2024 NEC Corporation
> > > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > > Copyright (C) 2015, 2021 VMware, Inc.
> > > This program is free software, covered by the GNU General Public License,
> > > and you are welcome to change it and/or distribute copies of it under
> > > certain conditions. Enter "help copying" to see the conditions.
> > > This program has absolutely no warranty. Enter "help warranty" for details.
> > >
> > > get_live_memory_source: /proc/kcore
> > > proc_kcore_data:
> > > flags: 500 (KCORE_LOCAL|KCORE_ELF64)
> > > segments: 28
> > > elf_header: 64e6aab34860
> > > header_size: 11828
> > > notes64: 64e6aab348a0
> > > load64: 64e6aab348d8
> > > notes32: 0
> > > load32: 0
> > > vmcoreinfo: 0
> > > size_vmcoreinfo: 0
> > >
> > > Elf64_Phdr:
> > > p_type: 4 (PT_NOTE)
> > > p_flags: 0
> > > p_offset: 698
> > > p_vaddr: 0
> > > p_paddr: 0
> > > p_filesz: 10140
> > > p_memsz: 0
> > > p_align: 0
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 7fffbc403000
> > > p_vaddr: ffffffffbc400000
> > > p_paddr: 915000000
> > > p_filesz: 50528256
> > > p_memsz: 50528256
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 277fc0003000
> > > p_vaddr: ffffa77fc0000000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 35184372088831
> > > p_memsz: 35184372088831
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 7fffc0003000
> > > p_vaddr: ffffffffc0000000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 1056964608
> > > p_memsz: 1056964608
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec00004000
> > > p_vaddr: ffff8fec00001000
> > > p_paddr: 1000
> > > p_filesz: 651264
> > > p_memsz: 651264
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 727480003000
> > > p_vaddr: fffff27480000000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 12288
> > > p_memsz: 12288
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec00103000
> > > p_vaddr: ffff8fec00100000
> > > p_paddr: 100000
> > > p_filesz: 162529280
> > > p_memsz: 162529280
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 727480007000
> > > p_vaddr: fffff27480004000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 2539520
> > > p_memsz: 2539520
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec09de4000
> > > p_vaddr: ffff8fec09de1000
> > > p_paddr: 9de1000
> > > p_filesz: 1175552
> > > p_memsz: 1175552
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 72748027a000
> > > p_vaddr: fffff27480277000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 20480
> > > p_memsz: 20480
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec09f3f000
> > > p_vaddr: ffff8fec09f3c000
> > > p_paddr: 9f3c000
> > > p_filesz: 169684992
> > > p_memsz: 169684992
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 72748027f000
> > > p_vaddr: fffff2748027c000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 2658304
> > > p_memsz: 2658304
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec18313000
> > > p_vaddr: ffff8fec18310000
> > > p_paddr: 18310000
> > > p_filesz: 14557184
> > > p_memsz: 14557184
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 72748060f000
> > > p_vaddr: fffff2748060c000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 229376
> > > p_memsz: 229376
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec190f6000
> > > p_vaddr: ffff8fec190f3000
> > > p_paddr: 190f3000
> > > p_filesz: 20480
> > > p_memsz: 20480
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec190fc000
> > > p_vaddr: ffff8fec190f9000
> > > p_paddr: 190f9000
> > > p_filesz: 29282304
> > > p_memsz: 29282304
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec1acea000
> > > p_vaddr: ffff8fec1ace7000
> > > p_paddr: 1ace7000
> > > p_filesz: 12288
> > > p_memsz: 12288
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 7274806b6000
> > > p_vaddr: fffff274806b3000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 4096
> > > p_memsz: 4096
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec1acee000
> > > p_vaddr: ffff8fec1aceb000
> > > p_paddr: 1aceb000
> > > p_filesz: 143360
> > > p_memsz: 143360
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec20e84000
> > > p_vaddr: ffff8fec20e81000
> > > p_paddr: 20e81000
> > > p_filesz: 694374400
> > > p_memsz: 694374400
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 72748083d000
> > > p_vaddr: fffff2748083a000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 10850304
> > > p_memsz: 10850304
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec4a6fa000
> > > p_vaddr: ffff8fec4a6f7000
> > > p_paddr: 4a6f7000
> > > p_filesz: 76910592
> > > p_memsz: 76910592
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 72748129e000
> > > p_vaddr: fffff2748129b000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 1208320
> > > p_memsz: 1208320
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec4f054000
> > > p_vaddr: ffff8fec4f051000
> > > p_paddr: 4f051000
> > > p_filesz: 71499776
> > > p_memsz: 71499776
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec58201000
> > > p_vaddr: ffff8fec581fe000
> > > p_paddr: 581fe000
> > > p_filesz: 31371264
> > > p_memsz: 31371264
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 72748160a000
> > > p_vaddr: fffff27481607000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 495616
> > > p_memsz: 495616
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fec59ff0000
> > > p_vaddr: ffff8fec59fed000
> > > p_paddr: 59fed000
> > > p_filesz: 8192
> > > p_memsz: 8192
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: fed00003000
> > > p_vaddr: ffff8fed00000000
> > > p_paddr: 100000000
> > > p_filesz: 66004713472
> > > p_memsz: 66004713472
> > > p_align: 4096
> > >
> > > Elf64_Phdr:
> > > p_type: 1 (PT_LOAD)
> > > p_flags: 7
> > > p_offset: 727484003000
> > > p_vaddr: fffff27484000000
> > > p_paddr: ffffffffffffffff
> > > p_filesz: 1031323648
> > > p_memsz: 1031323648
> > > p_align: 4096
> > >
> > > Elf64_Nhdr:
> > > n_namesz: 5 ("CORE")
> > > n_descsz: 336
> > > n_type: 1 (NT_PRSTATUS)
> > >
> > > Elf64_Nhdr:
> > > n_namesz: 5 ("CORE")
> > > n_descsz: 136
> > > n_type: 3 (NT_PRPSINFO)
> > >
> > > Elf64_Nhdr:
> > > n_namesz: 5 ("CORE")
> > > n_descsz: 6360
> > > n_type: 4 (NT_TASKSTRUCT)
> > >
> > > Elf64_Nhdr:
> > > n_namesz: 11 ("VMCOREINFO")
> > > n_descsz: 3224
> > > n_type: 0 (unknown)
> > >
> > > OSRELEASE=6.10.2-2-cachyos
> > > BUILD-ID=464ddea46107c047dd447fc06ce515bc716030fc
> > > PAGESIZE=4096
> > > SYMBOL(init_uts_ns)=ffffffffbe7c70c0
> > > OFFSET(uts_namespace.name)=0
> > > SYMBOL(node_online_map)=ffffffffbe81bb48
> > > SYMBOL(swapper_pg_dir)=ffffffffbe620000
> > > SYMBOL(_stext)=ffffffffbc400000
> > > NUMBER(VMALLOC_START)=0xffffa77fc0000000
> > > SYMBOL(vmemmap)=fffff27480000000
> > > SYMBOL(mem_section)=ffff8ffc5e2f7000
> > > LENGTH(mem_section)=2048
> > > SIZE(mem_section)=16
> > > OFFSET(mem_section.section_mem_map)=0
> > > NUMBER(SECTION_SIZE_BITS)=27
> > > NUMBER(MAX_PHYSMEM_BITS)=46
> > > SIZE(page)=64
> > > SIZE(pglist_data)=16704
> > > SIZE(zone)=1728
> > > SIZE(free_area)=104
> > > SIZE(list_head)=16
> > > SIZE(nodemask_t)=8
> > > OFFSET(page.flags)=0
> > > OFFSET(page._refcount)=52
> > > OFFSET(page.mapping)=24
> > > OFFSET(page.lru)=8
> > > OFFSET(page._mapcount)=48
> > > OFFSET(page.private)=40
> > > OFFSET(page.compound_head)=8
> > > OFFSET(pglist_data.node_zones)=0
> > > OFFSET(pglist_data.nr_zones)=13792
> > > OFFSET(pglist_data.node_start_pfn)=13800
> > > OFFSET(pglist_data.node_spanned_pages)=13816
> > > OFFSET(pglist_data.node_id)=13824
> > > OFFSET(zone.free_area)=256
> > > OFFSET(zone.vm_stat)=1536
> > > OFFSET(zone.spanned_pages)=152
> > > OFFSET(free_area.free_list)=0
> > > OFFSET(list_head.next)=0
> > > OFFSET(list_head.prev)=8
> > > LENGTH(zone.free_area)=11
> > > SYMBOL(prb)=ffffffffbe65b940
> > > SYMBOL(printk_rb_static)=ffffffffbe65b960
> > > SYMBOL(clear_seq)=ffffffffbf1351a0
> > > SIZE(printk_ringbuffer)=88
> > > OFFSET(printk_ringbuffer.desc_ring)=0
> > > OFFSET(printk_ringbuffer.text_data_ring)=48
> > > OFFSET(printk_ringbuffer.fail)=80
> > > SIZE(prb_desc_ring)=48
> > > OFFSET(prb_desc_ring.count_bits)=0
> > > OFFSET(prb_desc_ring.descs)=8
> > > OFFSET(prb_desc_ring.infos)=16
> > > OFFSET(prb_desc_ring.head_id)=24
> > > OFFSET(prb_desc_ring.tail_id)=32
> > > SIZE(prb_desc)=24
> > > OFFSET(prb_desc.state_var)=0
> > > OFFSET(prb_desc.text_blk_lpos)=8
> > > SIZE(prb_data_blk_lpos)=16
> > > OFFSET(prb_data_blk_lpos.begin)=0
> > > OFFSET(prb_data_blk_lpos.next)=8
> > > SIZE(printk_info)=88
> > > OFFSET(printk_info.seq)=0
> > > OFFSET(printk_info.ts_nsec)=8
> > > OFFSET(printk_info.text_len)=16
> > > OFFSET(printk_info.caller_id)=20
> > > OFFSET(printk_info.dev_info)=24
> > > SIZE(dev_printk_info)=64
> > > OFFSET(dev_printk_info.subsystem)=0
> > > LENGTH(printk_info_subsystem)=16
> > > OFFSET(dev_printk_info.device)=16
> > > LENGTH(printk_info_device)=48
> > > SIZE(prb_data_ring)=32
> > > OFFSET(prb_data_ring.size_bits)=0
> > > OFFSET(prb_data_ring.data)=8
> > > OFFSET(prb_data_ring.head_lpos)=16
> > > OFFSET(prb_data_ring.tail_lpos)=24
> > > SIZE(atomic_long_t)=8
> > > OFFSET(atomic_long_t.counter)=0
> > > SIZE(latched_seq)=24
> > > OFFSET(latched_seq.val)=8
> > > LENGTH(free_area.free_list)=6
> > > NUMBER(NR_FREE_PAGES)=0
> > > NUMBER(PG_lru)=5
> > > NUMBER(PG_private)=14
> > > NUMBER(PG_swapcache)=11
> > > NUMBER(PG_swapbacked)=18
> > > NUMBER(PAGE_SLAB_MAPCOUNT_VALUE)=-4097
> > > NUMBER(PG_hwpoison)=22
> > > NUMBER(PG_head_mask)=64
> > > NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129
> > > NUMBER(PAGE_HUGETLB_MAPCOUNT_VALUE)=-2049
> > > NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257
> > > SYMBOL(kallsyms_names)=ffffffffbda452f8
> > > SYMBOL(kallsyms_num_syms)=ffffffffbda452f0
> > > SYMBOL(kallsyms_token_table)=ffffffffbdc94af8
> > > SYMBOL(kallsyms_token_index)=ffffffffbdc94e98
> > > SYMBOL(kallsyms_offsets)=ffffffffbdc95098
> > > SYMBOL(kallsyms_relative_base)=ffffffffbdd549d8
> > > NUMBER(phys_base)=37996199936
> > > SYMBOL(init_top_pgt)=ffffffffbe620000
> > > NUMBER(pgtable_l5_enabled)=0
> > > SYMBOL(node_data)=ffffffffbe81a620
> > > LENGTH(node_data)=32
> > > KERNELOFFSET=3b400000
> > > NUMBER(KERNEL_IMAGE_SIZE)=1073741824
> > > NUMBER(sme_mask)=0
> > >
> > > /proc/version:
> > > Linux version 6.10.2-2-cachyos (linux-cachyos@cachyos) (gcc (GCC) 14.1.1 20240720, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Sat, 27 Jul 2024 19:37:59 +0000
> > > /usr/src/debug/linux-cachyos/vmlinux:
> > > Linux version 6.10.2-2-cachyos (linux-cachyos@cachyos) (gcc (GCC) 14.1.1 20240720, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Sat, 27 Jul 2024 19:37:59 +0000
> > > readmem: read_proc_kcore() -> /proc/kcore
> > > crash: pv_ops exists: ARCH_PVOPS
> > > VMCOREINFO: NUMBER(phys_base): 37996199936 -> 8d8c00000
> > > gdb /usr/src/debug/linux-cachyos/vmlinux
> > > GNU gdb (GDB) 10.2
> > > Copyright (C) 2021 Free Software Foundation, Inc.
> > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> > > This is free software: you are free to change and redistribute it.
> > > There is NO WARRANTY, to the extent permitted by law.
> > > Type "show copying" and "show warranty" for details.
> > > This GDB was configured as "x86_64-pc-linux-gnu".
> > > Type "show configuration" for configuration details.
> > > Find the GDB manual and other documentation resources online at:
> > > <http://www.gnu.org/software/gdb/documentation/>.
> > >
> > > For help, type "help".
> > > Type "apropos word" to search for commands related to "word"...
> > > GETBUF(344 -> 0)
> > > GETBUF(1500 -> 1)
> > >
> > > FREEBUF(1)
> > > FREEBUF(0)
> > > <readmem: ffffffff82b195c0, KVADDR, "page_offset_base", 8, (FOE|Q), 64e6a75445e8>
> > > <read_proc_kcore: addr: ffffffff82b195c0 paddr: 8db7195c0 cnt: 8>
> > > crash: seek error: kernel virtual address: ffffffff82b195c0 type: "page_offset_base"
> > > [root@archlinux jose]#
> > > --
> > > Crash-utility mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxxxxxx
> > > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxxxxxx
> > > https://${domain_name}/admin/lists/devel.lists.crash-utility.osci.io/
> > > Contribution Guidelines: https://github.com/crash-utility/crash/wiki
> >
>
--
Crash-utility mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxxxxxx
https://${domain_name}/admin/lists/devel.lists.crash-utility.osci.io/
Contribution Guidelines: https://github.com/crash-utility/crash/wiki
