On 24/08/03 09:54AM, Tao Liu wrote: > Hi Jose, > > On Fri, Aug 2, 2024 at 11:50 PM Jose Fernandez <jose.fernandez@xxxxxxxxx> wrote: > > > > On 24/08/02 06:54PM, Tao Liu wrote: > > > Hi Jose, > > > > > > Thanks for reporting the issue. It looks like a known issue which we > > > are currently dealing with. > > > > > > On Fri, Aug 2, 2024 at 6:11 PM Jose Fernandez <jose.fernandez@xxxxxxxxx> wrote: > > > > > > > > I'm trying to get crash to work on Arch Linux but I keep running into this error: > > > > > > > > crash: seek error: kernel virtual address: ffffffff82b195c0 type: "page_offset_base" > > > > > > > It looks like some kaslr issue to me, could you please execute the > > > following cmd and show me the results? > > > > > > $ nm vmlinux | grep kaslr_regions > > > > > > If there is no output, aka no such symbol is found, then you hit a > > > known kaslr issue. > > > > Hi Tao, here is the output for the CachyOS kernel: > > > > nm /usr/src/debug/linux-cachyos/vmlinux | grep kaslr > > ffffffff820af590 T init_trampoline_kaslr > > ffffffff82014e60 T kaslr_get_random_long > > ffffffff83a9d940 d kaslr_regions > > ffffffff820af580 T __pfx_init_trampoline_kaslr > > ffffffff82014e50 T __pfx_kaslr_get_random_long > > > > And here is the output for the upstream 6.11RC I built myself: > > nm vmlinux | grep kaslr > > ffffffff810ae060 T init_trampoline_kaslr > > ffffffff81deb600 T kaslr_get_random_long > > ffffffff83871940 d kaslr_regions > > ffffffff810ae050 T __pfx_init_trampoline_kaslr > > ffffffff81deb5f0 T __pfx_kaslr_get_random_long > > > Thanks for the info, looks like a different issue than which I'm > handling. I'm new to archlinux, how can I install the > "/usr/src/debug/linux-cachyos/vmlinux" in CachyOS? I'd like to have a > debug on crash. Tao, you need first need to add the CachyOS repositories. Instructions here: https://wiki.cachyos.org/cachyos_repositories/how_to_add_cachyos_repo/ Then you can install the kernel with: sudo pacman -S linux-cachyos and the vmlinux with the dbg package: sudo pacman -S linux-cachyos-dbg That last package will place vmlinux under: /usr/src/debug/linux-cachyos/vmlinux > > Thanks, > Tao Liu > > > Thanks, > > Jose > > > > > > > > Thanks, > > > Tao Liu > > > > > > > I've the upstream 6.11RC and the latest CachyOS kernel (6.10.2). I boot into > > > > the target kernel and then run crash with the respective vmlinux file. I'm > > > > simply trying to load /proc/kcore with this command: > > > > > > > > crash /usr/src/debug/linux-cachyos/vmlinux /proc/kcore -d10 > > > > > > > > Here is the full output: > > > > > > > > crash 8.0.5 > > > > Copyright (C) 2002-2024 Red Hat, Inc. > > > > Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation > > > > Copyright (C) 1999-2006 Hewlett-Packard Co > > > > Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited > > > > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. > > > > Copyright (C) 2005, 2011, 2020-2024 NEC Corporation > > > > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. > > > > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. > > > > Copyright (C) 2015, 2021 VMware, Inc. > > > > This program is free software, covered by the GNU General Public License, > > > > and you are welcome to change it and/or distribute copies of it under > > > > certain conditions. Enter "help copying" to see the conditions. > > > > This program has absolutely no warranty. Enter "help warranty" for details. > > > > > > > > get_live_memory_source: /proc/kcore > > > > proc_kcore_data: > > > > flags: 500 (KCORE_LOCAL|KCORE_ELF64) > > > > segments: 28 > > > > elf_header: 64e6aab34860 > > > > header_size: 11828 > > > > notes64: 64e6aab348a0 > > > > load64: 64e6aab348d8 > > > > notes32: 0 > > > > load32: 0 > > > > vmcoreinfo: 0 > > > > size_vmcoreinfo: 0 > > > > > > > > Elf64_Phdr: > > > > p_type: 4 (PT_NOTE) > > > > p_flags: 0 > > > > p_offset: 698 > > > > p_vaddr: 0 > > > > p_paddr: 0 > > > > p_filesz: 10140 > > > > p_memsz: 0 > > > > p_align: 0 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 7fffbc403000 > > > > p_vaddr: ffffffffbc400000 > > > > p_paddr: 915000000 > > > > p_filesz: 50528256 > > > > p_memsz: 50528256 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 277fc0003000 > > > > p_vaddr: ffffa77fc0000000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 35184372088831 > > > > p_memsz: 35184372088831 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 7fffc0003000 > > > > p_vaddr: ffffffffc0000000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 1056964608 > > > > p_memsz: 1056964608 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec00004000 > > > > p_vaddr: ffff8fec00001000 > > > > p_paddr: 1000 > > > > p_filesz: 651264 > > > > p_memsz: 651264 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 727480003000 > > > > p_vaddr: fffff27480000000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 12288 > > > > p_memsz: 12288 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec00103000 > > > > p_vaddr: ffff8fec00100000 > > > > p_paddr: 100000 > > > > p_filesz: 162529280 > > > > p_memsz: 162529280 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 727480007000 > > > > p_vaddr: fffff27480004000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 2539520 > > > > p_memsz: 2539520 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec09de4000 > > > > p_vaddr: ffff8fec09de1000 > > > > p_paddr: 9de1000 > > > > p_filesz: 1175552 > > > > p_memsz: 1175552 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 72748027a000 > > > > p_vaddr: fffff27480277000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 20480 > > > > p_memsz: 20480 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec09f3f000 > > > > p_vaddr: ffff8fec09f3c000 > > > > p_paddr: 9f3c000 > > > > p_filesz: 169684992 > > > > p_memsz: 169684992 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 72748027f000 > > > > p_vaddr: fffff2748027c000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 2658304 > > > > p_memsz: 2658304 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec18313000 > > > > p_vaddr: ffff8fec18310000 > > > > p_paddr: 18310000 > > > > p_filesz: 14557184 > > > > p_memsz: 14557184 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 72748060f000 > > > > p_vaddr: fffff2748060c000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 229376 > > > > p_memsz: 229376 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec190f6000 > > > > p_vaddr: ffff8fec190f3000 > > > > p_paddr: 190f3000 > > > > p_filesz: 20480 > > > > p_memsz: 20480 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec190fc000 > > > > p_vaddr: ffff8fec190f9000 > > > > p_paddr: 190f9000 > > > > p_filesz: 29282304 > > > > p_memsz: 29282304 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec1acea000 > > > > p_vaddr: ffff8fec1ace7000 > > > > p_paddr: 1ace7000 > > > > p_filesz: 12288 > > > > p_memsz: 12288 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 7274806b6000 > > > > p_vaddr: fffff274806b3000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 4096 > > > > p_memsz: 4096 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec1acee000 > > > > p_vaddr: ffff8fec1aceb000 > > > > p_paddr: 1aceb000 > > > > p_filesz: 143360 > > > > p_memsz: 143360 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec20e84000 > > > > p_vaddr: ffff8fec20e81000 > > > > p_paddr: 20e81000 > > > > p_filesz: 694374400 > > > > p_memsz: 694374400 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 72748083d000 > > > > p_vaddr: fffff2748083a000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 10850304 > > > > p_memsz: 10850304 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec4a6fa000 > > > > p_vaddr: ffff8fec4a6f7000 > > > > p_paddr: 4a6f7000 > > > > p_filesz: 76910592 > > > > p_memsz: 76910592 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 72748129e000 > > > > p_vaddr: fffff2748129b000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 1208320 > > > > p_memsz: 1208320 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec4f054000 > > > > p_vaddr: ffff8fec4f051000 > > > > p_paddr: 4f051000 > > > > p_filesz: 71499776 > > > > p_memsz: 71499776 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec58201000 > > > > p_vaddr: ffff8fec581fe000 > > > > p_paddr: 581fe000 > > > > p_filesz: 31371264 > > > > p_memsz: 31371264 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 72748160a000 > > > > p_vaddr: fffff27481607000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 495616 > > > > p_memsz: 495616 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fec59ff0000 > > > > p_vaddr: ffff8fec59fed000 > > > > p_paddr: 59fed000 > > > > p_filesz: 8192 > > > > p_memsz: 8192 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: fed00003000 > > > > p_vaddr: ffff8fed00000000 > > > > p_paddr: 100000000 > > > > p_filesz: 66004713472 > > > > p_memsz: 66004713472 > > > > p_align: 4096 > > > > > > > > Elf64_Phdr: > > > > p_type: 1 (PT_LOAD) > > > > p_flags: 7 > > > > p_offset: 727484003000 > > > > p_vaddr: fffff27484000000 > > > > p_paddr: ffffffffffffffff > > > > p_filesz: 1031323648 > > > > p_memsz: 1031323648 > > > > p_align: 4096 > > > > > > > > Elf64_Nhdr: > > > > n_namesz: 5 ("CORE") > > > > n_descsz: 336 > > > > n_type: 1 (NT_PRSTATUS) > > > > > > > > Elf64_Nhdr: > > > > n_namesz: 5 ("CORE") > > > > n_descsz: 136 > > > > n_type: 3 (NT_PRPSINFO) > > > > > > > > Elf64_Nhdr: > > > > n_namesz: 5 ("CORE") > > > > n_descsz: 6360 > > > > n_type: 4 (NT_TASKSTRUCT) > > > > > > > > Elf64_Nhdr: > > > > n_namesz: 11 ("VMCOREINFO") > > > > n_descsz: 3224 > > > > n_type: 0 (unknown) > > > > > > > > OSRELEASE=6.10.2-2-cachyos > > > > BUILD-ID=464ddea46107c047dd447fc06ce515bc716030fc > > > > PAGESIZE=4096 > > > > SYMBOL(init_uts_ns)=ffffffffbe7c70c0 > > > > OFFSET(uts_namespace.name)=0 > > > > SYMBOL(node_online_map)=ffffffffbe81bb48 > > > > SYMBOL(swapper_pg_dir)=ffffffffbe620000 > > > > SYMBOL(_stext)=ffffffffbc400000 > > > > NUMBER(VMALLOC_START)=0xffffa77fc0000000 > > > > SYMBOL(vmemmap)=fffff27480000000 > > > > SYMBOL(mem_section)=ffff8ffc5e2f7000 > > > > LENGTH(mem_section)=2048 > > > > SIZE(mem_section)=16 > > > > OFFSET(mem_section.section_mem_map)=0 > > > > NUMBER(SECTION_SIZE_BITS)=27 > > > > NUMBER(MAX_PHYSMEM_BITS)=46 > > > > SIZE(page)=64 > > > > SIZE(pglist_data)=16704 > > > > SIZE(zone)=1728 > > > > SIZE(free_area)=104 > > > > SIZE(list_head)=16 > > > > SIZE(nodemask_t)=8 > > > > OFFSET(page.flags)=0 > > > > OFFSET(page._refcount)=52 > > > > OFFSET(page.mapping)=24 > > > > OFFSET(page.lru)=8 > > > > OFFSET(page._mapcount)=48 > > > > OFFSET(page.private)=40 > > > > OFFSET(page.compound_head)=8 > > > > OFFSET(pglist_data.node_zones)=0 > > > > OFFSET(pglist_data.nr_zones)=13792 > > > > OFFSET(pglist_data.node_start_pfn)=13800 > > > > OFFSET(pglist_data.node_spanned_pages)=13816 > > > > OFFSET(pglist_data.node_id)=13824 > > > > OFFSET(zone.free_area)=256 > > > > OFFSET(zone.vm_stat)=1536 > > > > OFFSET(zone.spanned_pages)=152 > > > > OFFSET(free_area.free_list)=0 > > > > OFFSET(list_head.next)=0 > > > > OFFSET(list_head.prev)=8 > > > > LENGTH(zone.free_area)=11 > > > > SYMBOL(prb)=ffffffffbe65b940 > > > > SYMBOL(printk_rb_static)=ffffffffbe65b960 > > > > SYMBOL(clear_seq)=ffffffffbf1351a0 > > > > SIZE(printk_ringbuffer)=88 > > > > OFFSET(printk_ringbuffer.desc_ring)=0 > > > > OFFSET(printk_ringbuffer.text_data_ring)=48 > > > > OFFSET(printk_ringbuffer.fail)=80 > > > > SIZE(prb_desc_ring)=48 > > > > OFFSET(prb_desc_ring.count_bits)=0 > > > > OFFSET(prb_desc_ring.descs)=8 > > > > OFFSET(prb_desc_ring.infos)=16 > > > > OFFSET(prb_desc_ring.head_id)=24 > > > > OFFSET(prb_desc_ring.tail_id)=32 > > > > SIZE(prb_desc)=24 > > > > OFFSET(prb_desc.state_var)=0 > > > > OFFSET(prb_desc.text_blk_lpos)=8 > > > > SIZE(prb_data_blk_lpos)=16 > > > > OFFSET(prb_data_blk_lpos.begin)=0 > > > > OFFSET(prb_data_blk_lpos.next)=8 > > > > SIZE(printk_info)=88 > > > > OFFSET(printk_info.seq)=0 > > > > OFFSET(printk_info.ts_nsec)=8 > > > > OFFSET(printk_info.text_len)=16 > > > > OFFSET(printk_info.caller_id)=20 > > > > OFFSET(printk_info.dev_info)=24 > > > > SIZE(dev_printk_info)=64 > > > > OFFSET(dev_printk_info.subsystem)=0 > > > > LENGTH(printk_info_subsystem)=16 > > > > OFFSET(dev_printk_info.device)=16 > > > > LENGTH(printk_info_device)=48 > > > > SIZE(prb_data_ring)=32 > > > > OFFSET(prb_data_ring.size_bits)=0 > > > > OFFSET(prb_data_ring.data)=8 > > > > OFFSET(prb_data_ring.head_lpos)=16 > > > > OFFSET(prb_data_ring.tail_lpos)=24 > > > > SIZE(atomic_long_t)=8 > > > > OFFSET(atomic_long_t.counter)=0 > > > > SIZE(latched_seq)=24 > > > > OFFSET(latched_seq.val)=8 > > > > LENGTH(free_area.free_list)=6 > > > > NUMBER(NR_FREE_PAGES)=0 > > > > NUMBER(PG_lru)=5 > > > > NUMBER(PG_private)=14 > > > > NUMBER(PG_swapcache)=11 > > > > NUMBER(PG_swapbacked)=18 > > > > NUMBER(PAGE_SLAB_MAPCOUNT_VALUE)=-4097 > > > > NUMBER(PG_hwpoison)=22 > > > > NUMBER(PG_head_mask)=64 > > > > NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-129 > > > > NUMBER(PAGE_HUGETLB_MAPCOUNT_VALUE)=-2049 > > > > NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE)=-257 > > > > SYMBOL(kallsyms_names)=ffffffffbda452f8 > > > > SYMBOL(kallsyms_num_syms)=ffffffffbda452f0 > > > > SYMBOL(kallsyms_token_table)=ffffffffbdc94af8 > > > > SYMBOL(kallsyms_token_index)=ffffffffbdc94e98 > > > > SYMBOL(kallsyms_offsets)=ffffffffbdc95098 > > > > SYMBOL(kallsyms_relative_base)=ffffffffbdd549d8 > > > > NUMBER(phys_base)=37996199936 > > > > SYMBOL(init_top_pgt)=ffffffffbe620000 > > > > NUMBER(pgtable_l5_enabled)=0 > > > > SYMBOL(node_data)=ffffffffbe81a620 > > > > LENGTH(node_data)=32 > > > > KERNELOFFSET=3b400000 > > > > NUMBER(KERNEL_IMAGE_SIZE)=1073741824 > > > > NUMBER(sme_mask)=0 > > > > > > > > /proc/version: > > > > Linux version 6.10.2-2-cachyos (linux-cachyos@cachyos) (gcc (GCC) 14.1.1 20240720, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Sat, 27 Jul 2024 19:37:59 +0000 > > > > /usr/src/debug/linux-cachyos/vmlinux: > > > > Linux version 6.10.2-2-cachyos (linux-cachyos@cachyos) (gcc (GCC) 14.1.1 20240720, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Sat, 27 Jul 2024 19:37:59 +0000 > > > > readmem: read_proc_kcore() -> /proc/kcore > > > > crash: pv_ops exists: ARCH_PVOPS > > > > VMCOREINFO: NUMBER(phys_base): 37996199936 -> 8d8c00000 > > > > gdb /usr/src/debug/linux-cachyos/vmlinux > > > > GNU gdb (GDB) 10.2 > > > > Copyright (C) 2021 Free Software Foundation, Inc. > > > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > > > > This is free software: you are free to change and redistribute it. > > > > There is NO WARRANTY, to the extent permitted by law. > > > > Type "show copying" and "show warranty" for details. > > > > This GDB was configured as "x86_64-pc-linux-gnu". > > > > Type "show configuration" for configuration details. > > > > Find the GDB manual and other documentation resources online at: > > > > <http://www.gnu.org/software/gdb/documentation/>. > > > > > > > > For help, type "help". > > > > Type "apropos word" to search for commands related to "word"... > > > > GETBUF(344 -> 0) > > > > GETBUF(1500 -> 1) > > > > > > > > FREEBUF(1) > > > > FREEBUF(0) > > > > <readmem: ffffffff82b195c0, KVADDR, "page_offset_base", 8, (FOE|Q), 64e6a75445e8> > > > > <read_proc_kcore: addr: ffffffff82b195c0 paddr: 8db7195c0 cnt: 8> > > > > crash: seek error: kernel virtual address: ffffffff82b195c0 type: "page_offset_base" > > > > [root@archlinux jose]# > > > > -- > > > > Crash-utility mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxxxxxx > > > > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxxxxxx > > > > https://${domain_name}/admin/lists/devel.lists.crash-utility.osci.io/ > > > > Contribution Guidelines: https://github.com/crash-utility/crash/wiki > > > > > > -- Crash-utility mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxxxxxx https://${domain_name}/admin/lists/devel.lists.crash-utility.osci.io/ Contribution Guidelines: https://github.com/crash-utility/crash/wiki
