Hi Kazu,
Thanks a lot for your effort and comments. Sorry I was working on some
other tasks yesterday and today. I will look into it later when my
task is done. Thanks again for your help!
Thanks,
Tao Liu
On Thu, Nov 9, 2023 at 4:37 PM HAGIO KAZUHITO(萩尾 一仁) <k-hagio-ab@xxxxxxx> wrote:
>
> On 2023/11/08 17:16, HAGIO KAZUHITO(萩尾 一仁) wrote:
> > On 2023/11/08 12:01, HAGIO KAZUHITO(萩尾 一仁) wrote:
> >> Hi Tao,
> >>
> >> thank you for the information.
> >>
> >> I'm looking into it, I noticed that the unexpected symbol "floopy_module_init"
> >> is in section .init.text. Crash side doesn't have the symbol info, probably
> >> the address is already freed and reused by the virtio_blk module?
> >>
> >> crash> mod -S
> >> ...
> >> crash> sym -m floppy | grep MODULE
> >> ffffffffc0092000 MODULE START: floppy
> >> ffffffffc00a2f29 MODULE END: floppy
> >> crash> sym -m virtio_blk | grep MODULE
> >> ffffffffc00a4000 MODULE START: virtio_blk
> >> ffffffffc00a86ec MODULE END: virtio_blk
> >> crash> gdb info symbol floppy_module_init
> >> init_module in section .init.text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> >> virtblk_freeze + 33 in section .text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/virtio_blk.ko.debug
> >> crash> gdb info address floppy_module_init
> >> Symbol "floppy_module_init" is a function at address 0xffffffffc00a4131.
> >> crash> sym floppy_module_init
> >> symbol not found: floppy_module_init
> >> possible alternatives:
> >> (none found)
> >>
> >> So still not sure, but this might be an issue of handling the init
> >> section/symbols of modules..
> >
> > Hmm, it looks like several sections are mapped outside of the current
> > module range. What if they are not specified or they are mapped to
> > e.g. an unused range..? or any idea?
>
> First, "mod -S -r" can reproduce this without the patch 1/2 or the commands.
> (If the root cause is fixed, I think no need to apply the patch 1/2.)
>
> crash-ups> mod -S -r
> crash-ups> p virtio_mq_ops
> virtio_mq_ops = $1 = {
> queue_rq = 0xffffffffc00a45b0 <floppy_module_init+1151>,
> map_queue = 0xffffffff813015c0 <blk_mq_map_queue>,
>
>
> The following trial patch, which does not map the .init.text section
> specifically, fixes the issue.
>
> --- a/symbols.c
> +++ b/symbols.c
> @@ -13283,7 +13283,7 @@ add_symbol_file_kallsyms(struct load_module *lm, struct gnu_request *req)
> shift_string_right(req->buf, strlen(buf));
> BCOPY(buf, req->buf, strlen(buf));
> retval = TRUE;
> - } else {
> + } else if (!STREQ(section_name, ".init.text")) {
> sprintf(buf, " -s %s 0x%lx", section_name, section_vaddr);
> while ((len + strlen(buf)) >= buflen) {
> RESIZEBUF(req->buf, buflen, buflen * 2);
>
> crash-dev> mod -S -r
> crash-dev> p virtio_mq_ops
> virtio_mq_ops = $1 = {
> queue_rq = 0xffffffffc00a45b0 <virtio_queue_rq>,
> map_queue = 0xffffffff813015c0 <blk_mq_map_queue>,
> {
>
>
> In this way, it looks like the .init.text sections of modules are
> mapped to 0x0, but many other sections like .modinfo are mapped here
> even without the trial patch.
>
> crash-dev> gdb info address floppy_module_init
> Symbol "floppy_module_init" is a function at address 0x131.
>
> crash-dev> gdb info symbol floppy_module_init
> irq_stack_union + 305 in section .data..percpu of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/vmlinux
> local_init + 305 in section .init.text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-mod.ko.debug
> ____versions + 49 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-region-hash.ko.debug
> ____versions + 145 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio.ko.debug
> __UNIQUE_ID_vermagic5 + 30 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-log.ko.debug
> __UNIQUE_ID_srcversion7 + 1 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-mirror.ko.debug
> __UNIQUE_ID_alias7 + 30 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio_pci.ko.debug
> ____versions + 145 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio_ring.ko.debug
> __UNIQUE_ID_alias16 + 36 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/ata/ata_generic.ko.debug
> ____versions + 17 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/fs/mbcache.ko.debug
> __UNIQUE_ID_vermagic5 + 45 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/net/virtio_net.ko.debug
> __UNIQUE_ID_vermagic5 + 44 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/cdrom/cdrom.ko.debug
> __UNIQUE_ID_vermagic5 + 14 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/scsi/sr_mod.ko.debug
> __module_depends + 1 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.debug
> ____versions + 81 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/fs/binfmt_misc.ko.debug
> init_module in section .init.text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> ...
>
> If a module already does not have its init memory range, it might be
> a bit better to not specify "-s .init.text <addr>" to add-symbol-file..
>
> Thanks,
> Kazu
>
>
> >
> > crash-dev> sym -m floppy | grep MODULE
> > ffffffffc0092000 MODULE START: floppy
> > ffffffffc00a2f29 MODULE END: floppy
> > crash-dev> set debug 1
> > debug: 1
> > crash-dev> mod -s floppy
> > ffffffffc009a000 .note.gnu.build-id
> > ffffffffc0092000 .text
> > ffffffffc00a4000 .init.text <<< higher than MODULE END
> > ffffffffc009971d .text.unlikely
> > ffffffffc00998ea .exit.text
> > ffffffffc009a040 .rodata
> > ffffffffc009a4d0 .rodata.str1.1
> > ffffffffc009ad98 .rodata.str1.8
> > ffffffffc009be14 .smp_locks
> > ffffffffc009be98 .parainstructions
> > ffffffffc009beb8 __param
> > ffffffffc009bf18 __mcount_loc
> > ffffffffc009d000 .data
> > ffffffffc00a5000 .init.data <<<
> > ffffffffc009db00 .gnu.linkonce.this_module
> > ffffffffc009dd40 .bss
> > ffffffffc00a6000 .symtab <<<
> > ffffffffc00a80b8 .strtab <<<
> > add-symbol-file /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug 0xffffffffc0092000 -s .note.gnu.build-id 0xffffffffc009a000 -s .init.text 0xffffffffc00a4000 -s .text.unlikely 0xffffffffc009971d -s .exit.text 0xffffffffc00998ea -s .rodata 0xffffffffc009a040 -s .rodata.str1.1 0xffffffffc009a4d0 -s .rodata.str1.8 0xffffffffc009ad98 -s .smp_locks 0xffffffffc009be14 -s .parainstructions 0xffffffffc009be98 -s __param 0xffffffffc009beb8 -s __mcount_loc 0xffffffffc009bf18 -s .data 0xffffffffc009d000 -s .init.data 0xffffffffc00a5000 -s .gnu.linkonce.this_module 0xffffffffc009db00 -s .bss 0xffffffffc009dd40 -s .symtab 0xffffffffc00a6000 -s .strtab 0xffffffffc00a80b8
> > add symbol table from file "/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug" at
> > .text_addr = 0xffffffffc0092000
> > .note.gnu.build-id_addr = 0xffffffffc009a000
> > .init.text_addr = 0xffffffffc00a4000
> > .text.unlikely_addr = 0xffffffffc009971d
> > .exit.text_addr = 0xffffffffc00998ea
> > .rodata_addr = 0xffffffffc009a040
> > .rodata.str1.1_addr = 0xffffffffc009a4d0
> > .rodata.str1.8_addr = 0xffffffffc009ad98
> > .smp_locks_addr = 0xffffffffc009be14
> > .parainstructions_addr = 0xffffffffc009be98
> > __param_addr = 0xffffffffc009beb8
> > __mcount_loc_addr = 0xffffffffc009bf18
> > .data_addr = 0xffffffffc009d000
> > .init.data_addr = 0xffffffffc00a5000
> > .gnu.linkonce.this_module_addr = 0xffffffffc009db00
> > .bss_addr = 0xffffffffc009dd40
> > .symtab_addr = 0xffffffffc00a6000
> > .strtab_addr = 0xffffffffc00a80b8
> > warning: section .symtab not found in /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> > warning: section .strtab not found in /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> > MODULE NAME BASE SIZE OBJECT FILE
> > ffffffffc009db00 floppy ffffffffc0092000 69417 /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> >
--
Crash-utility mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxxxxxx
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Contribution Guidelines: https://github.com/crash-utility/crash/wiki
