Hi Kazu,
On Thu, Nov 9, 2023 at 4:37 PM HAGIO KAZUHITO(萩尾 一仁) <k-hagio-ab@xxxxxxx> wrote:
>
> On 2023/11/08 17:16, HAGIO KAZUHITO(萩尾 一仁) wrote:
> > On 2023/11/08 12:01, HAGIO KAZUHITO(萩尾 一仁) wrote:
> >> Hi Tao,
> >>
> >> thank you for the information.
> >>
> >> I'm looking into it, I noticed that the unexpected symbol "floopy_module_init"
> >> is in section .init.text. Crash side doesn't have the symbol info, probably
> >> the address is already freed and reused by the virtio_blk module?
> >>
> >> crash> mod -S
> >> ...
> >> crash> sym -m floppy | grep MODULE
> >> ffffffffc0092000 MODULE START: floppy
> >> ffffffffc00a2f29 MODULE END: floppy
> >> crash> sym -m virtio_blk | grep MODULE
> >> ffffffffc00a4000 MODULE START: virtio_blk
> >> ffffffffc00a86ec MODULE END: virtio_blk
> >> crash> gdb info symbol floppy_module_init
> >> init_module in section .init.text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> >> virtblk_freeze + 33 in section .text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/virtio_blk.ko.debug
> >> crash> gdb info address floppy_module_init
> >> Symbol "floppy_module_init" is a function at address 0xffffffffc00a4131.
> >> crash> sym floppy_module_init
> >> symbol not found: floppy_module_init
> >> possible alternatives:
> >> (none found)
> >>
> >> So still not sure, but this might be an issue of handling the init
> >> section/symbols of modules..
> >
> > Hmm, it looks like several sections are mapped outside of the current
> > module range. What if they are not specified or they are mapped to
> > e.g. an unused range..? or any idea?
>
> First, "mod -S -r" can reproduce this without the patch 1/2 or the commands.
> (If the root cause is fixed, I think no need to apply the patch 1/2.)
>
> crash-ups> mod -S -r
> crash-ups> p virtio_mq_ops
> virtio_mq_ops = $1 = {
> queue_rq = 0xffffffffc00a45b0 <floppy_module_init+1151>,
> map_queue = 0xffffffff813015c0 <blk_mq_map_queue>,
>
>
> The following trial patch, which does not map the .init.text section
> specifically, fixes the issue.
>
> --- a/symbols.c
> +++ b/symbols.c
> @@ -13283,7 +13283,7 @@ add_symbol_file_kallsyms(struct load_module *lm, struct gnu_request *req)
> shift_string_right(req->buf, strlen(buf));
> BCOPY(buf, req->buf, strlen(buf));
> retval = TRUE;
> - } else {
> + } else if (!STREQ(section_name, ".init.text")) {
> sprintf(buf, " -s %s 0x%lx", section_name, section_vaddr);
> while ((len + strlen(buf)) >= buflen) {
> RESIZEBUF(req->buf, buflen, buflen * 2);
>
> crash-dev> mod -S -r
> crash-dev> p virtio_mq_ops
> virtio_mq_ops = $1 = {
> queue_rq = 0xffffffffc00a45b0 <virtio_queue_rq>,
> map_queue = 0xffffffff813015c0 <blk_mq_map_queue>,
> {
>
>
> In this way, it looks like the .init.text sections of modules are
> mapped to 0x0, but many other sections like .modinfo are mapped here
> even without the trial patch.
>
> crash-dev> gdb info address floppy_module_init
> Symbol "floppy_module_init" is a function at address 0x131.
>
> crash-dev> gdb info symbol floppy_module_init
> irq_stack_union + 305 in section .data..percpu of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/vmlinux
> local_init + 305 in section .init.text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-mod.ko.debug
> ____versions + 49 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-region-hash.ko.debug
> ____versions + 145 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio.ko.debug
> __UNIQUE_ID_vermagic5 + 30 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-log.ko.debug
> __UNIQUE_ID_srcversion7 + 1 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/md/dm-mirror.ko.debug
> __UNIQUE_ID_alias7 + 30 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio_pci.ko.debug
> ____versions + 145 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/virtio/virtio_ring.ko.debug
> __UNIQUE_ID_alias16 + 36 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/ata/ata_generic.ko.debug
> ____versions + 17 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/fs/mbcache.ko.debug
> __UNIQUE_ID_vermagic5 + 45 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/net/virtio_net.ko.debug
> __UNIQUE_ID_vermagic5 + 44 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/cdrom/cdrom.ko.debug
> __UNIQUE_ID_vermagic5 + 14 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/scsi/sr_mod.ko.debug
> __module_depends + 1 in section .modinfo of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.debug
> ____versions + 81 in section __versions of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/fs/binfmt_misc.ko.debug
> init_module in section .init.text of /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> ...
>
> If a module already does not have its init memory range, it might be
> a bit better to not specify "-s .init.text <addr>" to add-symbol-file..
>
Thanks a lot for finding the root cause. My patch is just a
work-around, and I think your "trial" patch is better to be applied. I
agree the ".init.text" should not be added by add-symbol-file, since
this section will be freed and will be occupied by other modules after
kernel init, which will cause symbols overlap. Could you please draft
the "trial" patch to be the formal one?
Thanks,
Tao Liu
> Thanks,
> Kazu
>
>
> >
> > crash-dev> sym -m floppy | grep MODULE
> > ffffffffc0092000 MODULE START: floppy
> > ffffffffc00a2f29 MODULE END: floppy
> > crash-dev> set debug 1
> > debug: 1
> > crash-dev> mod -s floppy
> > ffffffffc009a000 .note.gnu.build-id
> > ffffffffc0092000 .text
> > ffffffffc00a4000 .init.text <<< higher than MODULE END
> > ffffffffc009971d .text.unlikely
> > ffffffffc00998ea .exit.text
> > ffffffffc009a040 .rodata
> > ffffffffc009a4d0 .rodata.str1.1
> > ffffffffc009ad98 .rodata.str1.8
> > ffffffffc009be14 .smp_locks
> > ffffffffc009be98 .parainstructions
> > ffffffffc009beb8 __param
> > ffffffffc009bf18 __mcount_loc
> > ffffffffc009d000 .data
> > ffffffffc00a5000 .init.data <<<
> > ffffffffc009db00 .gnu.linkonce.this_module
> > ffffffffc009dd40 .bss
> > ffffffffc00a6000 .symtab <<<
> > ffffffffc00a80b8 .strtab <<<
> > add-symbol-file /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug 0xffffffffc0092000 -s .note.gnu.build-id 0xffffffffc009a000 -s .init.text 0xffffffffc00a4000 -s .text.unlikely 0xffffffffc009971d -s .exit.text 0xffffffffc00998ea -s .rodata 0xffffffffc009a040 -s .rodata.str1.1 0xffffffffc009a4d0 -s .rodata.str1.8 0xffffffffc009ad98 -s .smp_locks 0xffffffffc009be14 -s .parainstructions 0xffffffffc009be98 -s __param 0xffffffffc009beb8 -s __mcount_loc 0xffffffffc009bf18 -s .data 0xffffffffc009d000 -s .init.data 0xffffffffc00a5000 -s .gnu.linkonce.this_module 0xffffffffc009db00 -s .bss 0xffffffffc009dd40 -s .symtab 0xffffffffc00a6000 -s .strtab 0xffffffffc00a80b8
> > add symbol table from file "/home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug" at
> > .text_addr = 0xffffffffc0092000
> > .note.gnu.build-id_addr = 0xffffffffc009a000
> > .init.text_addr = 0xffffffffc00a4000
> > .text.unlikely_addr = 0xffffffffc009971d
> > .exit.text_addr = 0xffffffffc00998ea
> > .rodata_addr = 0xffffffffc009a040
> > .rodata.str1.1_addr = 0xffffffffc009a4d0
> > .rodata.str1.8_addr = 0xffffffffc009ad98
> > .smp_locks_addr = 0xffffffffc009be14
> > .parainstructions_addr = 0xffffffffc009be98
> > __param_addr = 0xffffffffc009beb8
> > __mcount_loc_addr = 0xffffffffc009bf18
> > .data_addr = 0xffffffffc009d000
> > .init.data_addr = 0xffffffffc00a5000
> > .gnu.linkonce.this_module_addr = 0xffffffffc009db00
> > .bss_addr = 0xffffffffc009dd40
> > .symtab_addr = 0xffffffffc00a6000
> > .strtab_addr = 0xffffffffc00a80b8
> > warning: section .symtab not found in /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> > warning: section .strtab not found in /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> > MODULE NAME BASE SIZE OBJECT FILE
> > ffffffffc009db00 floppy ffffffffc0092000 69417 /home/vmcore/symbol_err/usr/lib/debug/lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/drivers/block/floppy.ko.debug
> >
--
Crash-utility mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxxxxxx
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Contribution Guidelines: https://github.com/crash-utility/crash/wiki
