Hi Dave,
Thank you very much,
I use crash in RHEL5.4.
crash version: 4.0-8.9.1.el5 gdb version: 6.1
The vmcore file is very huge (12GB). I will check it carefully.
Yuming
--- On Mon, 11/29/10, Dave Anderson <anderson@xxxxxxxxxx> wrote:
> From: Dave Anderson <anderson@xxxxxxxxxx>
> Subject: Re: crash "bt" and "dmesg" show different messages
> To: "Discussion list for crash utility usage, maintenance and development" <crash-utility@xxxxxxxxxx>
> Date: Monday, November 29, 2010, 10:03 PM
>
> ----- "Yuming Cheng" <chengyuming_ah@xxxxxxxxxxxx>
> wrote:
>
> > Hi all,
> >
> > When using kdump, I find crash "bt" and "dmesg" give
> different info.
> > which one is more reliable ?
> >
> > Thanks,
> > ---cym
>
> In this case, the dmesg output is more helpful because it
> contains the
> exception frame. It's pretty clear that the
> neigh_cleanup_and_release()
> has called a destructor function, but the address stored in
>
> neigh->parms->neigh_destructor (as stored in RAX)
> contains a bogus
> address of 0000000000000001:
>
> static void neigh_cleanup_and_release(struct neighbour
> *neigh)
> {
> if
> (neigh->parms->neigh_destructor)
>
> neigh->parms->neigh_destructor(neigh);
>
> __neigh_notify(neigh,
> RTM_DELNEIGH, 0);
> neigh_release(neigh);
> }
>
> crash> dis -r neigh_cleanup_and_release+0x13
> 0xffffffff8022108d <neigh_cleanup_and_release>:
> push %rbx
> 0xffffffff8022108e
> <neigh_cleanup_and_release+0x1>:
> mov 0x10(%rdi),%rax
> 0xffffffff80221092
> <neigh_cleanup_and_release+0x5>:
> mov %rdi,%rbx
> 0xffffffff80221095
> <neigh_cleanup_and_release+0x8>:
> mov 0x18(%rax),%rax
> 0xffffffff80221099
> <neigh_cleanup_and_release+0xc>:
> test %rax,%rax
> 0xffffffff8022109c
> <neigh_cleanup_and_release+0xf>:
> je
> 0xffffffff802210a0
> <neigh_cleanup_and_release+0x13>
> 0xffffffff8022109e
> <neigh_cleanup_and_release+0x11>:
> callq *%rax
> 0xffffffff802210a0
> <neigh_cleanup_and_release+0x13>: lock
> decl 0x70(%rbx)
>
>
> If you do a "bt -e" I would guess that the exception frame
> would be
> found and displayed, but it *should* have been displayed
> in-line by
> the "bt" command.
>
> I can't tell you why it was not displayed by "bt" unless I
> have the
> dumpfile. You also didn't mention what version of
> crash you were
> running -- there have been a few fixes for "missing"
> exception frames.
> If you want to make the dumpfile available to me, I can
> take a look
> at it.
>
> Dave
>
>
> >
> > dmesg
> > /****************************************/
> > Unable to handle kernel NULL pointer dereference at
> 0000000000000001
> > RIP: [<0000000000000001>]
> > PGD 323c6f067 PUD 323f13067 PMD 0
> > Oops: 0010 [1] SMP
> > last sysfs file: /devices/pci0000:00/0000:00:00.0/irq
> > CPU 6
> > Modules linked in: igb(U) bonding ipv6 xfrm_nalgo
> crypto_api autofs4
> > hidp rfcomm l2cap bluetooth lockd sunrpc dm_mirror
> dm_multipath
> > scsi_dh video hwmon backlight sbs i2c_ec button
> battery asus_acpi
> > acpi_memhotplug ac parport_pc lp parport sg ixgbe
> pcspkr i2c_i801
> > serio_raw i2c_core 8021q dca dm_raid45 dm_message
> dm_region_hash
> > dm_log dm_mod dm_mem_cache ahci libata shpchp mptsas
> mptscsih mptbase
> > scsi_transport_sas sd_mod scsi_mod ext3 jbd uhci_hcd
> ohci_hcd
> > ehci_hcd
> > Pid: 8894, comm: ifconfig Tainted: G
> 2.6.18-164.el5debug #1
> > RIP: 0010:[<0000000000000001>]
> [<0000000000000001>]
> > RSP: 0018:ffff810323dd9cf0 EFLAGS: 00010202
> > RAX: 0000000000000001 RBX: ffff81032a8e5b68 RCX:
> 0000000000000000
> > RDX: 0000000000000006 RSI: 0000000000000001 RDI:
> ffff81032a8e5b68
> > RBP: ffff81033aabc850 R08: 0000000000000002 R09:
> 0000000000000001
> > R10: ffff81032a8e5c30 R11: ffffffff80049ee3 R12:
> ffff81032a8e5ba8
> > R13: 0000000000000006 R14: ffff8103238be000 R15:
> ffffffff8846ad00
> > FS: 00002ba7032083f0(0000)
> GS:ffff810113a9e4c8(0000)
> > knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0:
> 000000008005003b
> > CR2: 0000000000000001 CR3: 00000003242b4000 CR4:
> 00000000000006e0
> > Process ifconfig (pid: 8894, threadinfo
> ffff810323dd8000, task
> > ffff810323f461c0)
> > Stack: ffffffff8023de3b ffff81032a8e5b68
> ffffffff8023e0d6
> > ffffffff8023e122
> > ffffffff88468eb0 ffff8103238be000
> ffffffff8846ad00 ffff8103238be000
> > 0000000000000000 ffffffff8846ae98
> ffffffff8023e12d 0000000000000002
> > Call Trace:
> > [<ffffffff8023de3b>]
> neigh_cleanup_and_release+0x13/0x2c
> > [<ffffffff8023e0d6>]
> neigh_flush_dev+0x9d/0xc3
> > [<ffffffff88439acb>]
> :ipv6:ndisc_netdev_event+0x30/0x3d
> > [<ffffffff8006ae76>]
> notifier_call_chain+0x20/0x32
> > [<ffffffff80238c52>] dev_close+0x6e/0x72
> > [<ffffffff80237d24>]
> dev_change_flags+0x5a/0x119
> > [<ffffffff8026cb77>]
> devinet_ioctl+0x235/0x59c
> > [<ffffffff8022f0e3>]
> sock_ioctl+0x1c7/0x1eb
> > [<ffffffff8004465d>] do_ioctl+0x21/0x6b
> > [<ffffffff80031f07>]
> vfs_ioctl+0x45d/0x4bf
> > [<ffffffff800c0b9d>]
> audit_syscall_entry+0x180/0x1b3
> > [<ffffffff8004ef9e>] sys_ioctl+0x59/0x78
> > [<ffffffff800602a6>] tracesys+0xd5/0xdf
> >
> > /****************************************/
> > crash btcrash> bt
> > PID: 8894 TASK:
> ffff810323f461c0 CPU: 6 COMMAND:
> "ifconfig"
> > #0 [ffff810323dd9a50] crash_kexec at
> ffffffff800b6eae
> > #1 [ffff810323dd9b10] __die at ffffffff80069087
> > #2 [ffff810323dd9b50] do_page_fault at
> ffffffff8006ad73
> > #3 [ffff810323dd9c40] error_exit at
> ffffffff80060e9d
> > #4 [ffff810323dd9c78] skb_dequeue at
> ffffffff80049ee3
> > #5 [ffff810323dd9cf0] neigh_cleanup_and_release
> at ffffffff8023de3b
> > #6 [ffff810323dd9d00] neigh_flush_dev at
> ffffffff8023e0d6
> > #7 [ffff810323dd9d40] neigh_ifdown at
> ffffffff8023e12d
> > #8 [ffff810323dd9d80] ndisc_netdev_event at
> ffffffff88439acb
> > #9 [ffff810323dd9d90] notifier_call_chain at
> ffffffff8006ae76
> > #10 [ffff810323dd9db0] dev_close at ffffffff80238c52
> > #11 [ffff810323dd9dc0] dev_change_flags at
> ffffffff80237d24
> > #12 [ffff810323dd9df0] devinet_ioctl at
> ffffffff8026cb77
> > #13 [ffff810323dd9e90] sock_ioctl at ffffffff8022f0e3
> > #14 [ffff810323dd9eb0] do_ioctl at ffffffff8004465d
> > #15 [ffff810323dd9ed0] vfs_ioctl at ffffffff80031f07
> > #16 [ffff810323dd9f40] sys_ioctl at ffffffff8004ef9e
> > #17 [ffff810323dd9f80] tracesys at ffffffff800602a6
> (via system_call)
> > RIP: 0000003749ccc557
> RSP: 00007fff470ea238 RFLAGS: 00000206
> > RAX: ffffffffffffffda
> RBX: ffffffff800602a6 RCX:
> > ffffffffffffffff
> > RDX: 00007fff470ea240
> RSI: 0000000000008914 RDI:
> > 0000000000000004
> > RBP:
> 0000000000000000 R8:
> 00007fff470ea244 R9:
> > 0000000000000002
> > R10: 0000000000000001
> R11: 0000000000000206 R12:
> > 00007fff470ea360
> > R13: 00000000fffffffe
> R14: 00007fff470ea530 R15:
> > 0000000000000004
> > ORIG_RAX:
> 0000000000000010 CS: 0033 SS: 002b
> >
> >
> >
> >
> >
> >
> >
> > --
> > Crash-utility mailing list
> > Crash-utility@xxxxxxxxxx
> > https://www.redhat.com/mailman/listinfo/crash-utility
>
> --
> Crash-utility mailing list
> Crash-utility@xxxxxxxxxx
> https://www.redhat.com/mailman/listinfo/crash-utility
>
--
Crash-utility mailing list
Crash-utility@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/crash-utility
