[I've trimmed the wide cc distribution that was inherited when I forked a different thread] On Mon, Jan 12, 2009 at 10:19 PM, Eric W. Biederman <ebiederm@xxxxxxxxxxxx> wrote: > "Mike Snitzer" <snitzer@xxxxxxxxx> writes: >> >> Now if only I could fix line numbers when debugging crashes in x86_64 >> modules with the crash utility! :) > > It's a userspace problem... > > All of the little usability things are userspace problems. > > I won't claim that it is trivial because it is a userspace problem, at the same > time there is no reason to wait for any kernel features to merge etc. Someone > just has to scratch an itch and go fix it. Yes, the crash utility (userspace) is clearly having problems getting line number for symbols in x86_64 modules. But I finally took some time to bisect the point in the kernel where the crash utility first started to fail, it appears to be: commit 7460ed2844ffad7141e30271c0c3da8336e66014 Author: john stultz <johnstul@xxxxxxxxxx> Date: Fri Feb 16 01:28:21 2007 -0800 [PATCH] time: x86_64: re-enable vsyscall support for x86_64 Cleanup and re-enable vsyscall gettimeofday using the generic clocksource infrastructure. [akpm@xxxxxxxx: cleanup] Signed-off-by: John Stultz <johnstul@xxxxxxxxxx> Cc: Ingo Molnar <mingo@xxxxxxx> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx> Cc: Andi Kleen <ak@xxxxxx> Cc: Roman Zippel <zippel@xxxxxxxxxxxxxx> Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> Signed-off-by: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx> arch/x86_64/Kconfig | 4 + arch/x86_64/kernel/hpet.c | 6 ++ arch/x86_64/kernel/time.c | 6 -- arch/x86_64/kernel/tsc.c | 7 ++ arch/x86_64/kernel/vmlinux.lds.S | 28 ++++------ arch/x86_64/kernel/vsyscall.c | 119 ++++++++++++++++++++++--------------- include/asm-x86_64/proto.h | 2 - include/asm-x86_64/timex.h | 1 - include/asm-x86_64/vsyscall.h | 29 ++-------- 9 files changed, 104 insertions(+), 98 deletions(-) Here is the full bisect log: git-bisect start # good: [62d0cfcb27cf755cebdc93ca95dabc83608007cd] Linux 2.6.20 git-bisect good 62d0cfcb27cf755cebdc93ca95dabc83608007cd # bad: [c8f71b01a50597e298dc3214a2f2be7b8d31170c] Linux 2.6.21-rc1 git-bisect bad c8f71b01a50597e298dc3214a2f2be7b8d31170c # good: [574009c1a895aeeb85eaab29c235d75852b09eb8] Merge branch 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus git-bisect good 574009c1a895aeeb85eaab29c235d75852b09eb8 # good: [087d7ecd5273b480d13f4309a159842700afe276] [POWERPC] mpic: set IPIs to be per-CPU git-bisect good 087d7ecd5273b480d13f4309a159842700afe276 # bad: [920841d8d1d61bc12b43f95a579a5374f6d98f81] Merge branch 'for-linus' of ssh://master.kernel.org/pub/scm/linux/kernel/git/ieee1394/linux1394-2.6 git-bisect bad 920841d8d1d61bc12b43f95a579a5374f6d98f81 # bad: [892705a1e1b4d0f9f6c5ac57f777b8055525bf68] USB: kernel-doc fixes git-bisect bad 892705a1e1b4d0f9f6c5ac57f777b8055525bf68 # good: [741673473a5b26497d5390f38d478362e27e22ad] i386 prepare for dyntick git-bisect good 741673473a5b26497d5390f38d478362e27e22ad # bad: [ef29498655b18d2bfd69048e20835d19333981ab] Merge master.kernel.org:/pub/scm/linux/kernel/git/davej/cpufreq git-bisect bad ef29498655b18d2bfd69048e20835d19333981ab # bad: [bec50c47aaf6f1f9247f1860547ab394a0802a4c] knfsd: nfsd4: acls: avoid unnecessary denies git-bisect bad bec50c47aaf6f1f9247f1860547ab394a0802a4c # bad: [7460ed2844ffad7141e30271c0c3da8336e66014] time: x86_64: re-enable vsyscall support for x86_64 git-bisect bad 7460ed2844ffad7141e30271c0c3da8336e66014 # good: [289f480af87e45f7a6de6ba9b4c061c2e259fe98] Add debugging feature /proc/timer_list git-bisect good 289f480af87e45f7a6de6ba9b4c061c2e259fe98 # good: [2d0c87c3bc49c60ab5bbac401fb1ef37ff10bbe2] time: x86_64: hpet_address cleanup git-bisect good 2d0c87c3bc49c60ab5bbac401fb1ef37ff10bbe2 # good: [1489939f0ab64b96998e04068c516c39afe29654] time: x86_64: convert x86_64 to use GENERIC_TIME git-bisect good 1489939f0ab64b96998e04068c516c39afe29654 I used version 4.0-7.6 of the crash utility to test if each commit was good or bad. I simply checked if ext3's module had correct line number info for the ext3_get_blocks_handle symbol with: sym ext3_get_blocks_handle I tried to revert 7460ed2844ffad7141e30271c0c3da8336e66014 from v2.6.21 but it had conflicts that I've not yet been able to put adequate time to resolving. That aside, I'd be very interested to know how/where this commit is impacting the crash utility. Has alignment of some module metadata structure been altered and that is the problem? This isn't my area of expertise but I have to believe others may have useful insight. Thanks, Mike -- Crash-utility mailing list Crash-utility@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/crash-utility