Re: [PATCH v5 20/20] scripts/gdb: Add basic documentation

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 01/29/2013 06:38:03 AM, Jan Kiszka wrote:
CC: Rob Landley <rob@xxxxxxxxxxx>
CC: linux-doc@xxxxxxxxxxxxxxx
Signed-off-by: Jan Kiszka <jan.kiszka@xxxxxxxxxxx>
---
Documentation/gdb-kernel-debugging.txt | 155 ++++++++++++++++++++++++++++++++
 1 files changed, 155 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/gdb-kernel-debugging.txt

diff --git a/Documentation/gdb-kernel-debugging.txt b/Documentation/gdb-kernel-debugging.txt
new file mode 100644
index 0000000..0ea46e1
--- /dev/null
+++ b/Documentation/gdb-kernel-debugging.txt
@@ -0,0 +1,155 @@
+Debugging kernel and modules via gdb
+====================================
+
+The kernel debugger kgdb, hypervisors like QEMU or JTAG-based hardware +interfaces allow to debug the Linux kernel and its modules during runtime
+using gdb.

This could use some clarification.

Technically they're attaching gdb to the hardware using the gdbserver protocol, instead of attaching to a specific process like the gdbserver client program does. (I think of it as "pid 0": your register state isn't a "register profile" but the actual contents of the hardware registers, and your memory map is physical.)

Two of these three methods (jtag and emulator) are independent of linux, and the fact the hardware you're attached to may or may not be running linux (or u-boot, or a bare metal hello world) is irrelevant. Only kgdb requires linux to be running to produce/consume gdbserver procol with information about current hardware state.

When Linux _is_ running, if you want symbol data you can point gdb at an unstripped vmlinux. (Which is an ELF image, the build usually feeds that through objdump and glues a wrapper on the front to make it bootable on bare metal, but this is the one with symbols still in it.) But even with symbols, trying to dig through Linux's various abstractions and data structures to find anything belonging to an actual process from outside the kernel is a flaming pain, so you've made some gdb plugins to help out there. Cool.

Gdb comes with a powerful scripting interface for python. The
+kernel provides a collection of helper scripts that can simplify typical +kernel debugging steps. This is a short tutorial about how to enable and use +them. It focuses on QEMU/KVM virtual machines as target, but the examples can
+be transferred to the other gdb stubs as well.
+
+
+Requirements
+------------
+
+ o gdb 7.1+ (recommended: 7.3+) with python support enabled (typically true
+   for distributions)
+
+
+Setup
+-----
+
+ o Create a virtual Linux machine for QEMU/KVM (see www.linux-kvm.org and
+   www.qemu.org for more details)

If it helps, I've got a dozen different targets as "system-image-*.tar.bz2" at http://landley.net/aboriginal/bin and the "run-emulator.sh" script inside each one gives the qemu command line to boot it to a shell prompt. At that shell prompt you can grab the relevant kernel config with "zcat /proc/config.gz" to build your own kernel, and the toolchains I did so with are in the same directory. (The README in there describes other categories of file in the directory.)

That at least gives you a "known working" version to compare your own attempts against.

+ o Build the kernel with CONFIG_DEBUG_INFO enabled, but leave
+   CONFIG_DEBUG_INFO_REDUCED off
+
+ o Install that kernel on the guest.
+
+ Alternatively, QEMU allows to boot the kernel directly using -kernel, + -append, -initrd command line switches. This is generally only useful if + you do not depend on modules. See QEMU documentation for more details on
+   this mode.

It's qemu's built-in bootloader, no different than any other bootloader. If your hardware needs a module to access the block device you want to keep the rest of your modules on, A) you're making a peverse architectural decision, B) stick it in the initramfs.

(I could do up an example of that sort of setup if you like. The images I linked to above keep a squashfs on a virtual hard drive, I could do an initramfs that loads the IDE module and then switch_root to it. I just haven't bothered because the easy way works.)

+ o Enable the gdb stub of QEMU/KVM, either
+    - at VM startup time by appending "-s" to the QEMU command line
+   or
+    - during runtime by issuing "gdbserver" from the QEMU monitor
+      console

Initiallly I thought this document would be more about kgdb, but this is about how to use a direct hardware attached debugger (or kgdb emulating one). Hence the clarification up top.

+ o cd /path/to/linux-build
+
+ o Start gdb: gdb vmlinux
+
+ Note: Some distros may restrict auto-loading of gdb scripts to known safe + directories. In case gdb reports to refuse loading vmlinux-gdb.py, add
+
+    add-add-auto-load-safe-path /path/to/linux-build
+
+   to ~/.gdbinit. See gdb help for more details.
+
+ o Attach to the booted guest:
+    (gdb) target remote :1234

I've found the "target remote |" option useful, where you pipe the I/O to another command that forwards data to the target board.

(I once had to debug a program on a board that only had a serial console, and that was always on and attached to another machine. So I piped to ssh which ran a term program which launched netcat on the board to connect the remote end because the FSF didn't bother to make a stdio option for gdbserver so you _have_ to use loopback. Although that was a few versions back, maybe they fixed this...)

*shrug* Possibly off topic...

The rest is about what the plugins do, looks fine to me. :)

Rob

+
+Examples of using the Linux-provided gdb helpers
+------------------------------------------------
+
+ o Load module (and main kernel) symbols:
+    (gdb) lx-symbols
+    loading vmlinux
+    scanning for modules in /home/user/linux/build
+ loading @0xffffffffa0020000: /home/user/linux/build/net/netfilter/xt_tcpudp.ko + loading @0xffffffffa0016000: /home/user/linux/build/net/netfilter/xt_pkttype.ko + loading @0xffffffffa0002000: /home/user/linux/build/net/netfilter/xt_limit.ko + loading @0xffffffffa00ca000: /home/user/linux/build/net/packet/af_packet.ko + loading @0xffffffffa003c000: /home/user/linux/build/fs/fuse/fuse.ko
+    ...
+ loading @0xffffffffa0000000: /home/user/linux/build/drivers/ata/ata_generic.ko
+
+ o Set a breakpoint on some not yet loaded module function, e.g.:
+    (gdb) b btrfs_init_sysfs
+    Function "btrfs_init_sysfs" not defined.
+ Make breakpoint pending on future shared library load? (y or [n]) y
+    Breakpoint 1 (btrfs_init_sysfs) pending.
+
+ o Continue the target
+    (gdb) c
+
+ o Load the module on the target and watch the symbols being loaded as well as
+   the breakpoint hit:
+ loading @0xffffffffa0034000: /home/user/linux/build/lib/libcrc32c.ko + loading @0xffffffffa0050000: /home/user/linux/build/lib/lzo/lzo_compress.ko + loading @0xffffffffa006e000: /home/user/linux/build/lib/zlib_deflate/zlib_deflate.ko + loading @0xffffffffa01b1000: /home/user/linux/build/fs/btrfs/btrfs.ko
+
+ Breakpoint 1, btrfs_init_sysfs () at /home/user/linux/fs/btrfs/sysfs.c:36 + 36 btrfs_kset = kset_create_and_add("btrfs", NULL, fs_kobj);
+
+ o Dump the log buffer of the target kernel:
+    (gdb) lx-dmesg
+    [     0.000000] Initializing cgroup subsys cpuset
+    [     0.000000] Initializing cgroup subsys cpu
+    [     0.000000] Linux version 3.8.0-rc4-dbg+ (...
+ [ 0.000000] Command line: root=/dev/sda2 resume=/dev/sda1 vga=0x314
+    [     0.000000] e820: BIOS-provided physical RAM map:
+ [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable + [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
+    ....
+
+ o Examine fields of the current task struct:
+    (gdb) p $lx_current().pid
+    $1 = 4998
+    (gdb) p $lx_current().comm
+    $2 = "modprobe\000\000\000\000\000\000\000"
+
+ o Make use of the per-cpu helper for the current or a specified CPU:
+    (gdb) p $lx_per_cpu("runqueues").nr_running
+    $3 = 1
+    (gdb) p $lx_per_cpu("runqueues", 2).nr_running
+    $4 = 0
+
+ o Dig into hrtimers using the container_of helper:
+ (gdb) set $next = $lx_per_cpu("hrtimer_bases").clock_base[0].active.next
+    (gdb) p *$container_of($next, "struct hrtimer", "node")
+    $5 = {
+      node = {
+        node = {
+          __rb_parent_color = 18446612133355256072,
+          rb_right = 0x0 <irq_stack_union>,
+          rb_left = 0x0 <irq_stack_union>
+        },
+        expires = {
+          tv64 = 1835268000000
+        }
+      },
+      _softexpires = {
+        tv64 = 1835268000000
+      },
+      function = 0xffffffff81078232 <tick_sched_timer>,
+      base = 0xffff88003fd0d6f0,
+      state = 1,
+      start_pid = 0,
+      start_site = 0xffffffff81055c1f <hrtimer_start_range_ns+20>,
+      start_comm = "swapper/2\000\000\000\000\000\000"
+    }
+
+
+List of commands and helper
+---------------------------
+
+The number of commands and convenience helpers may evolve over the time, this
+is just a snapshot of the initial version:
+
+ (gdb) apropos lx
+ function lx_current -- Return current task
+ function lx_module -- Find module by name and return the module variable
+ function lx_modvar -- Return global variable of a module
+ function lx_per_cpu -- Return per-cpu variable
+ function lx_task_by_pid -- Find Linux task by PID and return the task_struct variable + function lx_thread_info -- Calculate Linux thread_info from task variable
+ lx-dmesg -- Print Linux kernel log buffer
+ lx-lsmod -- List currently loaded modules
+ lx-symbols -- (Re-)load symbols of Linux kernel and currently loaded modules
--
1.7.3.4





--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]     [Linux Resources]

  Powered by Linux