Em Tue, 25 Oct 2016 17:20:01 -0600 Jonathan Corbet <corbet@xxxxxxx> escreveu: > On Mon, 24 Oct 2016 09:00:18 -0200 > Mauro Carvalho Chehab <mchehab@xxxxxxxxxxxxxxxx> wrote: > > > - Add a document title and remove its own index; > > - use monotonic fonts for paths; > > - use quote blocks where needed; > > - adjust/use spaces to properly format paragraphs; > > - add it to the user book. > > I'm going to start sounding monotonous (if I haven't already), but... > > > Last updated: 20 December 2005 > > That's a bad sign, but it's not quite entirely true - it got some tweaks > in 2008. > > The core of this file is a description of how to manually bisect a bug in > different versions of the 1.3 kernel. Are we really doing our users any > favors by providing this information? *Nobody* is going to do this now. On a very quick look on the document, it seems that the legacy info there are at this section of the document: - Finding it the old way IMHO, we can strip that section completely. The git bisect section is too small to my taste, as, IMHO, the best would be if it would be telling about the process, e. g. pointing or C/C the contents of articles like: - Fighting regressions with git bisect: https://www.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.html - Fully automated bisecting with "git bisect run" - https://lwn.net/Articles/317154/ - Using Git bisect to figure out when brokenness was introduced - http://webchick.net/node/99 I would actually prefer to add a quick summary there, and point to the articles for further reference, as hiperlinks tend to break as times goes by. What do you think about the patch below? I double-checked the "Fixing the bug" part, and what is there makes sense for me. I changed it a little bit to make it easier for newbies. Thanks, Mauro [PATCH] bug-hunting.rst: update info about bug hunting The document shows a really old procedure for bug hunting that nobody uses anymore. Remove such section, and update the remaining documentation to reflect the procedures used currently. Signed-off-by: Mauro Carvalho Chehab <mchehab@xxxxxxxxxxxxxxxx> diff --git a/Documentation/admin-guide/bug-hunting.rst b/Documentation/admin-guide/bug-hunting.rst index d35dd9fd1af0..987468fb4b83 100644 --- a/Documentation/admin-guide/bug-hunting.rst +++ b/Documentation/admin-guide/bug-hunting.rst @@ -1,7 +1,7 @@ Bug hunting +++++++++++ -Last updated: 20 December 2005 +Last updated: 28 October 2016 Introduction ============ @@ -20,120 +20,61 @@ Before you submit a bug report read Devices not appearing ===================== -Often this is caused by udev. Check that first before blaming it on the -kernel. +Often this is caused by udev/systemd. Check that first before blaming it +on the kernel. Finding patch that caused a bug =============================== - - -Finding using ``git-bisect`` ----------------------------- - Using the provided tools with ``git`` makes finding bugs easy provided the bug is reproducible. Steps to do it: -- start using git for the kernel source -- read the man page for ``git-bisect`` -- have fun - -Finding it the old way ----------------------- - -[Sat Mar 2 10:32:33 PST 1996 KERNEL_BUG-HOWTO lm@xxxxxxx (Larry McVoy)] - -This is how to track down a bug if you know nothing about kernel hacking. -It's a brute force approach but it works pretty well. - -You need: - - - A reproducible bug - it has to happen predictably (sorry) - - All the kernel tar files from a revision that worked to the - revision that doesn't - -You will then do: +- build the Kernel from its git source +- start bisect with [#f1]_:: - - Rebuild a revision that you believe works, install, and verify that. - - Do a binary search over the kernels to figure out which one - introduced the bug. I.e., suppose 1.3.28 didn't have the bug, but - you know that 1.3.69 does. Pick a kernel in the middle and build - that, like 1.3.50. Build & test; if it works, pick the mid point - between .50 and .69, else the mid point between .28 and .50. - - You'll narrow it down to the kernel that introduced the bug. You - can probably do better than this but it gets tricky. + $ git bisect start - - Narrow it down to a subdirectory +- mark the broken changeset with:: - - Copy kernel that works into "test". Let's say that 3.62 works, - but 3.63 doesn't. So you diff -r those two kernels and come - up with a list of directories that changed. For each of those - directories: + $ git bisect bad [commit] - Copy the non-working directory next to the working directory - as "dir.63". - One directory at time, try moving the working directory to - "dir.62" and mv dir.63 dir"time, try:: +- mark a changeset where the code is known to work with:: - mv dir dir.62 - mv dir.63 dir - find dir -name '*.[oa]' -print | xargs rm -f + $ git bisect good [commit] - And then rebuild and retest. Assuming that all related - changes were contained in the sub directory, this should - isolate the change to a directory. +- rebuild the Kernel and test +- interact with git bisect by using either:: - Problems: changes in header files may have occurred; I've - found in my case that they were self explanatory - you may - or may not want to give up when that happens. + $ git bisect good - - Narrow it down to a file + or:: - - You can apply the same technique to each file in the directory, - hoping that the changes in that file are self contained. + $ git bisect bad - - Narrow it down to a routine + depending if the bug happened on the changeset you're testing +- After some interactions, git bisect will give you the changeset that + likely caused the bug. - - You can take the old file and the new file and manually create - a merged file that has:: +- For example, if you know that the current version is bad, and version + 4.8 is good, you could do:: - #ifdef VER62 - routine() - { - ... - } - #else - routine() - { - ... - } - #endif + $ git bisect start + $ git bisect bad # Current version is bad + $ git bisect good v4.8 - And then walk through that file, one routine at a time and - prefix it with:: - #define VER62 - /* both routines here */ - #undef VER62 +.. [#f1] You can, optionally, provide both good and bad arguments at git + start:: + git bisect start [BAD] [GOOD] - Then recompile, retest, move the ifdefs until you find the one - that makes the difference. +For further references, please read: -Finally, you take all the info that you have, kernel revisions, bug -description, the extent to which you have narrowed it down, and pass -that off to whomever you believe is the maintainer of that section. -A post to linux.dev.kernel isn't such a bad idea if you've done some -work to narrow it down. - -If you get it down to a routine, you'll probably get a fix in 24 hours. - -My apologies to Linus and the other kernel hackers for describing this -brute force approach, it's hardly what a kernel hacker would do. However, -it does work and it lets non-hackers help fix bugs. And it is cool -because Linux snapshots will let you do this - something that you can't -do with vendor supplied releases. +- The man page for ``git-bisect`` +- `Fighting regressions with git bisect <https://www.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.html>`_ +- `Fully automated bisecting with "git bisect run" <https://lwn.net/Articles/317154/>'_ +- `Using Git bisect to figure out when brokenness was introduced <http://webchick.net/node/99>`_ Fixing the bug ============== @@ -141,13 +82,16 @@ Fixing the bug Nobody is going to tell you how to fix bugs. Seriously. You need to work it out. But below are some hints on how to use the tools. +objdump +------- + To debug a kernel, use objdump and look for the hex offset from the crash output to find the valid line of code/assembler. Without debug symbols, you will see the assembler code for the routine shown, but if your kernel has debug symbols the C code will also be available. (Debug symbols can be enabled in the kernel hacking menu of the menu configuration.) For example:: - objdump -r -S -l --disassemble net/dccp/ipv4.o + $ objdump -r -S -l --disassemble net/dccp/ipv4.o .. note:: @@ -157,7 +101,7 @@ in the kernel hacking menu of the menu configuration.) For example:: If you don't have access to the code you can also debug on some crash dumps e.g. crash dump output as shown by Dave Miller:: - EIP is at ip_queue_xmit+0x14/0x4c0 + EIP is at +0x14/0x4c0 ... Code: 44 24 04 e8 6f 05 00 00 e9 e8 fe ff ff 8d 76 00 8d bc 27 00 00 00 00 55 57 56 53 81 ec bc 00 00 00 8b ac 24 d0 00 00 00 8b 5d 08 @@ -185,6 +129,9 @@ e.g. crash dump output as shown by Dave Miller:: mov 0x8(%ebp), %ebx ! %ebx = skb->sk mov 0x13c(%ebx), %eax ! %eax = inet_sk(sk)->opt +gdb +--- + In addition, you can use GDB to figure out the exact file and line number of the OOPS from the ``vmlinux`` file. If you have ``CONFIG_DEBUG_INFO`` enabled, you can simply copy the EIP value from the @@ -194,7 +141,7 @@ OOPS:: And use GDB to translate that to human-readable form:: - gdb vmlinux + $ gdb vmlinux (gdb) l *0xc021e50e If you don't have ``CONFIG_DEBUG_INFO`` enabled, you use the function @@ -204,14 +151,32 @@ offset from the OOPS:: And recompile the kernel with ``CONFIG_DEBUG_INFO`` enabled:: - make vmlinux - gdb vmlinux + $ make vmlinux + $ gdb vmlinux + (gdb) l *vt_ioctl+0xda8 + 0x1888 is in vt_ioctl (drivers/tty/vt/vt_ioctl.c:293). + 288 { + 289 struct vc_data *vc = NULL; + 290 int ret = 0; + 291 + 292 console_lock(); + 293 if (VT_BUSY(vc_num)) + 294 ret = -EBUSY; + 295 else if (vc_num) + 296 vc = vc_deallocate(vc_num); + 297 console_unlock(); + +or, if you want to be more verbose:: + (gdb) p vt_ioctl - (gdb) l *(0x<address of vt_ioctl> + 0xda8) + $1 = {int (struct tty_struct *, unsigned int, unsigned long)} 0xae0 <vt_ioctl> + (gdb) l *0xae0+0xda8 -or, as one command:: +You could, instead, use the object file:: - (gdb) l *(vt_ioctl + 0xda8) + $ make drivers/tty/ + $ gdb drivers/tty/vt/vt_ioctl.o + (gdb) l *vt_ioctl+0xda8 If you have a call trace, such as:: @@ -221,17 +186,11 @@ If you have a call trace, such as:: [<ffffffff8802770b>] :jbd:journal_stop+0x1be/0x1ee ... -this shows the problem in the :jbd: module. You can load that module in gdb -and list the relevant code:: - - gdb fs/jbd/jbd.ko - (gdb) p log_wait_commit - (gdb) l *(0x<address> + 0xa3) - -or:: - - (gdb) l *(log_wait_commit + 0xa3) +this shows the problem likely in the :jbd: module. You can load that module +in gdb and list the relevant code:: + $ gdb fs/jbd/jbd.ko + (gdb) l *log_wait_commit+0xa3 Another very useful option of the Kernel Hacking section in menuconfig is Debug memory allocations. This will help you see whether data has been -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
