On Fri, Mar 04, 2011 at 10:22:03AM +0100, Jean-Philippe Menil wrote: > Le 03/03/2011 16:55, Michael S. Tsirkin a écrit : > >On Thu, Mar 03, 2011 at 04:26:11PM +0100, Jean-Philippe Menil wrote: > >>Le 03/03/2011 15:47, Michael S. Tsirkin a écrit : > >>>On Tue, Mar 01, 2011 at 03:39:12PM +0100, Jean-Philippe Menil wrote: > >>>>so this time the bug is: > >>>> > >>>>[17882.612303] BUG: unable to handle kernel paging request at > >>>>0000000000002458 > >>>>[17882.612342] IP: [<ffffffffa03898a0>] kvm_set_irq+0x30/0x140 [kvm] > >>>> > >>>>markup_oops give me this: > >>>> > >>>>root@ayrshire:~# cat bug-0103.txt | perl markup_oops.pl -m > >>>>/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko > >>>>/boot/vmlinuz-2.6.37.2-dsiun-110105+ > >>>>vmaoffset = 18446744072102621184 ffffffffa0389871: 48 89 e5 mov > >>>>%rsp,%rbp > >>>> ffffffffa0389874: 41 57 push %r15 > >>>> ffffffffa0389876: 41 89 cf mov %ecx,%r15d | %r15 > >>>>=> 1 %ecx = 1 > >>>> ffffffffa0389879: 41 56 push %r14 | %r14 > >>>>=> ffffffffa038aad0 > >>>> ffffffffa038987b: 41 55 push %r13 > >>>> ffffffffa038987d: 49 89 fd mov %rdi,%r13 | %edi > >>>>= 0 %r13 => 0 > >>>> ffffffffa0389880: 41 54 push %r12 | %r12 => 0 > >>>> ffffffffa0389882: 53 push %rbx > >>>> ffffffffa0389883: 89 d3 mov %edx,%ebx | %ebx => 1a > >>>> ffffffffa0389885: 48 81 ec a8 00 00 00 sub $0xa8,%rsp > >>>> ffffffffa038988c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx > >>>># ffffffffa0389892<kvm_set_irq+0x22> > >>>> ffffffffa0389892: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) | > >>>>%esi = 0 > >>>> ffffffffa0389898: 85 d2 test %edx,%edx | %edx => 0 > >>>> ffffffffa038989a: 0f 85 d5 00 00 00 jne ffffffffa0389975 > >>>><kvm_set_irq+0x105> > >>>>*ffffffffa03898a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax | > >>>>%eax = 0 %r13 = 0<--- faulting instruction > >>>> ffffffffa03898a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx > >>>> ffffffffa03898ad: 73 61 jae ffffffffa0389910 > >>>><kvm_set_irq+0xa0> > >>>> ffffffffa03898af: 89 db mov %ebx,%ebx > >>>> ffffffffa03898b1: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax > >>>> ffffffffa03898b8: 00 > >>>> ffffffffa03898b9: 48 85 c0 test %rax,%rax > >>>> ffffffffa03898bc: 74 52 je ffffffffa0389910 > >>>><kvm_set_irq+0xa0> > >>>> ffffffffa03898be: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx > >>>> ffffffffa03898c5: 31 db xor %ebx,%ebx > >>>> ffffffffa03898c7: 48 8b 08 mov (%rax),%rcx > >>>> ffffffffa03898ca: 83 c3 01 add $0x1,%ebx > >>>> ffffffffa03898cd: 0f 18 09 prefetcht0 (%rcx) > >>>> ffffffffa03898d0: 48 8b 48 e0 mov -0x20(%rax),%rcx > >>>> ffffffffa03898d4: 48 89 0a mov %rcx,(%rdx) > >>>> ffffffffa03898d7: 48 8b 48 e8 mov -0x18(%rax),%rcx > >>>> ffffffffa03898db: 48 89 4a 08 mov %rcx,0x8(%rdx) > >>>> ffffffffa03898df: 48 8b 48 f0 mov -0x10(%rax),%rcx > >>>> ffffffffa03898e3: 48 89 4a 10 mov %rcx,0x10(%rdx) > >>>> ffffffffa03898e7: 48 8b 48 f8 mov -0x8(%rax),%rcx > >>>> ffffffffa03898eb: 48 89 4a 18 mov %rcx,0x18(%rdx) > >>>> > >>>>wich correspond to offset 68a0 (from objdump): > >>>> > >>>>kvm_set_irq(): > >>>>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161 > >>>> 68a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax > >>>>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162 > >>>> 68a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx > >>>> > >>>>root@ayrshire:~# addr2line -e > >>>>/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko > >>>>0x68a0 > >>>>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161 > >>>> > >>>>So here kvm->irq_routing is null. > >>>> > >>>>How can it be? > >>>> > >>>>Regards. > >>>Not null, this seems to be invalid. > >>>I suspect use after free where the kvm pointer is > >>>pointing at some random memory. Use after free? > >>>Could you please try enabling a slab debugger, > >>>recompile and rerun the test? > >>> > >>Hi, > >> > >>I'm not sure to activate the right thing. > >>Is that what you want? > >> > >>CONFIG_SLAB=y > >>CONFIG_SLABINFO=y > >>CONFIG_DEBUG_SLAB=y > >>CONFIG_DEBUG_SLAB_LEAK=y > >> > >>Regards. > >Yes, maybe disable SLAB_LEAK. > > > >>-- > >>Jean-Philippe Menil - Pôle réseau Service IRTS > >>DSI Université de Nantes > >>jean-philippe.menil@xxxxxxxxxxxxxx > >>Tel : 02.53.48.49.27 - Fax : 02.53.48.49.09 > >-- > >To unsubscribe from this list: send the line "unsubscribe kvm" in > >the body of a message to majordomo@xxxxxxxxxxxxxxx > >More majordomo info at http://vger.kernel.org/majordomo-info.html > Hi, > > so this time, here is what markup_oops says: > > root@ayrshire:~# cat oops-0403.txt | perl markup_oops.pl -m > /lib/modules/2.6.37.2.999-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko > /boot/vmlinuz-2.6.37.2.999-dsiun-110105+ > vmaoffset = 18446744072102948864 ffffffffa03d9811: 48 89 e5 > mov %rsp,%rbp > ffffffffa03d9814: 41 57 push %r15 > ffffffffa03d9816: 41 89 cf mov %ecx,%r15d | > %r15 => 1 %ecx = 1 > ffffffffa03d9819: 41 56 push %r14 | > %r14 => ffffffffa03daa50 > ffffffffa03d981b: 41 55 push %r13 > ffffffffa03d981d: 49 89 fd mov %rdi,%r13 | > %edi = 6b6b6b6b6b6b6b6b %r13 => 6b6b6b6b6b6b6b6b > ffffffffa03d9820: 41 54 push %r12 | > %r12 => 6b6b6b6b6b6b6b6b > ffffffffa03d9822: 53 push %rbx > ffffffffa03d9823: 89 d3 mov %edx,%ebx | > %ebx => 6b6b6b6b > ffffffffa03d9825: 48 81 ec a8 00 00 00 sub $0xa8,%rsp > ffffffffa03d982c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx > # ffffffffa03d9832 <kvm_set_irq+0x22> > ffffffffa03d9832: 89 b5 3c ff ff ff mov > %esi,-0xc4(%rbp) | %esi = 0 > ffffffffa03d9838: 85 d2 test %edx,%edx | > %edx => 0 > ffffffffa03d983a: 0f 85 d5 00 00 00 jne > ffffffffa03d9915 <kvm_set_irq+0x105> > *ffffffffa03d9840: 49 8b 85 58 24 00 00 mov > 0x2458(%r13),%rax | %eax = 0 %r13 = 6b6b6b6b6b6b6b6b <--- faulting > instruction > ffffffffa03d9847: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx > ffffffffa03d984d: 73 61 jae > ffffffffa03d98b0 <kvm_set_irq+0xa0> > ffffffffa03d984f: 89 db mov %ebx,%ebx > ffffffffa03d9851: 48 8b 84 d8 30 01 00 mov > 0x130(%rax,%rbx,8),%rax > ffffffffa03d9858: 00 > ffffffffa03d9859: 48 85 c0 test %rax,%rax > ffffffffa03d985c: 74 52 je > ffffffffa03d98b0 <kvm_set_irq+0xa0> > ffffffffa03d985e: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx > ffffffffa03d9865: 31 db xor %ebx,%ebx > ffffffffa03d9867: 48 8b 08 mov (%rax),%rcx > ffffffffa03d986a: 83 c3 01 add $0x1,%ebx > ffffffffa03d986d: 0f 18 09 prefetcht0 (%rcx) > ffffffffa03d9870: 48 8b 48 e0 mov -0x20(%rax),%rcx > ffffffffa03d9874: 48 89 0a mov %rcx,(%rdx) > ffffffffa03d9877: 48 8b 48 e8 mov -0x18(%rax),%rcx > ffffffffa03d987b: 48 89 4a 08 mov %rcx,0x8(%rdx) > ffffffffa03d987f: 48 8b 48 f0 mov -0x10(%rax),%rcx > ffffffffa03d9883: 48 89 4a 10 mov %rcx,0x10(%rdx) > ffffffffa03d9887: 48 8b 48 f8 mov -0x8(%rax),%rcx > ffffffffa03d988b: 48 89 4a 18 mov %rcx,0x18(%rdx) > > > Is that you wanted, the "6b6b6b6b6b6b6b6b" ? > > Regards. Yes, excellent. So now we can detect the problem by comparing kvm with 6b6b6b6b6b6b6b6b, and print out stuff to understand where this comes from. I will prepare such a debugging patch. For that, could you please tell me which kernel version, exactly, are you using? -- MST -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
