On Fri, Aug 4, 2017 at 3:03 PM, Anatol Pomozov <anatol.pomozov@xxxxxxxxx> wrote: > Hello > > I am testing my own x86_64 kernel with QEMU/KVM and it report cryptic > error "KVM internal error. Suberror: 1". I believe this emulation > error happens in Linux kernel most likely in arch/x86/kvm/emulate.c > > I tested KVM with Linux 4.4.0, 4.12.3 and both versions return this > emulation error. It is strange as QEMU emulation and VMWare work fine > with the same kernel binary. > > > How to understand what is the KVM issue and why it fails? Is there a > way to obtain more information about the error? > > > Here is my code where the issue happens: > > 0000000000100c20 <console_write>: > 100c20: 48 85 f6 test %rsi,%rsi > 100c23: 0f 84 11 05 00 00 je 10113a <console_write+0x51a> > 100c29: 45 31 c0 xor %r8d,%r8d > 100c2c: b9 0e 8f 0b 00 mov $0xb8f0e,%ecx > 100c31: 66 66 66 66 66 66 2e data32 data32 data32 data32 > data32 nopw %cs:0x0(%rax,%rax,1) > 100c38: 0f 1f 84 00 00 00 00 > 100c3f: 00 > 100c40: 0f be 07 movsbl (%rdi),%eax > 100c43: 0f b7 d0 movzwl %ax,%edx > 100c46: 83 fa 0a cmp $0xa,%edx > 100c49: 0f 85 91 00 00 00 jne 100ce0 <console_write+0xc0> > 100c4f: 48 8b 05 aa 83 00 00 mov 0x83aa(%rip),%rax > # 109000 <row> > 100c56: 48 8d 50 01 lea 0x1(%rax),%rdx > 100c5a: 48 89 15 9f 83 00 00 mov %rdx,0x839f(%rip) > # 109000 <row> > 100c61: 48 83 fa 19 cmp $0x19,%rdx > 100c65: 0f 85 35 04 00 00 jne 1010a0 <console_write+0x480> > 100c6b: 31 d2 xor %edx,%edx > 100c6d: 0f 1f 00 nopl (%rax) > 100c70: 0f 10 84 51 92 f1 ff movups -0xe6e(%rcx,%rdx,2),%xmm0 Likely this instruction is accessing an MMIO region (probably a VGA buffer, given "mov $0xb8f0e,%ecx"). This will cause a trap to KVM and KVM will try to emulate the instruction. The instruction doing the MMIO access is an SSE instruction, which KVM does not have support to emulate, thus the internal error. If you didn't hand write this assembly I would try compiling your kernel with compilation flags to disable SSE code generation and see if that fixes your issue. > > ^^^^^^^^^^^^ KVM error happens here ^^^^^^^ See registers stack below > > 100c77: ff > 100c78: 0f 11 84 51 f2 f0 ff movups %xmm0,-0xf0e(%rcx,%rdx,2) > 100c7f: ff > 100c80: 48 83 c2 08 add $0x8,%rdx > 100c84: 48 81 fa 80 07 00 00 cmp $0x780,%rdx > 100c8b: 75 e3 jne 100c70 <console_write+0x50> > 100c8d: 48 89 05 6c 83 00 00 mov %rax,0x836c(%rip) > # 109000 <row> > 100c94: 48 c7 05 c1 3b 00 00 movq $0xb8f00,0x3bc1(%rip) > # 104860 <cursor> > 100c9b: 00 8f 0b 00 > 100c9f: 4c 8b 0d 62 83 00 00 mov 0x8362(%rip),%r9 > # 109008 <column> > 100ca6: 4d 85 c9 test %r9,%r9 > 100ca9: 0f 84 71 04 00 00 je 101120 <console_write+0x500> > 100caf: 49 8d 41 ff lea -0x1(%r9),%rax > 100cb3: 44 89 ca mov %r9d,%edx > 100cb6: 83 e2 07 and $0x7,%edx > 100cb9: 48 83 f8 07 cmp $0x7,%rax > 100cbd: 0f 83 f6 03 00 00 jae 1010b9 <console_write+0x499> > > > > > > Registers at KVM error as reported by QEMU > > RAX=0000000000000018 RBX=0000000000000001 RCX=00000000000b8f0e > RDX=0000000000000000 > RSI=0000000000000001 RDI=000000000010445a RBP=000000000010445b > RSP=000000000010af78 > R8 =0000000000000000 R9 =00000000000b8f56 R10=000000000010445c > R11=0000000000000004 > R12=000000000010445a R13=000000007fffffd3 R14=0000000000000001 > R15=000000000000002c > RIP=0000000000100c70 RFL=00010246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 > ES =0000 0000000000000000 ffffffff 00c00000 > CS =0008 0000000000000000 00000000 00209900 DPL=0 CS64 [--A] > SS =0000 0000000000000000 ffffffff 00c00000 > DS =0000 0000000000000000 ffffffff 00c00000 > FS =0000 0000000000000000 ffffffff 00c00000 > GS =0000 0000000000000000 ffffffff 00c00000 > LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT > TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy > GDT= 0000000000104518 0000000f > IDT= 0000000000105030 00000fff > CR0=80000033 CR2=0000000000000000 CR3=0000000000107000 CR4=00000620 > DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 > DR3=0000000000000000 > DR6=00000000ffff0ff0 DR7=0000000000000400 > EFER=0000000000000500
