On Tue, Jun 18, 2019 at 7:52 PM Borislav Petkov <bp@xxxxxxxxx> wrote: > > On Mon, Jun 17, 2019 at 11:13:22AM -0400, George Kennedy wrote: > > 319f3: 0f 01 da vmload <--- svm_vcpu_run+0xa83 > > Hmm, so VMLOAD can fault for a bunch of reasons if you look at its > description in the APM. > > Looking at your Code: section and building with your config, rIP and the > insns around it point to: > > All code > ======== > 0: 00 55 89 add %dl,-0x77(%rbp) > 3: d2 45 89 rolb %cl,-0x77(%rbp) > 6: c9 leaveq > 7: 48 89 e5 mov %rsp,%rbp > a: 8b 45 18 mov 0x18(%rbp),%eax > d: 50 push %rax > e: 8b 45 10 mov 0x10(%rbp),%eax > 11: 50 push %rax > 12: e8 8a 42 6b 00 callq 0x6b42a1 > 17: 58 pop %rax > 18: 5a pop %rdx > 19: c9 leaveq > 1a: c3 retq > 1b: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) > 21: 66 66 66 66 90 data16 data16 data16 xchg %ax,%ax > 26: 55 push %rbp > 27: 48 89 e5 mov %rsp,%rbp > 2a:* 0f 0b ud2 <-- trapping instruction > 2c: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1) > 31: 66 66 66 66 90 data16 data16 data16 xchg %ax,%ax > 36: 55 push %rbp > 37: 48 89 e5 mov %rsp,%rbp > 3a: 41 55 push %r13 > 3c: 41 54 push %r12 > 3e: 41 rex.B > 3f: 89 .byte 0x89 > > 0000000000017f30 <__bpf_trace_kvm_nested_vmexit>: > 17f30: 55 push %rbp > 17f31: 89 d2 mov %edx,%edx > 17f33: 45 89 c9 mov %r9d,%r9d > 17f36: 48 89 e5 mov %rsp,%rbp > 17f39: 8b 45 18 mov 0x18(%rbp),%eax > 17f3c: 50 push %rax > 17f3d: 8b 45 10 mov 0x10(%rbp),%eax > 17f40: 50 push %rax > 17f41: e8 00 00 00 00 callq 17f46 <__bpf_trace_kvm_nested_vmexit+0x16> > 17f46: 58 pop %rax > 17f47: 5a pop %rdx > 17f48: c9 leaveq > 17f49: c3 retq > 17f4a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) > > 0000000000017f50 <kvm_spurious_fault>: > 17f50: e8 00 00 00 00 callq 17f55 <kvm_spurious_fault+0x5> > 17f55: 55 push %rbp > 17f56: 48 89 e5 mov %rsp,%rbp > 17f59: 0f 0b ud2 > 17f5b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1) > > so the invalid opcode splat is dumping the insn bytes around rIP the > UD2 happens but in order to see which of the VMLOAD conditions causes > the fault, you'd probably need to intercept the fault handler and dump > registers in the hypervisor to check. > > There's something else that's bothering me though: your splat is from > the guest yet there is svm_vcpu_run() mentioned there which should be > called by the hypervisor and not by the guest. Unless you're doing > nested virt stuff... > > Anyway, sorry that I cannot be of more help - I'm sure KVM guys would > make much more sense of it. I am not a KVM folk either, but FWIW syzkaller is capable of creating a double-nested VM. The code is somewhat VMX-specific, but it should be capable at least executing some SVM instructions inside of guest. This code setups VM to run a given instruction sequences (should be generic): https://github.com/google/syzkaller/blob/34bf9440bd06034f86b5d9ac8afbf078129cbdae/executor/common_kvm_amd64.h The instruction generator is based on Intel XED so it may be somewhat Intel-biased, but at least I see some mentions of SVM there: https://raw.githubusercontent.com/google/syzkaller/34bf9440bd06034f86b5d9ac8afbf078129cbdae/pkg/ifuzz/gen/all-enc-instructions.txt And this code can setup a double-nested VM, but it's VMX-specific: https://github.com/google/syzkaller/blob/34bf9440bd06034f86b5d9ac8afbf078129cbdae/executor/kvm.S#L125 If there is specific interested in testing SVM, it can make sense to improve SVM support in syzkaller to at least match VMX.
