Hello, The following program triggers use-after-free in vmx_check_nested_events: https://gist.githubusercontent.com/dvyukov/30d798b75411474f29bc7dc203a7e5f0/raw/e1613e010ea88f20ee7a28fc44e8dd5861b0c048/gistfile1.txt BUG: KASAN: use-after-free in nested_cpu_has_preemption_timer arch/x86/kvm/vmx.c:1347 [inline] at addr ffff880063b62f68 BUG: KASAN: use-after-free in vmx_check_nested_events+0x6ab/0x720 arch/x86/kvm/vmx.c:10661 at addr ffff880063b62f68 Read of size 4 by task a.out/2998 CPU: 0 PID: 2998 Comm: a.out Not tainted 4.10.0+ #297 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:16 [inline] dump_stack+0x2fb/0x3fd lib/dump_stack.c:52 kasan_object_err+0x1c/0x90 mm/kasan/report.c:166 print_address_description mm/kasan/report.c:208 [inline] kasan_report_error mm/kasan/report.c:292 [inline] kasan_report.part.2+0x1b0/0x460 mm/kasan/report.c:314 kasan_report mm/kasan/report.c:346 [inline] __asan_report_load_n_noabort+0x24/0x30 mm/kasan/report.c:345 nested_cpu_has_preemption_timer arch/x86/kvm/vmx.c:1347 [inline] vmx_check_nested_events+0x6ab/0x720 arch/x86/kvm/vmx.c:10661 kvm_vcpu_running arch/x86/kvm/x86.c:7031 [inline] vcpu_run arch/x86/kvm/x86.c:7045 [inline] kvm_arch_vcpu_ioctl_run+0x33e/0x4840 arch/x86/kvm/x86.c:7207 kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2572 vfs_ioctl fs/ioctl.c:45 [inline] do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:685 SYSC_ioctl fs/ioctl.c:700 [inline] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691 entry_SYSCALL_64_fastpath+0x1f/0xc2 RIP: 0033:0x450199 RSP: 002b:00007efc5fbcfcd8 EFLAGS: 00000297 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000450199 RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000005 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000297 R12: 0000000000000000 R13: 0000000000000000 R14: 00007efc5fbd09c0 R15: 00007efc5fbd0700 Object at ffff880063b62c80, in cache kmalloc-4096 size: 4096 Allocated: PID = 2990 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59 save_stack+0x43/0xd0 mm/kasan/kasan.c:513 set_track mm/kasan/kasan.c:525 [inline] kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:616 kmem_cache_alloc_trace+0x10b/0x6e0 mm/slab.c:3638 kmalloc include/linux/slab.h:490 [inline] enter_vmx_operation arch/x86/kvm/vmx.c:7062 [inline] handle_vmon+0x3a4/0x6f0 arch/x86/kvm/vmx.c:7150 vmx_handle_exit+0xfc0/0x3f00 arch/x86/kvm/vmx.c:8528 vcpu_enter_guest arch/x86/kvm/x86.c:6984 [inline] vcpu_run arch/x86/kvm/x86.c:7046 [inline] kvm_arch_vcpu_ioctl_run+0x1418/0x4840 arch/x86/kvm/x86.c:7207 kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2572 vfs_ioctl fs/ioctl.c:45 [inline] do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:685 SYSC_ioctl fs/ioctl.c:700 [inline] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691 entry_SYSCALL_64_fastpath+0x1f/0xc2 Freed: PID = 2991 save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59 save_stack+0x43/0xd0 mm/kasan/kasan.c:513 set_track mm/kasan/kasan.c:525 [inline] kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:589 __cache_free mm/slab.c:3514 [inline] kfree+0xd3/0x250 mm/slab.c:3831 free_nested.part.79+0x2f6/0xc50 arch/x86/kvm/vmx.c:7239 vmx_leave_nested arch/x86/kvm/vmx.c:3257 [inline] vmx_set_msr+0x69d/0x1950 arch/x86/kvm/vmx.c:3325 kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1101 do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1130 __msr_io arch/x86/kvm/x86.c:2579 [inline] msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2616 kvm_arch_vcpu_ioctl+0x35b/0x46a0 arch/x86/kvm/x86.c:3499 kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2723 vfs_ioctl fs/ioctl.c:45 [inline] do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:685 SYSC_ioctl fs/ioctl.c:700 [inline] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:691 entry_SYSCALL_64_fastpath+0x1f/0xc2 Memory state around the buggy address: ffff880063b62e00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff880063b62e80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb >ffff880063b62f00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff880063b62f80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff880063b63000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ================================================================== On commit 4bdb75599690f0759b06adfc80d1bcf42e056473 with the following local diff: https://gist.githubusercontent.com/dvyukov/44429ac8fe26c43cd324bcc1212245d3/raw/206599fa169ea6b64a8def98fa3bb2fb1e4bc874/gistfile1.txt Sometimes it also causes (report on commit 44b4b461a0fb407507b46ea76a71376d74de7058): BUG: KASAN: use-after-free in vmcs12_guest_cr0 arch/x86/kvm/vmx.c:10649 [inline] at addr ffff8800658d6d68 BUG: KASAN: use-after-free in prepare_vmcs12 arch/x86/kvm/vmx.c:10775 [inline] at addr ffff8800658d6d68 BUG: KASAN: use-after-free in nested_vmx_vmexit+0x6c24/0x74d0 arch/x86/kvm/vmx.c:11080 at addr ffff8800658d6d68 Read of size 8 by task a.out/2926 CPU: 2 PID: 2926 Comm: a.out Not tainted 4.10.0-rc4+ #181 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:15 [inline] dump_stack+0x2ee/0x3ef lib/dump_stack.c:51 kasan_object_err+0x1c/0x70 mm/kasan/report.c:165 print_address_description mm/kasan/report.c:203 [inline] kasan_report_error mm/kasan/report.c:287 [inline] kasan_report+0x1b6/0x460 mm/kasan/report.c:307 __asan_report_load_n_noabort+0xf/0x20 mm/kasan/report.c:343 vmcs12_guest_cr0 arch/x86/kvm/vmx.c:10649 [inline] prepare_vmcs12 arch/x86/kvm/vmx.c:10775 [inline] nested_vmx_vmexit+0x6c24/0x74d0 arch/x86/kvm/vmx.c:11080 vmx_handle_exit+0xf82/0x3fc0 arch/x86/kvm/vmx.c:8571 vcpu_enter_guest arch/x86/kvm/x86.c:6905 [inline] vcpu_run arch/x86/kvm/x86.c:6964 [inline] kvm_arch_vcpu_ioctl_run+0xf7e/0x4890 arch/x86/kvm/x86.c:7122 kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2570 vfs_ioctl fs/ioctl.c:43 [inline] do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683 SYSC_ioctl fs/ioctl.c:698 [inline] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689 entry_SYSCALL_64_fastpath+0x1f/0xc2 RIP: 0033:0x450199 RSP: 002b:00007f8307392cd8 EFLAGS: 00000297 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 0000000000450199 RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000005 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000297 R12: 0000000000000000 R13: 0000000000000000 R14: 00007f83073939c0 R15: 00007f8307393700 Object at ffff8800658d6bc0, in cache kmalloc-4096 size: 4096 Allocated: PID = 2918 [<ffffffff812b2686>] save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57 [<ffffffff81a0e8c3>] save_stack+0x43/0xd0 mm/kasan/kasan.c:502 [<ffffffff81a0eb8a>] set_track mm/kasan/kasan.c:514 [inline] [<ffffffff81a0eb8a>] kasan_kmalloc+0xaa/0xd0 mm/kasan/kasan.c:605 [<ffffffff81a0b3db>] kmem_cache_alloc_trace+0x10b/0x670 mm/slab.c:3629 [<ffffffff811bc9cd>] kmalloc include/linux/slab.h:490 [inline] [<ffffffff811bc9cd>] handle_vmon+0x35d/0x790 arch/x86/kvm/vmx.c:7230 [<ffffffff811d7d66>] vmx_handle_exit+0xf96/0x3fc0 arch/x86/kvm/vmx.c:8634 [<ffffffff810f045e>] vcpu_enter_guest arch/x86/kvm/x86.c:6905 [inline] [<ffffffff810f045e>] vcpu_run arch/x86/kvm/x86.c:6964 [inline] [<ffffffff810f045e>] kvm_arch_vcpu_ioctl_run+0xf7e/0x4890 arch/x86/kvm/x86.c:7122 [<ffffffff8107a8a3>] kvm_vcpu_ioctl+0x673/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2570 [<ffffffff81aa5aaf>] vfs_ioctl fs/ioctl.c:43 [inline] [<ffffffff81aa5aaf>] do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683 [<ffffffff81aa710f>] SYSC_ioctl fs/ioctl.c:698 [inline] [<ffffffff81aa710f>] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689 [<ffffffff841cacc1>] entry_SYSCALL_64_fastpath+0x1f/0xc2 Freed: PID = 2919 [<ffffffff812b2686>] save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:57 [<ffffffff81a0e8c3>] save_stack+0x43/0xd0 mm/kasan/kasan.c:502 [<ffffffff81a0f1ff>] set_track mm/kasan/kasan.c:514 [inline] [<ffffffff81a0f1ff>] kasan_slab_free+0x6f/0xb0 mm/kasan/kasan.c:578 [<ffffffff81a0d0b3>] __cache_free mm/slab.c:3505 [inline] [<ffffffff81a0d0b3>] kfree+0xd3/0x250 mm/slab.c:3822 [<ffffffff811b6c26>] free_nested.part.83+0x2f6/0xc60 arch/x86/kvm/vmx.c:7348 [<ffffffff811eae95>] vmx_leave_nested arch/x86/kvm/vmx.c:3314 [inline] [<ffffffff811eae95>] vmx_set_msr+0x665/0x1910 arch/x86/kvm/vmx.c:3381 [<ffffffff810960b4>] kvm_set_msr+0xd4/0x170 arch/x86/kvm/x86.c:1097 [<ffffffff8109644e>] do_set_msr+0x11e/0x190 arch/x86/kvm/x86.c:1126 [<ffffffff810c755b>] __msr_io arch/x86/kvm/x86.c:2544 [inline] [<ffffffff810c755b>] msr_io+0x24b/0x450 arch/x86/kvm/x86.c:2581 [<ffffffff810db4bb>] kvm_arch_vcpu_ioctl+0x35b/0x46e0 arch/x86/kvm/x86.c:3462 [<ffffffff8107a462>] kvm_vcpu_ioctl+0x232/0x1120 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2721 [<ffffffff81aa5aaf>] vfs_ioctl fs/ioctl.c:43 [inline] [<ffffffff81aa5aaf>] do_vfs_ioctl+0x1bf/0x1790 fs/ioctl.c:683 [<ffffffff81aa710f>] SYSC_ioctl fs/ioctl.c:698 [inline] [<ffffffff81aa710f>] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:689 [<ffffffff841cacc1>] entry_SYSCALL_64_fastpath+0x1f/0xc2 Jim noted both paths are protected by vcpu run lock, so this is probably not a low-level race but rather a leftover dangling reference. nested_vmx_run is called only from handle_vmlaunch/vmresume. Could we exit from L2, release vcpu mutex, return to userspace, at this point cached_vmcs12 is freed, then we reacquire vcpu mutex and re-enter directly into L2? Looking at the report, it looks like what happened -- VMXON and UAF happened in different threads, so we obviously returned to userspace and dropped the mutex in between. And then somehow get into nested_vmx_vmexit which means that leave_guest_mode wasn't called after VMXON. Is it possible to return to userspace from vmx_handle_exit without leaving guest mode?
