Patch "KVM: Stop processing *all* memslots when "null" mmu_notifier handler is found" has been added to the 6.9-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    KVM: Stop processing *all* memslots when "null" mmu_notifier handler is found

to the 6.9-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     kvm-stop-processing-all-memslots-when-null-mmu_notif.patch
and it can be found in the queue-6.9 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 47a29c14df8610cf93fe93f05cf3becaf0d629da
Author: Babu Moger <babu.moger@xxxxxxx>
Date:   Wed Jun 12 09:41:51 2024 -0500

    KVM: Stop processing *all* memslots when "null" mmu_notifier handler is found
    
    [ Upstream commit c3f3edf73a8f854f8766a69d2734198a58762e33 ]
    
    Bail from outer address space loop, not just the inner memslot loop, when
    a "null" handler is encountered by __kvm_handle_hva_range(), which is the
    intended behavior.  On x86, which has multiple address spaces thanks to
    SMM emulation, breaking from just the memslot loop results in undefined
    behavior due to assigning the non-existent return value from kvm_null_fn()
    to a bool.
    
    In practice, the bug is benign as kvm_mmu_notifier_invalidate_range_end()
    is the only caller that passes handler=kvm_null_fn, and it doesn't set
    flush_on_ret, i.e. assigning garbage to r.ret is ultimately ignored.  And
    for most configuration the compiler elides the entire sequence, i.e. there
    is no undefined behavior at runtime.
    
      ------------[ cut here ]------------
      UBSAN: invalid-load in arch/x86/kvm/../../../virt/kvm/kvm_main.c:655:10
      load of value 160 is not a valid value for type '_Bool'
      CPU: 370 PID: 8246 Comm: CPU 0/KVM Not tainted 6.8.2-amdsos-build58-ubuntu-22.04+ #1
      Hardware name: AMD Corporation Sh54p/Sh54p, BIOS WPC4429N 04/25/2024
      Call Trace:
       <TASK>
       dump_stack_lvl+0x48/0x60
       ubsan_epilogue+0x5/0x30
       __ubsan_handle_load_invalid_value+0x79/0x80
       kvm_mmu_notifier_invalidate_range_end.cold+0x18/0x4f [kvm]
       __mmu_notifier_invalidate_range_end+0x63/0xe0
       __split_huge_pmd+0x367/0xfc0
       do_huge_pmd_wp_page+0x1cc/0x380
       __handle_mm_fault+0x8ee/0xe50
       handle_mm_fault+0xe4/0x4a0
       __get_user_pages+0x190/0x840
       get_user_pages_unlocked+0xe0/0x590
       hva_to_pfn+0x114/0x550 [kvm]
       kvm_faultin_pfn+0xed/0x5b0 [kvm]
       kvm_tdp_page_fault+0x123/0x170 [kvm]
       kvm_mmu_page_fault+0x244/0xaa0 [kvm]
       vcpu_enter_guest+0x592/0x1070 [kvm]
       kvm_arch_vcpu_ioctl_run+0x145/0x8a0 [kvm]
       kvm_vcpu_ioctl+0x288/0x6d0 [kvm]
       __x64_sys_ioctl+0x8f/0xd0
       do_syscall_64+0x77/0x120
       entry_SYSCALL_64_after_hwframe+0x6e/0x76
       </TASK>
      ---[ end trace ]---
    
    Fixes: 071064f14d87 ("KVM: Don't take mmu_lock for range invalidation unless necessary")
    Signed-off-by: Babu Moger <babu.moger@xxxxxxx>
    Link: https://lore.kernel.org/r/b8723d39903b64c241c50f5513f804390c7b5eec.1718203311.git.babu.moger@xxxxxxx
    [sean: massage changelog]
    Signed-off-by: Sean Christopherson <seanjc@xxxxxxxxxx>
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index ff0a20565f908..3b7fa9a62c62f 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -650,7 +650,7 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm,
 					range->on_lock(kvm);
 
 				if (IS_KVM_NULL_FN(range->handler))
-					break;
+					goto mmu_unlock;
 			}
 			r.ret |= range->handler(kvm, &gfn_range);
 		}
@@ -659,6 +659,7 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm,
 	if (range->flush_on_ret && r.ret)
 		kvm_flush_remote_tlbs(kvm);
 
+mmu_unlock:
 	if (r.found_memslot)
 		KVM_MMU_UNLOCK(kvm);