On Thu, Apr 06, 2023, bugzilla-daemon@xxxxxxxxxx wrote: > Assume KVM runs in L0, LHV runs in L1, the nested guest runs in L2. > > The code in LHV performs an experiment (called "Experiment 13" in serial > output) on CPU 0 to test the behavior of NMI blocking. The experiment steps > are: > 1. Prepare state such that the CPU is currently in L1 (LHV), and NMI is blocked > 2. Modify VMCS12 to make sure that L2 has virtual NMIs enabled (NMI exiting = > 1, Virtual NMIs = 1), and L2 does not block NMI (Blocking by NMI = 0) > 3. VM entry to L2 > 4. L2 performs VMCALL, get VM exit to L1 > 5. L1 checks whether NMI is blocked. > > The expected behavior is that NMI should be blocked, which is reproduced on > real hardware. According to Intel SDM, NMIs should be unblocked after VM entry > to L2 (step 3). After VM exit to L1 (step 4), NMI blocking does not change, so > NMIs are still unblocked. This behavior is reproducible on real hardware. > > However, when running on KVM, the experiment shows that at step 5, NMIs are > blocked in L1. Thus, I think NMI blocking is not implemented correctly in KVM's > nested virtualization. Ya, KVM blocks NMIs on nested NMI VM-Exits, but doesn't unblock NMIs for all other exit types. I believe this is the fix (untested): --- arch/x86/kvm/vmx/nested.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 96ede74a6067..4240a052628a 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4164,12 +4164,7 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu) nested_vmx_vmexit(vcpu, EXIT_REASON_EXCEPTION_NMI, NMI_VECTOR | INTR_TYPE_NMI_INTR | INTR_INFO_VALID_MASK, 0); - /* - * The NMI-triggered VM exit counts as injection: - * clear this one and block further NMIs. - */ vcpu->arch.nmi_pending = 0; - vmx_set_nmi_mask(vcpu, true); return 0; } @@ -4865,6 +4860,13 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR; } + /* + * NMIs are blocked on VM-Exit due to NMI, and unblocked by all + * other VM-Exit types. + */ + vmx_set_nmi_mask(vcpu, (u16)vm_exit_reason == EXIT_REASON_EXCEPTION_NMI && + !is_nmi(vmcs12->vm_exit_intr_info)); + if (vm_exit_reason != -1) trace_kvm_nested_vmexit_inject(vmcs12->vm_exit_reason, vmcs12->exit_qualification, base-commit: 0b87a6bfd1bdb47b766aa0641b7cf93f3d3227e9 -- > I am happy to explain how the experiment code works in detail. c.img also > reveals other NMI-related bugs in KVM. I am also happy to explain the other > bugs. I'm not sure I want to know ;-) If you can give a quick rundown of each bug, it would be quite helpful. Thanks!
