On Fri, Aug 21, 2020 at 8:25 PM Sean Christopherson <sean.j.christopherson@xxxxxxxxx> wrote: > > On Thu, Aug 20, 2020 at 04:05:45PM -0700, Peter Shier wrote: > > When L2 uses PAE, L0 intercepts of L2 writes to CR0/CR3/CR4 call > > load_pdptrs to read the possibly updated PDPTEs from the guest > > physical address referenced by CR3. It loads them into > > vcpu->arch.walk_mmu->pdptrs and sets VCPU_EXREG_PDPTR in > > vcpu->arch.regs_dirty. > > > > At the subsequent assumed reentry into L2, the mmu will call > > vmx_load_mmu_pgd which calls ept_load_pdptrs. ept_load_pdptrs sees > > VCPU_EXREG_PDPTR set in vcpu->arch.regs_dirty and loads > > VMCS02.GUEST_PDPTRn from vcpu->arch.walk_mmu->pdptrs[]. This all works > > if the L2 CRn write intercept always resumes L2. > > > > The resume path calls vmx_check_nested_events which checks for > > exceptions, MTF, and expired VMX preemption timers. If > > vmx_check_nested_events finds any of these conditions pending it will > > reflect the corresponding exit into L1. Live migration at this point > > would also cause a missed immediate reentry into L2. > > > > After L1 exits, vmx_vcpu_run calls vmx_register_cache_reset which > > clears VCPU_EXREG_PDPTR in vcpu->arch.regs_dirty. When L2 next > > resumes, ept_load_pdptrs finds VCPU_EXREG_PDPTR clear in > > vcpu->arch.regs_dirty and does not load VMCS02.GUEST_PDPTRn from > > vcpu->arch.walk_mmu->pdptrs[]. prepare_vmcs02 will then load > > VMCS02.GUEST_PDPTRn from vmcs12->pdptr0/1/2/3 which contain the stale > > values stored at last L2 exit. A repro of this bug showed L2 entering > > triple fault immediately due to the bad VMCS02.GUEST_PDPTRn values. > > > > When L2 is in PAE paging mode add a call to ept_load_pdptrs before > > leaving L2. This will update VMCS02.GUEST_PDPTRn if they are dirty in > > vcpu->arch.walk_mmu->pdptrs[]. > > > > Tested: > > kvm-unit-tests with new directed test: vmx_mtf_pdpte_test. > > Verified that test fails without the fix. > > > > Also ran Google internal VMM with an Ubuntu 16.04 4.4.0-83 guest running a > > custom hypervisor with a 32-bit Windows XP L2 guest using PAE. Prior to fix > > would repro readily. Ran 14 simultaneous L2s for 140 iterations with no > > failures. > > > > Signed-off-by: Peter Shier <pshier@xxxxxxxxxx> > > Reviewed-by: Jim Mattson <jmattson@xxxxxxxxxx> > > --- > > Reviewed-by: Sean Christopherson <sean.j.christopherson@xxxxxxxxx> Ping. Thx
