On Thu, Aug 20, 2020 at 04:05:45PM -0700, Peter Shier wrote: > When L2 uses PAE, L0 intercepts of L2 writes to CR0/CR3/CR4 call > load_pdptrs to read the possibly updated PDPTEs from the guest > physical address referenced by CR3. It loads them into > vcpu->arch.walk_mmu->pdptrs and sets VCPU_EXREG_PDPTR in > vcpu->arch.regs_dirty. > > At the subsequent assumed reentry into L2, the mmu will call > vmx_load_mmu_pgd which calls ept_load_pdptrs. ept_load_pdptrs sees > VCPU_EXREG_PDPTR set in vcpu->arch.regs_dirty and loads > VMCS02.GUEST_PDPTRn from vcpu->arch.walk_mmu->pdptrs[]. This all works > if the L2 CRn write intercept always resumes L2. > > The resume path calls vmx_check_nested_events which checks for > exceptions, MTF, and expired VMX preemption timers. If > vmx_check_nested_events finds any of these conditions pending it will > reflect the corresponding exit into L1. Live migration at this point > would also cause a missed immediate reentry into L2. > > After L1 exits, vmx_vcpu_run calls vmx_register_cache_reset which > clears VCPU_EXREG_PDPTR in vcpu->arch.regs_dirty. When L2 next > resumes, ept_load_pdptrs finds VCPU_EXREG_PDPTR clear in > vcpu->arch.regs_dirty and does not load VMCS02.GUEST_PDPTRn from > vcpu->arch.walk_mmu->pdptrs[]. prepare_vmcs02 will then load > VMCS02.GUEST_PDPTRn from vmcs12->pdptr0/1/2/3 which contain the stale > values stored at last L2 exit. A repro of this bug showed L2 entering > triple fault immediately due to the bad VMCS02.GUEST_PDPTRn values. > > When L2 is in PAE paging mode add a call to ept_load_pdptrs before > leaving L2. This will update VMCS02.GUEST_PDPTRn if they are dirty in > vcpu->arch.walk_mmu->pdptrs[]. > > Tested: > kvm-unit-tests with new directed test: vmx_mtf_pdpte_test. > Verified that test fails without the fix. > > Also ran Google internal VMM with an Ubuntu 16.04 4.4.0-83 guest running a > custom hypervisor with a 32-bit Windows XP L2 guest using PAE. Prior to fix > would repro readily. Ran 14 simultaneous L2s for 140 iterations with no > failures. > > Signed-off-by: Peter Shier <pshier@xxxxxxxxxx> > Reviewed-by: Jim Mattson <jmattson@xxxxxxxxxx> > --- Reviewed-by: Sean Christopherson <sean.j.christopherson@xxxxxxxxx>
