> On Jun 8, 2020, at 22:13, Paolo Bonzini <pbonzini@xxxxxxxxxx> wrote:
>
> On 06/06/20 06:26, Eiichi Tsukata wrote:
>> Commit b1394e745b94 ("KVM: x86: fix APIC page invalidation") tried to
>> fix inappropriate APIC page invalidation by re-introducing arch specific
>> kvm_arch_mmu_notifier_invalidate_range() and calling it from
>> kvm_mmu_notifier_invalidate_range_start. But threre could be the
>> following race because VMCS APIC address cache can be updated
>> *before* it is unmapped.
>>
>> Race:
>> (Invalidator) kvm_mmu_notifier_invalidate_range_start()
>> (Invalidator) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD)
>> (KVM VCPU) vcpu_enter_guest()
>> (KVM VCPU) kvm_vcpu_reload_apic_access_page()
>> (Invalidator) actually unmap page
>>
>> Symptom:
>> The above race can make Guest OS see already freed page and Guest OS
>> will see broken APIC register values.
>
> This is not exactly the issue. The values in the APIC-access page do
> not really matter, the problem is that the host physical address values
> won't match between the page tables and the APIC-access page address.
> Then the processor will not trap APIC accesses, and will instead show
> the raw contents of the APIC-access page (zeroes), and cause the crash
> as you mention below.
>
> Still, the race explains the symptoms and the patch matches this text in
> include/linux/mmu_notifier.h:
>
> * If the subsystem
> * can't guarantee that no additional references are taken to
> * the pages in the range, it has to implement the
> * invalidate_range() notifier to remove any references taken
> * after invalidate_range_start().
>
> where the "additional reference" is in the VMCS: because we have to
> account for kvm_vcpu_reload_apic_access_page running between
> invalidate_range_start() and invalidate_range_end(), we need to
> implement invalidate_range().
>
> The patch seems good, but I'd like Andrea Arcangeli to take a look as
> well so I've CCed him.
>
> Thank you very much!
>
> Paolo
>
Hello Paolo
Thanks for detailed explanation!
I’ll fix the commit message like this:
```
Symptom:
The above race can cause mismatch between the page tables and the
APIC-access page address in VMCS.Then the processor will not trap APIC
accesses, and will instead show the raw contents of the APIC-access page
(zeroes). Especially, Windows OS checks LAPIC modification so it can cause
BSOD crash with BugCheck CRITICAL_STRUCTURE_CORRUPTION (109). These symptoms
are the same as we previously saw in
https://bugzilla.kernel.org/show_bug.cgi?id=197951
and we are currently seeing in
https://bugzilla.redhat.com/show_bug.cgi?id=1751017.
To prevent mismatch between page tables and APIC-access page address,
this patch calls kvm_arch_mmu_notifier_invalidate_range() from
kvm_mmu_notifier_invalidate_range() instead of ..._range_start().
We need to implement invalidate_range() because we have to
account for kvm_vcpu_reload_apic_access_page() running between
invalidate_range_start() and invalidate_range_end().
```
Best
Eiichi
