On Tue, 2021-07-27 at 00:34 +0200, Paolo Bonzini wrote: > On 13/07/21 16:20, Maxim Levitsky wrote: > > + mutex_lock(&vcpu->kvm->apicv_update_lock); > > + > > vcpu->arch.apicv_active = kvm_apicv_activated(vcpu->kvm); > > kvm_apic_update_apicv(vcpu); > > static_call(kvm_x86_refresh_apicv_exec_ctrl)(vcpu); > > @@ -9246,6 +9248,8 @@ void kvm_vcpu_update_apicv(struct kvm_vcpu *vcpu) > > */ > > if (!vcpu->arch.apicv_active) > > kvm_make_request(KVM_REQ_EVENT, vcpu); > > + > > + mutex_unlock(&vcpu->kvm->apicv_update_lock); > > Does this whole piece of code need the lock/unlock? Does it work and/or > make sense to do the unlock immediately after mutex_lock()? This makes > it clearer that the mutex is being to synchronize against the requestor. Yes, I do need to hold the mutex for the whole duration. The requester does the following: 1. Take the mutex 2. Kick all the vCPUs out of the guest mode with KVM_REQ_EVENT At that point all these vCPUs will be (or soon will be) stuck on the mutex and guaranteed to be outside of the guest mode. which is exactly what I need to avoid them entering the guest mode as long as the AVIC's memslot state is not up to date. 3. Update kvm->arch.apicv_inhibit_reasons. I removed the cmpchg loop since it is now protected under the lock anyway. This step doesn't have to be done at this point, but should be done while mutex is held so that there is no need to cmpchg and such. This itself isn't the justification for the mutex. 4. Update the memslot 5. Release the mutex. Only now all other vCPUs are permitted to enter the guest mode again (since only now the memslot is up to date) and they will also update their per-vcpu AVIC enablement prior to entering it. I think it might be possible to avoid the mutex, but I am not sure if this is worth it: First of all, the above sync sequence is only really needed when we enable AVIC. (Because from the moment we enable the memslot and to the moment the vCPU enables the AVIC, it must not be in guest mode as otherwise it will access the dummy page in the memslot without VMexit, instead of going through AVIC vmexit/acceleration. The other way around is OK. IF we disable the memslot, and a vCPU still has a enabled AVIC, it will just get a page fault which will be correctly emulated as APIC read/write by the MMIO page fault. If I had a guarantee that when I enable the memslot (either as it done today or using the kvm_zap_gfn_range (which I strongly prefer), would always raise a TLB flush request on all vCPUs, then I could (ab)use that request to update local AVIC state. Or I can just always check if local AVIC state matches the memslot and update if it doesn't prior to guest mode entry. I still think I would prefer a mutex to be 100% sure that there are no races, since the whole AVIC disablement isn't something that is done often anyway. Best regards, Maxim Levitsky > > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > index ed4d1581d502..ba5d5d9ebc64 100644 > > --- a/virt/kvm/kvm_main.c > > +++ b/virt/kvm/kvm_main.c > > @@ -943,6 +943,7 @@ static struct kvm *kvm_create_vm(unsigned long type) > > mutex_init(&kvm->irq_lock); > > mutex_init(&kvm->slots_lock); > > mutex_init(&kvm->slots_arch_lock); > > + mutex_init(&kvm->apicv_update_lock); > > INIT_LIST_HEAD(&kvm->devices); > > > > BUILD_BUG_ON(KVM_MEM_SLOTS_NUM > SHRT_MAX); > > > > Please add comments above fields that are protected by this lock > (anything but apicv_inhibit_reasons?), and especially move it to kvm->arch. I agree, I will do this. > > Paolo
