On Wed, 2022-08-31 at 18:19 +0000, Sean Christopherson wrote:
> On Wed, Aug 31, 2022, Maxim Levitsky wrote:
> > On Wed, 2022-08-31 at 00:35 +0000, Sean Christopherson wrote:
> > > +static void avic_kick_vcpu_by_logical_id(struct kvm *kvm, u32 *avic_logical_id_table,
> > > + u32 logid_index, u32 icrl)
> > > +{
> > > + u32 physical_id;
> > > +
> > > + if (!avic_logical_id_table) {
> > ^ Typo, the '!' shoudn't be there.
>
> Ouch. I suspect the tests pass because this just ends up routing events through
> the slow path. I try to concoct a testcase to expose this bug.
>
> > > +static bool is_optimized_logical_map_enabled(struct kvm *kvm)
> > > +{
> > > + struct kvm_apic_map *map;
> > > + bool enabled;
> > > +
> > > + rcu_read_lock();
> > > + map = rcu_dereference(kvm->arch.apic_map);
> > > + enabled = map && map->logical_mode != KVM_APIC_MODE_MAP_DISABLED;
> > > + rcu_read_unlock();
> > > + return enabled;
> > > +}
> >
> > This function doesn't belong to avic, it should be in common KVM code.
>
> I'll move it. I'm not expecting any additional users, but I agree it belongs
> elsewhere. Actually, might be a moot point (see below).
>
> > > @@ -394,50 +449,27 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
> > > if (unlikely(!bitmap))
> > > return 0;
> > >
> > > - if (!is_power_of_2(bitmap))
> > > - /* multiple logical destinations, use slow path */
> > > + /*
> > > + * Use the slow path if more than one bit is set in the bitmap
> > > + * and KVM's optimized logical map is disabled to avoid kicking
> > > + * a vCPU multiple times. If the optimized map is disabled, a
> > > + * vCPU _may_ have multiple bits set in its logical ID, i.e.
> > > + * may have multiple entries in the logical table.
> > > + */
> > > + if (!is_power_of_2(bitmap) &&
> > > + !is_optimized_logical_map_enabled(kvm))
> > > return -EINVAL;
> >
> > I hate to say it but there is another issue here, which I know about for a while
> > but haven't gotten yet to fix.
> >
> > The issue is that AVIC's logical to physical map can't cover all the corner cases
> > that you discovered - it only supports the sane subset: for each cluster, and for each bit
> > in the mask, it has a physical apic id - so things like logical ids with multiple bits,
> > having same logical id for multiple vcpus and so on can't work.
> >
> > In this case we need to either inhibit AVIC (I support this 100%),
>
> I like the idea of inhibiting.
>
> > or clear its logical ID map, so all logicical IPIs VM exit, and then they
> > can be emulated.
> >
> > I haven't studied it formally but the code which rebuilds the AVIC's logical ID map
> > starts at 'avic_handle_ldr_update'.
>
> I suspected there are issues here, but the new tests passed (somewhat surprisingly)
> so I stopped trying to decipher the AVIC LDR handling.
>
> Eww. And the VM-Exit trap logic is broken too. If the guest updates and disables
> its LDR, SVM returns immediately and doesn't call into common APIC code, i.e. doesn't
> recalc the optimized map. E.g. if the guest clears its LDR, the optimized map will
> be left as is and the vCPU will receive interrupts using its old LDR.
>
> case APIC_LDR:
> if (avic_handle_ldr_update(vcpu))
> return 0;
> break;
>
> Rather than handling this purely in AVIC code, what if we a key off of
> the optimized map being enabled? E.g. drop the return from avic_handle_ldr_update()
> and in the kvm_recalculate_apic_map() do:
>
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index 3b6ef36b3963..6e188010b614 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -364,6 +364,11 @@ void kvm_recalculate_apic_map(struct kvm *kvm)
> cluster[ldr] = apic;
> }
> out:
> + if (!new || new->logical_mode == KVM_APIC_MODE_MAP_DISABLED)
> + kvm_set_apicv_inhibit(kvm, APICV_INHIBIT_REASON_LOGICAL_MAP_DISABLED);
> + else
> + kvm_clear_apicv_inhibit(kvm, APICV_INHIBIT_REASON_LOGICAL_MAP_DISABLED);
> +
This looks very good, it will even work on APICv, because the 'check_apicv_inhibit_reasons'
will not return true for this new reason (APICv IPIv I think doesn't deal with logical destination at all);
Best regards,
Maxim Levitsky
> old = rcu_dereference_protected(kvm->arch.apic_map,
> lockdep_is_held(&kvm->arch.apic_map_lock));
> rcu_assign_pointer(kvm->arch.apic_map, new);
>
