RE: The vcpu won't be wakened for a long time

["Longpeng (Mike, Cloud Infrastructure Service Product Dept.)" <longpeng2@xxxxxxxxxx>]

Hi Sean,

> -----Original Message-----
> From: Sean Christopherson [mailto:seanjc@xxxxxxxxxx]
> Sent: Wednesday, December 15, 2021 1:36 AM
> To: Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
> <longpeng2@xxxxxxxxxx>
> Cc: pbonzini@xxxxxxxxxx; kvm@xxxxxxxxxxxxxxx; Gonglei (Arei)
> <arei.gonglei@xxxxxxxxxx>; Huangzhichao <huangzhichao@xxxxxxxxxx>; Wanpeng Li
> <wanpengli@xxxxxxxxxxx>; Vitaly Kuznetsov <vkuznets@xxxxxxxxxx>; Jim Mattson
> <jmattson@xxxxxxxxxx>; Joerg Roedel <joro@xxxxxxxxxx>; linux-kernel
> <linux-kernel@xxxxxxxxxxxxxxx>
> Subject: Re: The vcpu won't be wakened for a long time
> 
> On Tue, Dec 14, 2021, Longpeng (Mike, Cloud Infrastructure Service Product Dept.)
> wrote:
> > Hi guys,
> >
> > We find a problem in kvm_vcpu_block().
> >
> > The testcase is:
> >  - VM configured with 1 vcpu and 1 VF (using vfio-pci passthrough)
> >  - the vfio interrupt and the vcpu are bound to the same pcpu
> >  - using remapped mode IRTE, NOT posted mode
> 
> What exactly is configured to force remapped mode?
> 

It's a misconfigure in one of our test machines.

> > The bug was triggered when the vcpu executed HLT instruction:
> >
> > kvm_vcpu_block:
> >     prepare_to_rcuwait(&vcpu->wait);
> >     for (;;) {
> >         set_current_state(TASK_INTERRUPTIBLE);
> >
> >         if (kvm_vcpu_check_block(vcpu) < 0)
> >             break;
> > 					<------------ (*)
> >         waited = true;
> >         schedule();
> >     }
> >     finish_rcuwait(&vcpu->wait);
> >
> > The vcpu will go to sleep even if an interrupt from the VF is fired at (*)
> and
> > the PIR and ON bit will be set ( in vmx_deliver_posted_interrupt ), so the
> vcpu
> > won't be wakened by subsequent interrupts.
> >
> > Any suggestions ? Thanks.
> 
> What kernel version?  There have been a variety of fixes/changes in the area
> in
> recent kernels.

The kernel version is 4.18, and it seems the latest kernel also has this problem.

The following code can fixes this bug, I've tested it on 4.18.

(4.18)

@@ -3944,6 +3944,11 @@ static void vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)
        if (pi_test_and_set_on(&vmx->pi_desc))
                return;
 
+       if (swq_has_sleeper(kvm_arch_vcpu_wq(vcpu))) {
+               kvm_vcpu_kick(vcpu);
+               return;
+       }
+
        if (vcpu != kvm_get_running_vcpu() &&
                !kvm_vcpu_trigger_posted_interrupt(vcpu, false))
                kvm_vcpu_kick(vcpu);


(latest)

@@ -3959,6 +3959,11 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)
        if (pi_test_and_set_on(&vmx->pi_desc))
                return 0;
 
+       if (rcuwait_active(&vcpu->wait)) {
+               kvm_vcpu_kick(vcpu);
+               return 0;
+       }
+
        if (vcpu != kvm_get_running_vcpu() &&
            !kvm_vcpu_trigger_posted_interrupt(vcpu, false))
                kvm_vcpu_kick(vcpu);

Do you have any suggestions ?
Thnaks.