[RFC V2] LoongArch: KVM: Handle interrupt early before enabling irq

[Bibo Mao <maobibo@xxxxxxxxxxx>]

If interrupt arrive when vCPU is running, vCPU will exit because of
interrupt exception. Currently interrupt exception is handled after
local_irq_enable() is called, and it is handled by host kernel rather
than KVM hypervisor. It will introduce extra another interrupt
exception and then host will handle irq.

If KVM hypervisor detect that it is interrupt exception, interrupt
can be handle early in KVM hypervisor before local_irq_enable() is
called.

On 3C5000 dual-way machine, there will be 10% -- 15% performance
improvement with netperf UDP_RR option with 10G ethernet card.
                   original     with patch    improvement
  netperf UDP_RR     7200          8100           +12%

The total performance is low because irqchip is emulated in qemu VMM,
however from the same testbed, there is performance improvement
actually.

Signed-off-by: Bibo Mao <maobibo@xxxxxxxxxxx>
---
v1 ... v2:
  1. Move guest_timing_exit_irqoff() after host interrupt handling like
     other architectures.
  2. Construct interrupt context pt_regs from guest entering context
  3. Add cond_resched() after irq enabling
---
 arch/loongarch/kernel/traps.c |  1 +
 arch/loongarch/kvm/vcpu.c     | 36 ++++++++++++++++++++++++++++++++++-
 2 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/arch/loongarch/kernel/traps.c b/arch/loongarch/kernel/traps.c
index 2ec3106c0da3..eed0d8b02ee3 100644
--- a/arch/loongarch/kernel/traps.c
+++ b/arch/loongarch/kernel/traps.c
@@ -1114,6 +1114,7 @@ asmlinkage void noinstr do_vint(struct pt_regs *regs, unsigned long sp)
 
 	irqentry_exit(regs, state);
 }
+EXPORT_SYMBOL(do_vint);
 
 unsigned long eentry;
 unsigned long tlbrentry;
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 9e1a9b4aa4c6..bab7a71eb965 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -5,6 +5,7 @@
 
 #include <linux/kvm_host.h>
 #include <linux/entry-kvm.h>
+#include <asm/exception.h>
 #include <asm/fpu.h>
 #include <asm/lbt.h>
 #include <asm/loongarch.h>
@@ -304,6 +305,23 @@ static int kvm_pre_enter_guest(struct kvm_vcpu *vcpu)
 	return ret;
 }
 
+static void kvm_handle_irq(struct kvm_vcpu *vcpu)
+{
+	struct pt_regs regs, *old;
+
+	/*
+	 * Construct pseudo pt_regs, only necessary registers is added
+	 * Interrupt context coming from guest enter context
+	 */
+	old = (struct pt_regs *)(vcpu->arch.host_sp - sizeof(struct pt_regs));
+	/* Disable preemption in irq exit function irqentry_exit() */
+	regs.csr_prmd = 0;
+	regs.regs[LOONGARCH_GPR_SP] = vcpu->arch.host_sp;
+	regs.regs[LOONGARCH_GPR_FP] = old->regs[LOONGARCH_GPR_FP];
+	regs.csr_era = old->regs[LOONGARCH_GPR_RA];
+	do_vint(&regs, (unsigned long)&regs);
+}
+
 /*
  * Return 1 for resume guest and "<= 0" for resume host.
  */
@@ -321,8 +339,23 @@ static int kvm_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu)
 
 	kvm_lose_pmu(vcpu);
 
-	guest_timing_exit_irqoff();
 	guest_state_exit_irqoff();
+
+	/*
+	 * VM exit because of host interrupts
+	 * Handle irq directly before enabling irq
+	 */
+	if (!ecode && intr)
+		kvm_handle_irq(vcpu);
+
+	/*
+	 * Wait until after servicing IRQs to account guest time so that any
+	 * ticks that occurred while running the guest are properly accounted
+	 * to the guest. Waiting until IRQs are enabled degrades the accuracy
+	 * of accounting via context tracking, but the loss of accuracy is
+	 * acceptable for all known use cases.
+	 */
+	guest_timing_exit_irqoff();
 	local_irq_enable();
 
 	trace_kvm_exit(vcpu, ecode);
@@ -331,6 +364,7 @@ static int kvm_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu)
 	} else {
 		WARN(!intr, "vm exiting with suspicious irq\n");
 		++vcpu->stat.int_exits;
+		cond_resched();
 	}
 
 	if (ret == RESUME_GUEST)

base-commit: 80e54e84911a923c40d7bee33a34c1b4be148d7a
-- 
2.39.3