On Thu, 21 Nov 2019 at 11:17, Wanpeng Li <kernellwp@xxxxxxxxx> wrote: > > From: Wanpeng Li <wanpengli@xxxxxxxxxxx> > > ICR and TSCDEADLINE MSRs write cause the main MSRs write vmexits in our > product observation, multicast IPIs are not as common as unicast IPI like > RESCHEDULE_VECTOR and CALL_FUNCTION_SINGLE_VECTOR etc. > > This patch introduce a mechanism to handle certain performance-critical > WRMSRs in a very early stage of KVM VMExit handler. > > This mechanism is specifically used for accelerating writes to x2APIC ICR > that attempt to send a virtual IPI with physical destination-mode, fixed > delivery-mode and single target. Which was found as one of the main causes > of VMExits for Linux workloads. > > The reason this mechanism significantly reduce the latency of such virtual > IPIs is by sending the physical IPI to the target vCPU in a very early stage > of KVM VMExit handler, before host interrupts are enabled and before expensive > operations such as reacquiring KVM’s SRCU lock. > Latency is reduced even more when KVM is able to use APICv posted-interrupt > mechanism (which allows to deliver the virtual IPI directly to target vCPU > without the need to kick it to host). > > Testing on Xeon Skylake server: > > The virtual IPI latency from sender send to receiver receive reduces > more than 200+ cpu cycles. Testing by IPI microbenchmark(https://lkml.org/lkml/2017/12/19/141): Normal IPI: Improved 3% Broadcast IPI: Improved 5% w/ --overcommit cpu-pm=on Normal IPI: Improved 14% Broadcast IPI: Improved 3.6% Wanpeng
