Re: [PATCH 0/5] IPI virtualization support for VM

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 7/16/2021 5:25 PM, Wanpeng Li wrote:
On Fri, 16 Jul 2021 at 15:14, Zeng Guang <guang.zeng@xxxxxxxxx> wrote:
Current IPI process in guest VM will virtualize the writing to interrupt
command register(ICR) of the local APIC which will cause VM-exit anyway
on source vCPU. Frequent VM-exit could induce much overhead accumulated
if running IPI intensive task.

IPI virtualization as a new VT-x feature targets to eliminate VM-exits
when issuing IPI on source vCPU. It introduces a new VM-execution
control - "IPI virtualization"(bit4) in the tertiary processor-based
VM-exection controls and a new data structure - "PID-pointer table
address" and "Last PID-pointer index" referenced by the VMCS. When "IPI
virtualization" is enabled, processor emulateds following kind of writes
to APIC registers that would send IPIs, moreover without causing VM-exits.
- Memory-mapped ICR writes
- MSR-mapped ICR writes
- SENDUIPI execution

This patch series implement IPI virtualization support in KVM.

Patches 1-3 add tertiary processor-based VM-execution support
framework.

Patch 4 implement interrupt dispatch support in x2APIC mode with
APIC-write VM exit. In previous platform, no CPU would produce
APIC-write VM exit with exit qulification 300H when the "virtual x2APIC
mode" VM-execution control was 1.

Patch 5 implement IPI virtualization related function including
feature enabling through tertiary processor-based VM-execution in
various scenario of VMCS configuration, PID table setup in vCPU creation
and vCPU block consideration.

Document for IPI virtualization is now available at the latest "Intel
Architecture Instruction Set Extensions Programming Reference".

Document Link:
https://software.intel.com/content/www/us/en/develop/download/intel-architecture-instruction-set-extensions-programming-reference.html

We did experiment to measure average time sending IPI from source vCPU
to the target vCPU completing the IPI handling by kvm unittest w/ and
w/o IPI virtualization. When IPI virtualizatin enabled, it will reduce
22.21% and 15.98% cycles comsuming in xAPIC mode and x2APIC mode
respectly.

KMV unittest:vmexit/ipi, 2 vCPU, AP runs without halt to ensure no VM
exit impact on target vCPU.

                 Cycles of IPI
                 xAPIC mode              x2APIC mode
         test    w/o IPIv  w/ IPIv       w/o IPIv  w/ IPIv
         1       6106      4816          4265      3768
         2       6244      4656          4404      3546
         3       6165      4658          4233      3474
         4       5992      4710          4363      3430
         5       6083      4741          4215      3551
         6       6238      4904          4304      3547
         7       6164      4617          4263      3709
         8       5984      4763          4518      3779
         9       5931      4712          4645      3667
         10      5955      4530          4332      3724
         11      5897      4673          4283      3569
         12      6140      4794          4178      3598
         13      6183      4728          4363      3628
         14      5991      4994          4509      3842
         15      5866      4665          4520      3739
         16      6032      4654          4229      3701
         17      6050      4653          4185      3726
         18      6004      4792          4319      3746
         19      5961      4626          4196      3392
         20      6194      4576          4433      3760

Average cycles  6059      4713.1        4337.85   3644.8
%Reduction                -22.21%                 -15.98%
Commit a9ab13ff6e (KVM: X86: Improve latency for single target IPI
fastpath) mentioned that the whole ipi fastpath feature reduces the
latency from 4238 to 3293 around 22.3% on SKX server, why your IPIv
hardware acceleration is worse than software emulation? In addition,
please post the IPI microbenchmark score w/ and w/o the
patchset.(https://lore.kernel.org/kvm/20171219085010.4081-1-ynorov@xxxxxxxxxxxxxxxxxx),
I found that the hardware acceleration is not always outstanding.
https://lore.kernel.org/kvm/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@xxxxxxxxxxxxxx
We will check on it and get back later.  Thanks for your point.
     Wanpeng



[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux