On 10/21/21 4:12 AM, Sean Christopherson wrote:
On Wed, Oct 20, 2021, Wanpeng Li wrote:
On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@xxxxxxxxxxxxx> wrote:
Although host side exposes KVM PV SEND IPI feature to guest side,
guest should still have a chance to disable it.
A typicall case of this parameter:
If the host AMD server enables AVIC feature, the flat mode of APIC
get better performance in the guest.
Hmm, I didn't find enough valuable information in your posting. We
observe AMD a lot before.
https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@xxxxxxxxxxxxxx/T/#u
I too would like to see numbers. I suspect the answer is going to be that
AVIC performs poorly in CPU overcommit scenarios because of the cost of managing
the tables and handling "failed delivery" exits, but that AVIC does quite well
when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host.
Test env:
CPU: AMD EPYC 7642 48-Core Processor
Kmod args(enable avic and disable nested):
modprobe kvm-amd nested=0 avic=1 npt=1
QEMU args(disable x2apic):
... -cpu host,x2apic=off ...
Benchmark tool:
https://github.com/bytedance/kvm-utils/tree/master/microbenchmark/apic-ipi
~# insmod apic_ipi.ko options=5 && dmesg -c
apic_ipi: 1 NUMA node(s)
apic_ipi: apic [flat]
apic_ipi: apic->send_IPI[default_send_IPI_single+0x0/0x40]
apic_ipi: apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10]
apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1]
apic_ipi: total cycles 375671259, avg 3756
apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1]
apic_ipi: total cycles 221961822, avg 2219
apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10]
-> This line show current send_IPI_mask is kvm_send_ipi_mask(because
of PV SEND IPI FEATURE)
apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1]
apic_ipi: total cycles 375671259, avg 3756
-->These lines show the average cycles of each kvm_send_ipi_mask: 3756
apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1]
apic_ipi: total cycles 221961822, avg 2219
-->These lines show the average cycles of each flat_send_IPI_mask: 2219
--
zhenwei pi
