On Thu, 21 Oct 2021 at 11:05, zhenwei pi <pizhenwei@xxxxxxxxxxxxx> wrote: > > > On 10/21/21 4:12 AM, Sean Christopherson wrote: > > On Wed, Oct 20, 2021, Wanpeng Li wrote: > >> On Wed, 20 Oct 2021 at 20:08, zhenwei pi <pizhenwei@xxxxxxxxxxxxx> wrote: > >>> > >>> Although host side exposes KVM PV SEND IPI feature to guest side, > >>> guest should still have a chance to disable it. > >>> > >>> A typicall case of this parameter: > >>> If the host AMD server enables AVIC feature, the flat mode of APIC > >>> get better performance in the guest. > >> > >> Hmm, I didn't find enough valuable information in your posting. We > >> observe AMD a lot before. > >> https://lore.kernel.org/all/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@xxxxxxxxxxxxxx/T/#u > > > > I too would like to see numbers. I suspect the answer is going to be that > > AVIC performs poorly in CPU overcommit scenarios because of the cost of managing > > the tables and handling "failed delivery" exits, but that AVIC does quite well > > when vCPUs are pinned 1:1 and IPIs rarely require an exit to the host. > > > > Test env: > CPU: AMD EPYC 7642 48-Core Processor > > Kmod args(enable avic and disable nested): > modprobe kvm-amd nested=0 avic=1 npt=1 > > QEMU args(disable x2apic): > ... -cpu host,x2apic=off ... > > Benchmark tool: > https://github.com/bytedance/kvm-utils/tree/master/microbenchmark/apic-ipi > > ~# insmod apic_ipi.ko options=5 && dmesg -c > > apic_ipi: 1 NUMA node(s) > apic_ipi: apic [flat] > apic_ipi: apic->send_IPI[default_send_IPI_single+0x0/0x40] > apic_ipi: apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] > apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 375671259, avg 3756 > apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 221961822, avg 2219 > > > apic->send_IPI_mask[kvm_send_ipi_mask+0x0/0x10] > -> This line show current send_IPI_mask is kvm_send_ipi_mask(because > of PV SEND IPI FEATURE) > > apic_ipi: IPI[kvm_send_ipi_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 375671259, avg 3756 > -->These lines show the average cycles of each kvm_send_ipi_mask: 3756 > > apic_ipi: IPI[flat_send_IPI_mask] from CPU[0] to CPU[1] > apic_ipi: total cycles 221961822, avg 2219 > -->These lines show the average cycles of each flat_send_IPI_mask: 2219 Just single target IPI is not eough. Wanpeng
