Liang Li <liliang324@xxxxxxxxx> writes:
>> > Hi Vitaly,
>> >
>> > I found a case that the virtualization overhead was almost doubled
>> > when turning on Hper-v related features compared to that without any
>> > no hyper-v feature. It happens when running a 3D game in windows
>> > guest in qemu kvm environment.
>> >
>> > By investigation, I found there are a lot of IPIs triggered by guest,
>> > when turning on the hyer-v related features including stimer, for the
>> > apicv is turned off, at least two vm exits are needed for processing a
>> > single IPI.
>> >
>> >
>> > perf stat will show something like below [recorded for 5 seconds]
>> >
>> > ---------
>> >
>> > Analyze events for all VMs, all VCPUs:
>> > VM-EXIT Samples Samples% Time% Min Time Max
>> > Time Avg time
>> > EXTERNAL_INTERRUPT 471831 59.89% 68.58% 0.64us
>> > 65.42us 2.34us ( +- 0.11% )
>> > MSR_WRITE 238932 30.33% 23.07% 0.48us
>> > 41.05us 1.56us ( +- 0.14% )
>> >
>> > Total Samples:787803, Total events handled time:1611193.84us.
>> >
>> > I tried turning off hyper-v for the same workload and repeat the test,
>> > the overall virtualization overhead reduced by about of 50%:
>> >
>> > -------
>> >
>> > Analyze events for all VMs, all VCPUs:
>> >
>> > VM-EXIT Samples Samples% Time% Min Time Max
>> > Time Avg time
>> > APIC_WRITE 255152 74.43% 50.72% 0.49us
>> > 50.01us 1.42us ( +- 0.14% )
>> > EPT_MISCONFIG 39967 11.66% 40.58% 1.55us
>> > 686.05us 7.27us ( +- 0.43% )
>> > DR_ACCESS 35003 10.21% 4.64% 0.32us
>> > 40.03us 0.95us ( +- 0.32% )
>> > EXTERNAL_INTERRUPT 6622 1.93% 2.08% 0.70us
>> > 57.38us 2.25us ( +- 1.42% )
>> >
>> > Total Samples:342788, Total events handled time:715695.62us.
>> >
>> > For this scenario, hyper-v works really bad. stimer works better
>> > than hpet, but on the other hand, it relies on SynIC which has
>> > negative effects for IPI intensive workloads.
>> > Do you have any plans for improvement?
>> >
>>
>> Hey,
>>
>> the above can be caused by the fact that when 'hv-synic' is enabled, KVM
>> automatically disables APICv and this can explain the overhead and the
>> fact that you're seeing more vmexits. KVM disables APICv because SynIC's
>> 'AutoEOI' feature is incompatible with it. We can, however, tell Windows
>> to not use AutoEOI ('Recommend deprecating AutoEOI' bit) and only
>> inhibit APICv if the recommendation was ignored. This is implemented in
>> the following KVM patch series:
>> https://lore.kernel.org/kvm/20210518144339.1987982-1-vkuznets@xxxxxxxxxx/
>>
>> It will, however, require a new 'hv-something' flag to QEMU. For now, it
>> can be tested with 'hv-passthrough'.
>>
>> It would be great if you could give it a spin!
>>
>> --
>> Vitaly
>
> It's great to know that you already have a solution for this. :)
>
> By the way, is there any requirement for the version of windows or
> windows updates for the new feature to work?
AFAIR, 'Recommend deprecating AutoEOI' bit appeared in WS2012 so I'd
expect WS2008 to ignore it completely (and thus SynIC will always be
disabling APICv for it).
--
Vitaly
