On 2019-05-15 6:07 p.m., Wanpeng Li wrote:
On Thu, 16 May 2019 at 02:42, Ankur Arora <ankur.a.arora@xxxxxxxxxx> wrote:
On 5/14/19 6:50 AM, Marcelo Tosatti wrote:
On Mon, May 13, 2019 at 05:20:37PM +0800, Wanpeng Li wrote:
On Wed, 8 May 2019 at 02:57, Marcelo Tosatti <mtosatti@xxxxxxxxxx> wrote:
Certain workloads perform poorly on KVM compared to baremetal
due to baremetal's ability to perform mwait on NEED_RESCHED
bit of task flags (therefore skipping the IPI).
KVM supports expose mwait to the guest, if it can solve this?
Regards,
Wanpeng Li
Unfortunately mwait in guest is not feasible (uncompatible with multiple
guests). Checking whether a paravirt solution is possible.
Hi Marcelo,
I was also looking at making MWAIT available to guests in a safe manner:
whether through emulation or a PV-MWAIT. My (unsolicited) thoughts
MWAIT emulation is not simple, here is a research
https://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/mwait.html
Agreed. I had outlined my attempt to do that below and come
to the conclusion that we would need a PV-MWAIT.
Ankur
Regards,
Wanpeng Li
follow.
We basically want to handle this sequence:
monitor(monitor_address);
if (*monitor_address == base_value)
mwaitx(max_delay);
Emulation seems problematic because, AFAICS this would happen:
guest hypervisor
===== ====
monitor(monitor_address);
vmexit ===> monitor(monitor_address)
if (*monitor_address == base_value)
mwait();
vmexit ====> mwait()
There's a context switch back to the guest in this sequence which seems
problematic. Both the AMD and Intel specs list system calls and
far calls as events which would lead to the MWAIT being woken up:
"Voluntary transitions due to fast system call and far calls (occurring
prior to issuing MWAIT but after setting the monitor)".
We could do this instead:
guest hypervisor
===== ====
monitor(monitor_address);
vmexit ===> cache monitor_address
if (*monitor_address == base_value)
mwait();
vmexit ====> monitor(monitor_address)
mwait()
But, this would miss the "if (*monitor_address == base_value)" check in
the host which is problematic if *monitor_address changed simultaneously
when monitor was executed.
(Similar problem if we cache both the monitor_address and
*monitor_address.)
So, AFAICS, the only thing that would work is the guest offloading the
whole PV-MWAIT operation.
AFAICS, that could be a paravirt operation which needs three parameters:
(monitor_address, base_value, max_delay.)
This would allow the guest to offload this whole operation to
the host:
monitor(monitor_address);
if (*monitor_address == base_value)
mwaitx(max_delay);
I'm guessing you are thinking on similar lines?
High level semantics: If the CPU doesn't have any runnable threads, then
we actually do this version of PV-MWAIT -- arming a timer if necessary
so we only sleep until the time-slice expires or the MWAIT max_delay does.
If the CPU has any runnable threads then this could still finish its
time-quanta or we could just do a schedule-out.
So the semantics guaranteed to the host would be that PV-MWAIT returns
after >= max_delay OR with the *monitor_address changed.
Ankur
