The VMX-preemption timer is a feature on VMX, it counts down, from the value loaded by VM entry, in VMX nonroot operation. When the timer counts down to zero, it stops counting down and a VM exit occurs. This patchset utilize VMX preemption timer for tsc deadline timer virtualization. The VMX preemption timer is armed before the vm-entry if the tsc deadline timer is enabled. A VMExit will happen if the virtual TSC deadline timer expires. When the vCPU thread is blocked because of HLT instruction, the tsc deadline timer virtualization will be switched to use the current solution, i.e. use the timer for it. It's switched back to VMX preemption timer when the vCPU thread is unblocked. This solution replace the complex OS's hrtimer system, and also the host timer interrupt handling cost, with a preemption_timer VMexit. It fits well for some NFV usage scenario, when the vCPU is bound to a pCPU and the pCPU is isolated, or some similar scenarioes. It adds a little bit latency for each VM-entry because we need setup the preemption timer each time. Signed-off-by: Yunhong Jiang <yunhong.jiang@xxxxxxxxx> Performance Evalaution: Host: [nfv@otcnfv02 ~]$ cat /proc/cpuinfo .... cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz Guest: Two vCPU with vCPU pinned to isolated pCPUs, idle=poll on guest kernel. When the vCPU is not pinned, the benefit is smaller than pinned situation. Test tools: cyclictest [1] running 10 minutes with 1ms interval, i.e. 600000 loop in total. 1. enable_hv_timer=Y. # Histogram ...... 000003 000000 000004 002174 000005 042961 000006 479383 000007 071123 000008 003720 000009 000467 000010 000078 000011 000011 000012 000009 ...... # Min Latencies: 00004 # Avg Latencies: 00007 2. enable_hv_timer=N. # Histogram ...... 000003 000000 000004 000000 000005 000042 000006 000772 000007 008262 000008 200759 000009 381126 000010 008056 000011 000234 000012 000367 ...... # Min Latencies: 00005 # Avg Latencies: 00010 Changes since v2 [3]: * Switch on HLT instruction instead of sched_out/sched_in. * VMX preemption timer is broken on some CPU, added the check. * Reduce the overhead to the vm-entry code path. We calculate the host deadline tsc in advance and set the vmcs exec_control earlier. * Adding the TSC scaling support. This codepath is not tested yet because still looking for a platform with TSC scaling capability. * Checking if the host delta TSC, after the multication, will be more than 32 bit, which is the width of the vmcs field. Changes since v1 [2]: * Remove the vmx_sched_out and no changes to kvm_x86_ops for it. * Remove the two expired timer checkings on each vm-entry. * Rename the hwemul_timer to hv_timer * Clear vmx_x86_ops's membership if preemption timer is not usable. * Cache cpu_preemption_timer_multi. * Keep the tracepoint with the function patch. * Other minor changes based on Paolo's review. [1] https://rt.wiki.kernel.org/index.php/Cyclictest [2] http://www.spinics.net/lists/kvm/msg132895.html [3] http://www.spinics.net/lists/kvm/msg133185.html Yunhong Jiang (5): Rename the vmx_pre/post_block to pi_pre/post_block Add function for left shift and 64 bit division Utilize the vmx preemption timer Separate the start_sw_tscdeadline Utilize the vmx preemption timer for tsc deadline timer arch/x86/include/asm/div64.h | 18 +++++ arch/x86/include/asm/kvm_host.h | 8 ++ arch/x86/kvm/lapic.c | 121 ++++++++++++++++++++++------ arch/x86/kvm/lapic.h | 5 ++ arch/x86/kvm/trace.h | 16 ++++ arch/x86/kvm/vmx.c | 171 +++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 17 +++- 7 files changed, 329 insertions(+), 27 deletions(-) -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
