The VMX-preemption timer is a feature on VMX, it counts down, from the value loaded by VM entry, in VMX nonroot operation. When the timer counts down to zero, it stops counting down and a VM exit occurs. This patchset utilize VMX preemption timer for tsc deadline timer virtualization. The VMX preemption timer is armed before the vm-entry if the tsc deadline timer is enabled. A VMExit will happen if the virtual TSC deadline timer expires. When the vCPU thread is blocked because of HLT instruction, the tsc deadline timer virtualization will be switched to use the current solution, i.e. use the timer for it. It's switched back to VMX preemption timer when the vCPU thread is unblocked. This solution replace the complex OS's hrtimer system, and also the host timer interrupt handling cost, with a preemption_timer VMexit. It fits well for some NFV usage scenario, when the vCPU is bound to a pCPU and the pCPU is isolated, or some similar scenarioes. The benefits offset the small extra work to do on each VM-entry to setup the preemption timer Signed-off-by: Yunhong Jiang <yunhong.jiang@xxxxxxxxx> Performance Evalaution: Host: [nfv@otcnfv02 ~]$ cat /proc/cpuinfo .... cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz Guest: Two vCPU with vCPU pinned to isolated pCPUs, idle=poll on guest kernel. Test tools: cyclictest [1] running 10 minutes with 1ms interval, i.e. 600000 loop in total. 1. enable_hv_timer=Y. # Histogram ...... 000003 000000 000004 000024 000005 049862 000006 470060 000007 066930 000008 009364 000009 002496 000010 001063 000011 000106 000012 000023 ...... # Total: 000599993 # Min Latencies: 00004 # Avg Latencies: 00006 2. enable_hv_timer=N. # Histogram ...... 000003 000000 000004 000000 000005 000169 000006 000804 000007 017241 000008 207048 000009 271618 000010 100960 000011 000507 000012 000562 ...... # Total: 000599994 # Min Latencies: 00005 # Avg Latencies: 00008 Changes since v3 [4]: * On kvm_arch_vcpu_load, do the set_hv_timer() again. * On vmx_arm_hv_timer(), check the vcpu->arch.hv_deadline_tsc, instead of the apic->lapic_timer.hv_timer_in_use. * Change return of vmx_set_hv_timer() from -1 to ERANGE. * Initialize the kvm_x86_ops->set_hv_timer/cancel_hv_timer() only on CONFIG_X86_64 Changes since v2 [3]: * Switch on HLT instruction instead of sched_out/sched_in. * VMX preemption timer is broken on some CPU, added the check. * Reduce the overhead to the vm-entry code path. We calculate the host deadline tsc in advance and set the vmcs exec_control earlier. * Adding the TSC scaling support. This codepath is not tested yet because still looking for a platform with TSC scaling capability. * Checking if the host delta TSC, after the multication, will be more than 32 bit, which is the width of the vmcs field. Changes since v1 [2]: * Remove the vmx_sched_out and no changes to kvm_x86_ops for it. * Remove the two expired timer checkings on each vm-entry. * Rename the hwemul_timer to hv_timer * Clear vmx_x86_ops's membership if preemption timer is not usable. * Cache cpu_preemption_timer_multi. * Keep the tracepoint with the function patch. * Other minor changes based on Paolo's review. [1] https://rt.wiki.kernel.org/index.php/Cyclictest [2] http://www.spinics.net/lists/kvm/msg132895.html [3] http://www.spinics.net/lists/kvm/msg133185.html [4] http://www.spinics.net/lists/kvm/msg133538.html Yunhong Jiang (4): Rename the vmx_pre/post_block to pi_pre/post_block Utilize the vmx preemption timer Separate the start_sw_tscdeadline Utilize the vmx preemption timer for tsc deadline timer arch/x86/include/asm/kvm_host.h | 6 ++ arch/x86/kvm/lapic.c | 122 ++++++++++++++++++++----- arch/x86/kvm/lapic.h | 5 ++ arch/x86/kvm/trace.h | 16 ++++ arch/x86/kvm/vmx.c | 192 +++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 5 ++ 6 files changed, 320 insertions(+), 26 deletions(-) -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
