On Tue, 2009-09-01 at 12:47 +0300, Avi Kivity wrote: > On 09/01/2009 12:44 PM, Avi Kivity wrote: > > Instead of saving the debug registers from the processor to a kvm data > > structure, rely in the debug registers stored in the thread structure. > > This allows us not to save dr6 and dr7. > > > > Reduces lightweight vmexit cost by 350 cycles, or 11 percent. > > > > Andrew, this is now available as the 'debugreg' branch of kvm.git. > Given the massive performance improvement, it will be interesting to see > how the test results change. > > Marcelo, please queue this for 2.6.32, and I think it's even suitable > for -stable. > Here's a run from branch debugreg with thread debugreg storage + conditionally reload dr6: user nice system irq softirq guest idle iowait 5.79 0.00 9.28 0.08 1.00 20.81 58.78 4.26 total busy: 36.97 Previous run that had avoided calling adjust_vmx_controls twice: user nice system irq softirq guest idle iowait 5.81 0.00 9.48 0.08 1.04 21.32 57.86 4.41 total busy: 37.73 A relative reduction CPU cycles of 2% new oprofile: > samples % app name symbol name > 876648 54.1555 kvm-intel.ko vmx_vcpu_run > 37595 2.3225 qemu-system-x86_64 cpu_physical_memory_rw > 35623 2.2006 qemu-system-x86_64 phys_page_find_alloc > 24874 1.5366 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 native_write_msr_safe > 17710 1.0940 libc-2.5.so memcpy > 14664 0.9059 kvm.ko kvm_arch_vcpu_ioctl_run > 14577 0.9005 qemu-system-x86_64 qemu_get_ram_ptr > 12528 0.7739 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 native_read_msr_safe > 10979 0.6782 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 copy_user_generic_string > 9979 0.6165 qemu-system-x86_64 virtqueue_get_head > 9371 0.5789 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 schedule > 8333 0.5148 qemu-system-x86_64 virtqueue_avail_bytes > 7899 0.4880 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 fget_light > 7289 0.4503 qemu-system-x86_64 main_loop_wait > 7217 0.4458 qemu-system-x86_64 lduw_phys > 6821 0.4214 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 audit_syscall_exit > 6749 0.4169 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 do_select > 5919 0.3657 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 audit_syscall_entry > 5466 0.3377 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 kfree > 4887 0.3019 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 fput > 4689 0.2897 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 __switch_to > 4636 0.2864 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 mwait_idle > 4505 0.2783 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 getnstimeofday > 4453 0.2751 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 system_call > 4403 0.2720 kvm.ko kvm_load_guest_fpu > 4285 0.2647 kvm.ko kvm_put_guest_fpu > 4241 0.2620 libpthread-2.5.so pthread_mutex_lock > 4172 0.2577 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 unroll_tree_refs > 4100 0.2533 qemu-system-x86_64 kvm_run > 4044 0.2498 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 __down_read > 3978 0.2457 qemu-system-x86_64 ldl_phys > 3669 0.2267 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 do_vfs_ioctl > 3655 0.2258 vmlinux-2.6.31-rc5_debugreg_v2.6.31-rc3-3441-g479fa73-autokern1 __up_read > A diff of this and previous run's oprofile: > profile1 is [./oprofile.before] > profile2 is [./oprofile.after] > Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 10000000 > total samples (ts1) for profile1 is 1661542 > total samples (ts2) for profile2 is 1618760 (includes multiplier of 1.000000) > functions which have a abs(pct2-pct1) < 0.02 are not displayed > > pct2: pct1: > 100* 100* pct2 > s1 s2 s2/s1 s2/ts1 s1/ts1 -pct1 symbol bin > --------- --------- ------- ------- ------- ------ ------ --- > 1559 2747 1.76/1 0.165 0.094 0.071 dput vmlinux > 34764 35623 1.02/1 2.144 2.092 0.052 phys_page_find_alloc qemu > 5170 5919 1.14/1 0.356 0.311 0.045 audit_syscall_entry vmlinux > 3593 4172 1.16/1 0.251 0.216 0.035 unroll_tree_refs vmlinux > 896 1414 1.58/1 0.085 0.054 0.031 kstat_irqs_cpu vmlinux > 199 676 3.40/1 0.041 0.012 0.029 /usr/lib64/perl5/5.8 libperl.so > 14189 14664 1.03/1 0.883 0.854 0.029 kvm_arch_vcpu_ioctl_ kvm.ko > 1314 1779 1.35/1 0.107 0.079 0.028 kvm_arch_post_kvm_ru qemu > 1390 1785 1.28/1 0.107 0.084 0.024 vfs_read vmlinux > 9015 9371 1.04/1 0.564 0.543 0.021 schedule vmlinux > 1265 1610 1.27/1 0.097 0.076 0.021 qemu_get_clock qemu > 600 941 1.57/1 0.057 0.036 0.021 preempt_notifier_reg vmlinux > 3339 2996 1/1.11 0.180 0.201 -0.021 virtqueue_num_heads qemu > 822 463 1/1.78 0.028 0.049 -0.022 vmx_vcpu_put kvm > 1116 743 1/1.50 0.045 0.067 -0.022 cap_file_ioctl vmlinux > 2967 2587 1/1.15 0.156 0.179 -0.023 native_read_tsc vmlinux > 3487 3096 1/1.13 0.186 0.210 -0.024 task_rq_lock vmlinux > 1780 1381 1/1.29 0.083 0.107 -0.024 math_state_restore vmlinux > 25278 24874 1/1.02 1.497 1.521 -0.024 native_write_msr_saf vmlinux > 1733 1329 1/1.30 0.080 0.104 -0.024 rw_verify_area vmlinux > 2305 1897 1/1.22 0.114 0.139 -0.025 __hrtimer_start_rang vmlinux > 1685 1273 1/1.32 0.077 0.101 -0.025 tcp_poll vmlinux > 1059 637 1/1.66 0.038 0.064 -0.025 kvm_lapic_get_cr8 kvm.ko > 2855 2425 1/1.18 0.146 0.172 -0.026 signalfd_poll vmlinux > 4100 3669 1/1.12 0.221 0.247 -0.026 do_vfs_ioctl vmlinux > 1941 1498 1/1.30 0.090 0.117 -0.027 complete_pio kvm.ko > 3506 3052 1/1.15 0.184 0.211 -0.027 vmcs_writel kvm > 18205 17710 1/1.03 1.066 1.096 -0.030 memcpy libc > 1717 1114 1/1.54 0.067 0.103 -0.036 rb_insert_color vmlinux > 10871 9979 1/1.09 0.601 0.654 -0.054 virtqueue_get_head qemu > 7805 6749 1/1.16 0.406 0.470 -0.064 do_select vmlinux > 9080 7899 1/1.15 0.475 0.546 -0.071 fget_light vmlinux > 3550 0 0.000 0.214 -0.214 native_get_debugreg vmlinux > 885444 876648 1/1.01 52.761 53.290 -0.529 vmx_vcpu_run kvm > 12380 0 0.000 0.745 -0.745 native_set_debugreg vmlinux > ------- ------- ------ > 94.916 97.469 -2.553 > Notice native_set/get_debugreg is now not an issue :) All of these still use qemu-kvm-87 -Andrew -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
