Hi Steven,
On Tue, Feb 26, 2019 at 6:24 PM Steven Rostedt <rostedt@xxxxxxxxxxx> wrote:
>
> Hi Ceco,
>
> I was walking Yordan down the code in the kernel to show him the ideas
> I have about using kvm_exit as the trace event on the host side, when I
> noticed that it's called *after* interrupts and preemption are enabled!
>
> This totally breaks the design, as it is not a reliable event, because
> the process running the guest is likely to get an interrupt or
> preempted before recording that event, which can cause a huge
> difference.
>
> kvm_enter is fine, but the reason that kvm_exit is not, is because, in
> order to record the reason for the exit, it needs to call into either
> the Intel or AMD code (as they are different), and that is done after
> interrupts and preemption are enabled.
>
> Instead, we can trace the function kvm_read_l1_tsc, which happens while
> interrupts are still enabled.
I changed the implementation to track kvm_read_l1_tsc() instead of
kvm_exit() and tested on my machine. The results were very odd, I
cannot see any traces of kvm_read_l1_tsc() called by
vcpu_enter_guest(), there are only bunch of calls like that:
CPU 1/KVM-5555 [005] .... 4530.180230: kvm_read_l1_tsc
<-vmx_set_hv_timer
CPU 1/KVM-5555 [005] d... 4530.180238: kvm_read_l1_tsc <-start_sw_timer
CPU 0/KVM-5554 [007] d... 4530.183991: kvm_read_l1_tsc
<-wait_lapic_expire
With the help of Slavi, we found that kvm_read_l1_tsc() is compiled as
inline on my kernel (4.20.11, shipped by Fedora), in this specific
flow. Looks like a gcc optimization ? We tested with
vmx_read_l1_tsc_offset(), instead of kvm_read_l1_tsc() - it appears
as expected in the traces, and the results are much more accurate
than the case when kvm_exit() is used. The problem is that
vmx_read_l1_tsc_offset() is Intel specific, most probably it is not
used on non-Intel CPUs.
>
> Would you be able to test with that function instead of using kvm_exit?
> Also, if you still see a drift, try setting both trace clocks to
> x86-tsc, and see if you still get a drift.
>
> Thanks!
>
> -- Steve
--
Tzvetomir (Ceco) Stoyanov
VMware Open Source Technology Center
|