Hello Sean,
Thank you very much again. I didn't know about the tracepointing
methodology. This is very helpful, indeed.
In addition to my requirement of recording tsc values, I am trying to
track when the VMEXITs happen in userspace(QEMU). This allows me to
correlate recorded TSC values with the VMEXIT sequence. So, causing a
VMEXIT in userspace is absolutely essential to me and I would love to
have some pointers on how to do it.
I run a server within the guest and I'm okay with performance dropping
down to within a factor of 2 compared to running the server on native
hardware. I run this experiment for about 20-30 seconds.
Best Regards,
Arnabjyoti Kalita
On Mon, Apr 26, 2021 at 9:35 PM Sean Christopherson <seanjc@xxxxxxxxxx> wrote:
>
> On Mon, Apr 26, 2021, Arnabjyoti Kalita wrote:
> > Hello Sean,
> >
> > Thank you very much for your answer again. I could actually see that the
> > RDTSC handler gets called successfully. I added a "printk" statement to the
> > handler and I can see the messages being printed out to the kernel syslog.
> >
> > However, I am not sure if a VMEXIT is happening in userspace. I use QEMU
> > and I do not see any notification from QEMU that tells me that a VMEXIT
> > happened due to RDTSC being executed. Is there a mechanism to confirm this?
>
> Without further modification, KVM will _not_ exit to userspace in this case.
>
> > My actual requirement to record tsc values read out as a result of RDTSC
> > execution still stands.
>
> Your requirement didn't clarify that userspace needed to record the values ;-)
>
> Forcing an exit to userspace is very doable, but will tank guest performance and
> possibly interfere with whatever you're trying to do. I would try adding a
> tracepoint and using that to capture the TSC values. Adding guest RIP, etc...
> as needed is trivial.
>
> diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
> index a61c015870e3..e962e813ba04 100644
> --- a/arch/x86/kvm/trace.h
> +++ b/arch/x86/kvm/trace.h
> @@ -821,6 +821,23 @@ TRACE_EVENT(
> __entry->gpa_match ? "GPA" : "GVA")
> );
>
> +TRACE_EVENT(kvm_emulate_rdtsc,
> + TP_PROTO(unsigned int vcpu_id, __u64 tsc),
> + TP_ARGS(vcpu_id, tsc),
> +
> + TP_STRUCT__entry(
> + __field( unsigned int, vcpu_id )
> + __field( __u64, tsc )
> + ),
> +
> + TP_fast_assign(
> + __entry->vcpu_id = vcpu_id;
> + __entry->tsc = tsc;
> + ),
> +
> + TP_printk("vcpu=%u tsc=%llu", __entry->vcpu_id, __entry->tsc)
> +);
> +
> TRACE_EVENT(kvm_write_tsc_offset,
> TP_PROTO(unsigned int vcpu_id, __u64 previous_tsc_offset,
> __u64 next_tsc_offset),
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index 61bf0b86770b..1fbeef520349 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -5254,6 +5254,8 @@ static int handle_rdtsc(struct kvm_vcpu *vcpu)
> {
> u64 tsc = kvm_read_l1_tsc(vcpu, rdtsc());
>
> + trace_kvm_emulate_rdtsc(vcpu->vcpu_id, tsc);
> +
> kvm_rax_write(vcpu, tsc & -1u);
> kvm_rdx_write(vcpu, (tsc >> 32) & -1u);
> return kvm_skip_emulated_instruction(vcpu);
