Thank you for your input, Sean. I went through the SDM virtual machine extensions chapter, and some KVM patches and it helped me a lot. My understanding is: If the RDTSC existing flied in the VMCS is not set, then the rdtsc from non-root model won’t cause VM-exit. In this case, the TSC returned to non-root is the value of the physical TSC * scaling + offset, if scaling and offset are set by KVM. The TSC offset and scaling of a vCPU can be set from root-mode through KVM APIs using command KVM_VCPU_TSC_CTRL & KVM_SET_TSC_KHZ , and they are written to the vCPU’s VMCS fields. Next time, non-root mode calls rdtsc, the VMX hardware will add the offset & scaling to the physical TSC. Is my understanding correct? Of cause, I will do some testing to verify it. Thank you, Yifei > On Oct 24, 2023, at 10:33 AM, Sean Christopherson <seanjc@xxxxxxxxxx> wrote: > > On Tue, Oct 24, 2023, Yifei Ma wrote: >> Hi KVM community, >> >> I am trying to figure out how TSC is virtualized in KVM-VMX world. >> According to the kernel documentation, reading TSC register through MSR >> can be trapped into KVM and VMX. I am trying to figure out the KVM code >> handing this trap. > > Key word "can". KVM chooses not to intercept RDMSR to MSR_IA32_TSC because > hardware handles the necessary offset and scaling. KVM does still emulate reads > in kvm_get_msr_common(), e.g. if KVM is forced to emulate a RDMSR, but that's a > very, very uncommon path. > > Ditto for the RDTSC instruction, which isn't subject to MSR intercpetion bitmaps > and has a dedicated control. KVM will emulate RDTSC if KVM is already emulating, > but otherwise the guest can execute RDTSC without triggering a VM-Exit. > > Modern CPUs provide both a offset and a scaling factor for VMX guests, i.e. the > CPU itself virtualizes guest TSC. See the RDMSR and RDTSC bullet points in the > "CHANGES TO INSTRUCTION BEHAVIOR IN VMX NON-ROOT OPERATION" section of the SDM > for details. > >> In order to understand it, I have run a kernel traced by GDB, and added >> break points to the code I thought they may handle the MSR trap, e.g., >> kvm_get_msr, vmx_exec_control, etc. Then ran rdtsc from guest application, >> however, it didn’t trigger these breakpoints. I am a little lost in how >> TSC is virtualized. >> >> Two questions: >> - does the TSC MRS instructions are emulated and trapped into KVM? > > Nope, see above. > >> - if TSC is trapped, which code handles it? > > Also see above :-) > >> Any background about TSC virtualization and suggestions on tracing its >> virtualization are appreciated.
