On Wed, Mar 01, 2023 at 12:49:06AM +0000,
"Huang, Kai" <kai.huang@xxxxxxxxx> wrote:
> On Tue, 2023-02-28 at 16:35 -0800, Isaku Yamahata wrote:
> > On Tue, Feb 28, 2023 at 09:49:10PM +0000,
> > "Huang, Kai" <kai.huang@xxxxxxxxx> wrote:
> >
> > > On Tue, 2023-02-28 at 12:18 -0800, Isaku Yamahata wrote:
> > > > On Tue, Feb 28, 2023 at 11:52:59AM +0000,
> > > > "Huang, Kai" <kai.huang@xxxxxxxxx> wrote:
> > > >
> > > > > On Tue, 2023-02-28 at 03:06 -0800, Isaku Yamahata wrote:
> > > > > > > > + if (!e)
> > > > > > > > + return -ENOMEM;
> > > > > > > > + *e = (struct kvm_cpuid_entry2) {
> > > > > > > > + .function = 1, /* Features for X2APIC */
> > > > > > > > + .index = 0,
> > > > > > > > + .eax = 0,
> > > > > > > > + .ebx = 0,
> > > > > > > > + .ecx = 1ULL << 21, /* X2APIC */
> > > > > > > > + .edx = 0,
> > > > > > > > + };
> > > > > > > > + vcpu->arch.cpuid_entries = e;
> > > > > > > > + vcpu->arch.cpuid_nent = 1;
> > > > > > >
> > > > > > > As mentioned above, why doing it here? Won't be this be overwritten later in
> > > > > > > KVM_SET_CPUID2?
> > > > > >
> > > > > > Yes, user space VMM can overwrite cpuid[0x1] and APIC base MSR. But it
> > > > > > doesn't
> > > > > > matter because it's a bug of user space VMM. user space VMM has to keep the
> > > > > > consistency of cpuid and MSRs.
> > > > > > Because TDX module virtualizes cpuid[0x1].x2apic to fixed 1, KVM value doesn't
> > > > > > matter after vcpu creation.
> > > > > > Because KVM virtualizes APIC base as read only to guest, cpuid[0x1].x2apic
> > > > > > doesn't matter after vcpu creation as long as user space VMM keeps KVM APIC
> > > > > > BASE
> > > > > > value.
> > > > > >
> > > > >
> > > > > Contrary, can we depend on userspace VMM to set x2APIC in CPUID, but not do this
> > > > > in KVM? If userspace doesn't do it, we treat it as userspace's bug.
> > > > >
> > > > > Plus, userspace anyway needs to set x2APIC in CPUID regardless whether you have
> > > > > done above here, correct?
> > > > >
> > > > > I don't see the point of doing above in KVM because you are neither enforcing
> > > > > anything in KVM, nor you are reducing effort of userspace.
> > > >
> > > > Good idea. I can drop cpuid part from tdx_vcpu_create() and apic base part from
> > > > tdx_vcpu_reset(). It needs to modify tdx_has_emulated_msr() to allow user space
> > > > VMM to update APIC BASE MSR.
> > >
> > > My personal preference would be:
> > >
> > > 1) In KVM_SET_CPUID2, we do sanity check of CPUIDs provided by userspace, and
> > > return error if not met (i.e X2APIC isn't advertised). We already have cases
> > > that KVM_SET_CPUID2 can fail, so extending to do TDX-specific check seems
> > > reasonable to me too.
> >
> > This is moot. The current check does only check maxphys address bit size and
> > specified xfeatures are supported by host. It's bare minimum for kvm to work.
> > It doesn't try to check consistency.
> >
> >
> > > 2) For APIC_BASE, you can just initialize the MSR in tdx_vcpu_reset() and ignore
> > > any update (+pr_warn()?) to MSR_IA32_APIC_BASE.
> >
> > The x86 common code for KVM_CREATE_VCPU, kvm_arch_vcpu_create(), calls vcpu_create,
> > creates lapic, and calls vcpu_reset().
> >
> > Setting ACPI BASE MSR with X2APIC enabled, checks if cpuid x2apic bit is set.
> > Please notice guest_cpuid_has(vcpu, X86_FEATURE_X2APIC) in kvm_set_apic_base().
> > To work around it, one way is set cpuid artificially in create method as this
> > patch does. Other way would be to introduce another version of
> > kvm_set_apic_base() that doesn't check cpuid dedicated for this purpose.
> > The third option is to make it user space responsibility to set initial reset
> > value of APIC BASE MSR.
> >
> > Which option do you prefer?
> >
>
> I just recall you have already set all CPUIDs via tdx_td_init(). I would do
> below:
>
> 1) keep all CPUIDs in tdx_td_init(), and make vcpu->cpuid point to that.
> 2) Ignore KVM_SET_CPUID2 for TDX guest (+ pr_warn(), etc).
> 3) Set TDX-fixed CPU registers/msrs, etc in reset_vcpu().
Finally I come up with the following flow.
- KVM_CREATE_VCPU
- tdx_vcpu_create()
no cpuid, no msr operation
- KVM_SET_CPUID2
user space has to set cpuid...x2apic=1
- KVM_TDX_INIT_VCPU
tdx_vcpu_ioctl() sets APIC_BASE MSR to x2apic enabled.
Here if user space VMM doesn't set cpuid properly, it results in error.
ACPI_BASE MSR is read only for both user space VMM and guest TD.
cpuid:
After KVM_TDX_INIT_VCPU, user space VMM can update it by KVM_SET_CPUID2.
KVM doesn't care. Guest TD see cpuid.x2apic value virtualized by TDX module
while KVM internally doesn't use the value because APIC_BASE won't change.
Thanks,
--
Isaku Yamahata <isaku.yamahata@xxxxxxxxx>
