On Tue, 18 Jan 2022 17:34:09 +0100
Vitaly Kuznetsov <vkuznets@xxxxxxxxxx> wrote:
> Igor Mammedov <imammedo@xxxxxxxxxx> writes:
>
> > On Mon, 17 Jan 2022 16:05:38 +0100
> > Vitaly Kuznetsov <vkuznets@xxxxxxxxxx> wrote:
> >
> >> Changes since v1:
> >> - Drop the allowlist of items which were allowed to change and just allow
> >> the exact same CPUID data [Sean, Paolo]. Adjust selftest accordingly.
> >> - Drop PATCH1 as the exact same change got merged upstream.
> >>
> >> Recently, KVM made it illegal to change CPUID after KVM_RUN but
> >> unfortunately this change is not fully compatible with existing VMMs.
> >> In particular, QEMU reuses vCPU fds for CPU hotplug after unplug and it
> >> calls KVM_SET_CPUID2. Relax the requirement by implementing an allowing
> >> KVM_SET_CPUID{,2} with the exact same data.
> >
> >
> > Can you check following scenario:
> > * on host that has IA32_TSX_CTRL and TSX enabled (RTM/HLE cpuid bits present)
> > * boot 2 vcpus VM with TSX enabled on VMM side but with tsx=off on kernel CLI
> >
> > that should cause kernel to set MSR_IA32_TSX_CTRL to 3H from initial 0H
> > and clear RTM+HLE bits in CPUID, check that RTM/HLE cpuid it
> > cleared
>
> Forgive me my ignorance around (not only) TSX :-) I took a "Intel(R)
> Xeon(R) CPU E3-1270 v5 @ 3.60GHz" host which seems to have rtm/hle and
> booted a guest with 'cpu=host' and with (and without) 'tsx=off' on the
> kernel command line. I decided to check what's is MSR_IA32_TSX_CTRL but
> I see the following:
>
> # rdmsr 0x122
> rdmsr: CPU 0 cannot read MSR 0x00000122
>
> I tried adding 'tsx_ctrl' to my QEMU command line but it complains with
> qemu-system-x86_64: warning: host doesn't support requested feature: MSR(10AH).tsx-ctrl [bit 7]
>
> so I think my host is not good enough :-(
I've seen it being available on "COOPER LAKE" Xeon
>
> Also, I've looked at tsx_clear_cpuid() but it actually writes to
> MSR_TSX_FORCE_ABORT MSR (0x10F), not MSR_IA32_TSX_CTRL so I'm confused.
look at tsx_disable()
> > * hotunplug a VCPU and then replug it again
> > if IA32_TSX_CTRL is reset to initial state, that should re-enable
> > RTM/HLE cpuid bits and KVM_SET_CPUID2 might fail due to difference
>
> Could you please teach me this kung-fu, I mean hot to unplug a
> cold-plugged CPU with QMP? Previoulsy, I only did un-plugging for what
> I've hotplugged, something like:
>
> (QEMU) device_add driver=host-x86_64-cpu socket-id=0 core-id=2 thread-id=0 id=cpu2
> {"return": {}}
> (QEMU) device_del id=cpu2
> {"return": {}}
>
> What's the ids of the cold-plugged CPUs?
it doesn't have to be coldplugged, hot(plug/unplug/plug) sequence is fine as well.
fyi you can use qom_path with device _del from 'info hotpluggable-cpus' output
> > and as Sean pointed out there might be other non constant leafs,
> > where exact match check could leave userspace broken.
>
> Indeed, while testing your suggestion I've stumbled upon
> CPUID.(EAX=0x12, ECX=1) (SGX) where we mangle ECX from
> kvm_vcpu_after_set_cpuid():
>
> best = kvm_find_cpuid_entry(vcpu, 0x12, 0x1);
> if (best) {
> best->ecx &= vcpu->arch.guest_supported_xcr0 & 0xffffffff;
> best->edx &= vcpu->arch.guest_supported_xcr0 >> 32;
> best->ecx |= XFEATURE_MASK_FPSSE;
> }
>
> In theory, we should just move this to __kvm_update_cpuid_runtime()...
> I'll take a look tomorrow.
>
