Resurrecting Oliver's series to quirk nVMX's manipulation of the VMX MSRs. KVM has a quirk where nVMX overwrites select VMX MSR bits in response to CPUID updates. Specifically, KVM forces CR{0,4}_FIXED1 bits and the VM-Entry/VM-Exit control bits for BNDCFGS and PERF_GLOBAL_CTRL to align with the guest vCPU model. Add a quirk to (a) allow userspace to opt out of the existing behavior and (b) make it clear the the existing behavior should not be propagated to new features. Patches 0-4 are tangentially related fixes for correctly handling CR4 checks on nested VMXON and VM-Enter. They are included here because there is a subtle dependency created by the fix in patch 02, as it changes the resulting behavior of patch 10, "Extend VMX MSRs quirk to CR0/4 fixed1 bits". Patch 05 fixes a bug where KVM forces incoming VMX MSR values to be a subset of _current_ value, not of KVM's support valu. E.g. if userspace clears an allowed-1 bit, it can never set that bit back to the original value. Patch 06, "Keep KVM updates to BNDCFGS ctrl bits across MSR write", fixes a related bug where KVM's original quirky behavior kept the VMX MSRs up-to-date (almost) all the time. This series is technically based on my selftests overhaul[*], but practically speaking that only affects the selftests. The KVM should apply cleanly on kvm/queue, 55371f1d0c01 ("KVM: x86/pmu: Update global ...") I have a KUT test for patch 3 (VMXON fixes) that I'l post separately (yet more cleanup involved, ugh). I spot tested patch 2 by fudging PKU in guest CPUID, but I'm not planning on submitting an official test anywhere (though it could be done without too much pain in selftests). v5: - Rebase (see above). - Fix "CR4 valid for nVMX" bugs. - Modify PERF_GLOBAL_CTRL bits iff the MSR exists. - Fix a bug where KVM doesn't allow userspace to restore VMX MSRs to _KVM's_ allowed values. - Fix the UMIP emulation goof. - Extend the quirk to CR0/4_FIXED1 MSRs. - Add a helper to identify if the vCPU has a vPMU. - Rewrote the selftests to more exhaustively test combos, and to test the aforementioned bugs fixed in v5. v4: - https://lore.kernel.org/all/20220301060351.442881-1-oupton@xxxxxxxxxx - Rebased to kvm/queue. Avoids conflicts with new CAPs and commit 0bcd556e15f9 ("KVM: nVMX: Refactor PMU refresh to avoid referencing kvm_x86_ops.pmu_ops") on kvm/queue. - Grabbed KVM_CAP_DISABLE_QUIRKS2 patch, since this series also introduces a quirk. - Fix typo in KVM_CAP_DISABLE_QUIRKS2 documentation (Sean) - Eliminated the need to refresh 'load IA32_PGC' bits from PMU refresh. - Use consistent formatting to make test cases more easily readable (David Dunn) - Use correct 'Fixes: ' tag and correct a typo in Patch 2 changelog. Oliver Upton (4): KVM: nVMX: Keep KVM updates to BNDCFGS ctrl bits across MSR write KVM: nVMX: Keep KVM updates to PERF_GLOBAL_CTRL ctrl bits across MSR write KVM: nVMX: Drop nested_vmx_pmu_refresh() KVM: nVMX: Add a quirk for KVM tweaks to VMX MSRs Sean Christopherson (11): KVM: x86: Split kvm_is_valid_cr4() and export only the non-vendor bits KVM: nVMX: Account for KVM reserved CR4 bits in consistency checks KVM: nVMX: Inject #UD if VMXON is attempted with incompatible CR0/CR4 KVM: nVMX: Rename handle_vm{on,off}() to handle_vmx{on,off}() KVM: nVMX: Let userspace set nVMX MSR to any _host_ supported value KVM: VMX: Add helper to check if the guest PMU has PERF_GLOBAL_CTRL KVM: nVMX: Set UMIP bit CR4_FIXED1 MSR when emulating UMIP KVM: nVMX: Extend VMX MSRs quirk to CR0/4 fixed1 bits KVM: selftests: Add test to verify KVM's VMX MSRs quirk for controls KVM: selftests: Extend VMX MSRs test to cover CR4_FIXED1 (and its quirks) KVM: selftests: Verify VMX MSRs can be restored to KVM-supported values Documentation/virt/kvm/api.rst | 29 ++ arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/svm/nested.c | 3 +- arch/x86/kvm/vmx/nested.c | 199 ++++++------ arch/x86/kvm/vmx/nested.h | 5 +- arch/x86/kvm/vmx/pmu_intel.c | 7 +- arch/x86/kvm/vmx/vmx.c | 23 +- arch/x86/kvm/vmx/vmx.h | 14 + arch/x86/kvm/x86.c | 12 +- arch/x86/kvm/x86.h | 2 +- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/processor.h | 8 + .../selftests/kvm/include/x86_64/vmx.h | 2 + .../selftests/kvm/x86_64/vmx_msrs_test.c | 287 ++++++++++++++++++ 16 files changed, 493 insertions(+), 104 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86_64/vmx_msrs_test.c base-commit: 081ad4bbae8d503c79fae45f463766d28b2f3241 -- 2.36.1.255.ge46751e96f-goog