Linus, The following changes since commit 523caed9efbb049339706b124185c9358c1b6477: x86/sgx: Mark sgx_vepc_vm_ops static (2021-04-12 19:48:32 +0200) are available in the Git repository at: https://git.kernel.org/pub/scm/virt/kvm/kvm.git tags/for-linus for you to fetch changes up to 3bf0fcd754345d7ea63e1446015ba65ece6788ca: KVM: selftests: Speed up set_memory_region_test (2021-04-26 12:21:27 -0400) It's a fairly large pull request for KVM standards. There were quite a few common topic branches and crossover series with other trees, especially within KVM ARM, which I need to mention explicitly. In particular: - the coresight/next-ETE-TRBE branch from the KVM ARM tree hasn't yet reached you, so I am CCing the maintainer. Since he sent the patches as a pull request to Marc Zyngier (the KVM ARM maintainer) at https://lore.kernel.org/kvmarm/20210406224437.330939-1-mathieu.poirier@xxxxxxxxxx/T/#u, I actually suspect that from his point of view he's done. - Herbert Xu and Tom Lendacky asked to include AMD PSP driver changes in this pull request - the KVM ARM tree also brought in a handful of "Get rid of oprofile leftovers" patches for other architectures, but they all have maintainer acks and are just cleanups, so I am a bit less worried there. I hope all this is okay. Despite the juggling of topic branches, a couple conflicts escaped: - the worst is with the tip rework of x86 TLB flushes. Basically, tip renamed kvm_flush_tlb_others to kvm_flush_tlb_multi, added a comment and moved it to a different position. KVM moved it somewhere else, so after merging you'll have both kvm_flush_tlb_multi and kvm_flush_tlb_others. You need to take kvm_flush_tlb_multi and paste it in place of kvm_flush_tlb_others. For everything else the KVM code is the correct one, again except for s/kvm_flush_tlb_others/kvm_flush_tlb_multi/. I have placed the relevant hunks of the "diff --combined" output at the end of this message. - arch/mips/kvm/trap_emul.c has been deleted in the MIPS tree, which will cause a modify/delete conflict either now or when merging the MIPS tree. The conflict is of course solved just by deleting the file. For the future, Thomas Bogendorfer will either not take KVM patches or ensure that we have a common topic branch. Thanks, Paolo ---------------------------------------------------------------- ARM: - Stage-2 isolation for the host kernel when running in protected mode - Guest SVE support when running in nVHE mode - Force W^X hypervisor mappings in nVHE mode - ITS save/restore for guests using direct injection with GICv4.1 - nVHE panics now produce readable backtraces - Guest support for PTP using the ptp_kvm driver - Performance improvements in the S2 fault handler x86: - Optimizations and cleanup of nested SVM code - AMD: Support for virtual SPEC_CTRL - Optimizations of the new MMU code: fast invalidation, zap under read lock, enable/disably dirty page logging under read lock - /dev/kvm API for AMD SEV live migration (guest API coming soon) - support SEV virtual machines sharing the same encryption context - support SGX in virtual machines - add a few more statistics - improved directed yield heuristics - Lots and lots of cleanups Generic: - Rework of MMU notifier interface, simplifying and optimizing the architecture-specific code - Some selftests improvements ---------------------------------------------------------------- Alexandru Elisei (4): Documentation: KVM: Document KVM_GUESTDBG_USE_HW control flag for arm64 KVM: arm64: Initialize VCPU mdcr_el2 before loading it KVM: arm64: Don't print warning when trapping SPE registers KVM: arm64: Don't advertise FEAT_SPE to guests Andrew Scull (5): bug: Remove redundant condition check in report_bug bug: Factor out a getter for a bug's file line bug: Assign values once in bug_get_file_line() KVM: arm64: Use BUG and BUG_ON in nVHE hyp KVM: arm64: Log source when panicking from nVHE hyp Anshuman Khandual (5): arm64: Add TRBE definitions coresight: core: Add support for dedicated percpu sinks coresight: sink: Add TRBE driver Documentation: coresight: trbe: Sysfs ABI description Documentation: trace: Add documentation for TRBE Babu Moger (2): x86/cpufeatures: Add the Virtual SPEC_CTRL feature KVM: SVM: Add support for Virtual SPEC_CTRL Ben Gardon (13): KVM: x86/mmu: Re-add const qualifier in kvm_tdp_mmu_zap_collapsible_sptes KVM: x86/mmu: Move kvm_mmu_(get|put)_root to TDP MMU KVM: x86/mmu: use tdp_mmu_free_sp to free roots KVM: x86/mmu: Merge TDP MMU put and free root KVM: x86/mmu: Refactor yield safe root iterator KVM: x86/mmu: Make TDP MMU root refcount atomic KVM: x86/mmu: handle cmpxchg failure in kvm_tdp_mmu_get_root KVM: x86/mmu: Protect the tdp_mmu_roots list with RCU KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock KVM: x86/mmu: Allow zapping collapsible SPTEs to use MMU read lock KVM: x86/mmu: Allow enabling/disabling dirty logging under MMU read lock KVM: x86/mmu: Fast invalidation for TDP MMU KVM: x86/mmu: Tear down roots before kvm_mmu_zap_all_fast returns Bhaskar Chowdhury (1): KVM: s390: Fix comment spelling in kvm_s390_vcpu_start() Brijesh Singh (6): KVM: SVM: Add KVM_SEV SEND_START command KVM: SVM: Add KVM_SEND_UPDATE_DATA command KVM: SVM: Add KVM_SEV_SEND_FINISH command KVM: SVM: Add support for KVM_SEV_RECEIVE_START command KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command Cathy Avery (4): KVM: SVM: Use a separate vmcb for the nested L2 guest KVM: nSVM: Track the physical cpu of the vmcb vmrun through the vmcb KVM: nSVM: Track the ASID generation of the vmcb vmrun through the vmcb KVM: nSVM: Optimize vmcb12 to vmcb02 save area copies Claudio Imbrenda (5): KVM: s390: split kvm_s390_logical_to_effective KVM: s390: extend kvm_s390_shadow_fault to return entry pointer KVM: s390: VSIE: correctly handle MVPG when in VSIE KVM: s390: split kvm_s390_real_to_abs KVM: s390: VSIE: fix MVPG handling for prefixing and MSO Daniel Kiss (1): KVM: arm64: Enable SVE support for nVHE David Brazdil (1): KVM: arm64: Support PREL/PLT relocs in EL2 code David Edmondson (5): KVM: x86: dump_vmcs should not assume GUEST_IA32_EFER is valid KVM: x86: dump_vmcs should not conflate EFER and PAT presence in VMCS KVM: x86: dump_vmcs should consider only the load controls of EFER/PAT KVM: x86: dump_vmcs should show the effective EFER KVM: x86: dump_vmcs should include the autoload/autostore MSR lists Dongli Zhang (1): KVM: x86: to track if L1 is running L2 VM Emanuele Giuseppe Esposito (1): doc/virt/kvm: move KVM_CAP_PPC_MULTITCE in section 8 Eric Auger (11): KVM: arm64: vgic-v3: Fix some error codes when setting RDIST base KVM: arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST_REGION read KVM: arm64: vgic-v3: Fix error handling in vgic_v3_set_redist_base() KVM: arm/arm64: vgic: Reset base address on kvm_vgic_dist_destroy() docs: kvm: devices/arm-vgic-v3: enhance KVM_DEV_ARM_VGIC_CTRL_INIT doc KVM: arm64: Simplify argument passing to vgic_uaccess_[read|write] kvm: arm64: vgic-v3: Introduce vgic_v3_free_redist_region() KVM: arm64: vgic-v3: Expose GICR_TYPER.Last for userspace KVM: selftests: aarch64/vgic-v3 init sequence tests KVM: selftests: vgic_init kvm selftests fixup KVM: arm/arm64: Fix KVM_VGIC_V3_ADDR_TYPE_REDIST read Gavin Shan (3): KVM: arm64: Hide kvm_mmu_wp_memory_region() KVM: arm64: Use find_vma_intersection() KVM: arm64: Don't retrieve memory slot again in page fault handler Haiwei Li (2): KVM: vmx: add mismatched size assertions in vmcs_check32() KVM: x86: Take advantage of kvm_arch_dy_has_pending_interrupt() Heiko Carstens (1): KVM: s390: fix guarded storage control register handling Isaku Yamahata (1): KVM: VMX: use EPT_VIOLATION_GVA_TRANSLATED instead of 0x100 Jianyong Wu (4): ptp: Reorganize ptp_kvm.c to make it arch-independent clocksource: Add clocksource id for arm arch counter KVM: arm64: Add support for the KVM PTP service ptp: arm/arm64: Enable ptp_kvm for arm/arm64 Jon Hunter (1): ptp: Don't print an error if ptp_kvm is not supported Keqian Zhu (1): KVM: x86: Remove unused function declaration Krish Sadhukhan (4): KVM: nSVM: Add missing checks for reserved bits to svm_set_nested_state() KVM: nSVM: If VMRUN is single-stepped, queue the #DB intercept in nested_svm_vmexit() KVM: SVM: Define actual size of IOPM and MSRPM tables nSVM: Check addresses of MSR and IO permission maps Marc Zyngier (47): KVM: arm64: Provide KVM's own save/restore SVE primitives KVM: arm64: Use {read,write}_sysreg_el1 to access ZCR_EL1 KVM: arm64: Let vcpu_sve_pffr() handle HYP VAs KVM: arm64: Introduce vcpu_sve_vq() helper arm64: sve: Provide a conditional update accessor for ZCR_ELx KVM: arm64: Rework SVE host-save/guest-restore KVM: arm64: Map SVE context at EL2 when available KVM: arm64: Save guest's ZCR_EL1 before saving the FPSIMD state KVM: arm64: Trap host SVE accesses when the FPSIMD state is dirty KVM: arm64: Save/restore SVE state for nVHE arm64: Use INIT_SCTLR_EL1_MMU_OFF to disable the MMU on CPU restart KVM: arm64: Use INIT_SCTLR_EL2_MMU_OFF to disable the MMU on KVM teardown KVM: arm64: Turn SCTLR_ELx_FLAGS into INIT_SCTLR_EL2_MMU_ON KVM: arm64: Force SCTLR_EL2.WXN when running nVHE KVM: arm64: Fix host's ZCR_EL2 restore on nVHE Merge tag 'v5.12-rc3' into kvm-arm64/host-stage2 irqchip/gic-v3-its: Add a cache invalidation right after vPE unmapping KVM: arm64: Generate final CTR_EL0 value when running in Protected mode KVM: arm64: Drop the CPU_FTR_REG_HYP_COPY infrastructure KVM: arm64: Elect Alexandru as a replacement for Julien as a reviewer KVM: arm64: Mark the kvmarm ML as moderated for non-subscribers KVM: arm64: Fix table format for PTP documentation Merge remote-tracking branch 'coresight/next-ETE-TRBE' into kvmarm-master/next KVM: arm64: Clarify vcpu reset behaviour KVM: arm64: Fully zero the vcpu state on reset Merge branch 'kvm-arm64/debug-5.13' into kvmarm-master/next Merge branch 'kvm-arm64/host-stage2' into kvmarm-master/next Merge branch 'kvm-arm64/memslot-fixes' into kvmarm-master/next Merge branch 'kvm-arm64/misc-5.13' into kvmarm-master/next Merge branch 'kvm-arm64/nvhe-panic-info' into kvmarm-master/next Merge branch 'kvm-arm64/nvhe-sve' into kvmarm-master/next Merge branch 'kvm-arm64/nvhe-wxn' into kvmarm-master/next Merge branch 'kvm-arm64/ptp' into kvmarm-master/next Merge branch 'kvm-arm64/vgic-5.13' into kvmarm-master/next Merge branch 'kvm-arm64/vlpi-save-restore' into kvmarm-master/next Merge remote-tracking branch 'arm64/for-next/vhe-only' into kvmarm-master/next Merge remote-tracking branch 'arm64/for-next/neon-softirqs-disabled' into kvmarm-master/next Merge remote-tracking branch 'coresight/next-ETE-TRBE' into kvmarm-master/next bug: Provide dummy version of bug_get_file_line() when !GENERIC_BUG Merge branch 'kvm-arm64/nvhe-panic-info' into kvmarm-master/next Merge branch 'kvm-arm64/ptp' into kvmarm-master/next KVM: arm64: Divorce the perf code from oprofile helpers arm64: Get rid of oprofile leftovers s390: Get rid of oprofile leftovers sh: Get rid of oprofile leftovers perf: Get rid of oprofile leftovers Merge branch 'kvm-arm64/kill_oprofile_dependency' into kvmarm-master/next Maxim Levitsky (10): KVM: nSVM: always use vmcb01 to for vmsave/vmload of guest state KVM: x86: determine if an exception has an error code only when injecting it. KVM: x86: mmu: initialize fault.async_page_fault in walk_addr_generic KVM: x86: add guest_cpuid_is_intel KVM: nSVM: improve SYSENTER emulation on AMD KVM: nSVM: call nested_svm_load_cr3 on nested state load KVM: x86: pending exceptions must not be blocked by an injected event KVM: x86: implement KVM_CAP_SET_GUEST_DEBUG2 KVM: aarch64: implement KVM_CAP_SET_GUEST_DEBUG2 KVM: s390x: implement KVM_CAP_SET_GUEST_DEBUG2 Nathan Tempelman (1): KVM: x86: Support KVM VMs sharing SEV context Paolo Bonzini (27): KVM: nSVM: rename functions and variables according to vmcbXY nomenclature KVM: nSVM: do not copy vmcb01->control blindly to vmcb02->control KVM: nSVM: do not mark all VMCB01 fields dirty on nested vmexit KVM: nSVM: do not mark all VMCB02 fields dirty on nested vmexit KVM: nSVM: only copy L1 non-VMLOAD/VMSAVE data in svm_set_nested_state() KVM: SVM: merge update_cr0_intercept into svm_set_cr0 KVM: SVM: Pass struct kvm_vcpu to exit handlers (and many, many other places) KVM: SVM: move VMLOAD/VMSAVE to C code Merge branch 'kvm-fix-svm-races' into HEAD Merge branch 'kvm-tdp-fix-flushes' into HEAD Merge branch 'kvm-tdp-fix-rcu' into HEAD Merge tag 'kvm-s390-next-5.13-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD Merge tag 'kvm-s390-next-5.13-2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD Merge remote-tracking branch 'tip/x86/sgx' into kvm-next KVM: MMU: load PDPTRs outside mmu_lock KVM: introduce KVM_CAP_SET_GUEST_DEBUG2 KVM: MMU: protect TDP MMU pages only down to required level KVM: constify kvm_arch_flush_remote_tlbs_memslot KVM: MIPS: rework flush_shadow_* callbacks into one that prepares the flush KVM: MIPS: let generic code call prepare_flush_shadow KVM: MIPS: defer flush to generic MMU notifier code KVM: selftests: Always run vCPU thread with blocked SIG_IPI KVM: x86: document behavior of measurement ioctls with len==0 Merge branch 'kvm-sev-cgroup' into HEAD Merge tag 'kvmarm-5.13' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD KVM: documentation: fix sphinx warnings KVM: SEV: Mask CPUID[0x8000001F].eax according to supported features Peter Xu (1): KVM: selftests: Sync data verify of dirty logging with guest sync Pierre Morel (1): KVM: s390: diag9c (directed yield) forwarding Quentin Perret (35): KVM: arm64: Initialize kvm_nvhe_init_params early KVM: arm64: Avoid free_page() in page-table allocator KVM: arm64: Factor memory allocation out of pgtable.c KVM: arm64: Introduce a BSS section for use at Hyp KVM: arm64: Make kvm_call_hyp() a function call at Hyp KVM: arm64: Allow using kvm_nvhe_sym() in hyp code KVM: arm64: Introduce an early Hyp page allocator KVM: arm64: Stub CONFIG_DEBUG_LIST at Hyp KVM: arm64: Introduce a Hyp buddy page allocator KVM: arm64: Enable access to sanitized CPU features at EL2 KVM: arm64: Provide __flush_dcache_area at EL2 KVM: arm64: Factor out vector address calculation arm64: asm: Provide set_sctlr_el2 macro KVM: arm64: Prepare the creation of s1 mappings at EL2 KVM: arm64: Elevate hypervisor mappings creation at EL2 KVM: arm64: Use kvm_arch for stage 2 pgtable KVM: arm64: Use kvm_arch in kvm_s2_mmu KVM: arm64: Set host stage 2 using kvm_nvhe_init_params KVM: arm64: Refactor kvm_arm_setup_stage2() KVM: arm64: Refactor __load_guest_stage2() KVM: arm64: Refactor __populate_fault_info() KVM: arm64: Make memcache anonymous in pgtable allocator KVM: arm64: Reserve memory for host stage 2 KVM: arm64: Sort the hypervisor memblocks KVM: arm64: Always zero invalid PTEs KVM: arm64: Use page-table to track page ownership KVM: arm64: Refactor the *_map_set_prot_attr() helpers KVM: arm64: Add kvm_pgtable_stage2_find_range() KVM: arm64: Introduce KVM_PGTABLE_S2_NOFWB stage 2 flag KVM: arm64: Introduce KVM_PGTABLE_S2_IDMAP stage 2 flag KVM: arm64: Provide sanitized mmfr* registers at EL2 KVM: arm64: Wrap the host with a stage 2 KVM: arm64: Page-align the .hyp sections KVM: arm64: Disable PMU support in protected mode KVM: arm64: Protect the .hyp sections from the host Ricardo Koller (1): KVM: x86: Move reverse CPUID helpers to separate header file Sean Christopherson (147): KVM: x86/mmu: Alloc page for PDPTEs when shadowing 32-bit NPT with 64-bit KVM: x86/mmu: Capture 'mmu' in a local variable when allocating roots KVM: x86/mmu: Allocate the lm_root before allocating PAE roots KVM: x86/mmu: Allocate pae_root and lm_root pages in dedicated helper KVM: x86/mmu: Ensure MMU pages are available when allocating roots KVM: x86/mmu: Check PDPTRs before allocating PAE roots KVM: x86/mmu: Fix and unconditionally enable WARNs to detect PAE leaks KVM: x86/mmu: Set the C-bit in the PDPTRs and LM pseudo-PDPTRs KVM: nVMX: Defer the MMU reload to the normal path on an EPTP switch KVM: x86: Defer the MMU unload to the normal path on an global INVPCID KVM: x86/mmu: Unexport MMU load/unload functions KVM: x86/mmu: Sync roots after MMU load iff load as successful KVM: x86/mmu: WARN on NULL pae_root or lm_root, or bad shadow root level KVM: SVM: Don't strip the C-bit from CR2 on #PF interception KVM: nSVM: Set the shadow root level to the TDP level for nested NPT KVM: x86: Move nVMX's consistency check macro to common code KVM: nSVM: Trace VM-Enter consistency check failures KVM: x86: Handle triple fault in L2 without killing L1 KVM: nSVM: Add helper to synthesize nested VM-Exit without collateral KVM: nSVM: Add VMLOAD/VMSAVE helper to deduplicate code KVM: x86: Move XSETBV emulation to common code KVM: x86: Move trivial instruction-based exit handlers to common code KVM: x86: Move RDPMC emulation to common code KVM: SVM: Don't manually emulate RDPMC if nrips=0 KVM: SVM: Skip intercepted PAUSE instructions after emulation KVM: x86/mmu: Remove spurious TLB flush from TDP MMU's change_pte() hook KVM: x86/mmu: WARN if TDP MMU's set_tdp_spte() sees multiple GFNs KVM: x86/mmu: Use 'end' param in TDP MMU's test_age_gfn() KVM: x86/mmu: Add typedefs for rmap/iter handlers KVM: x86/mmu: Add convenience wrapper for acting on single hva in TDP MMU KVM: x86/mmu: Check for shadow-present SPTE before querying A/D status KVM: x86/mmu: Bail from fast_page_fault() if SPTE is not shadow-present KVM: x86/mmu: Disable MMIO caching if MMIO value collides with L1TF KVM: x86/mmu: Retry page faults that hit an invalid memslot KVM: x86/mmu: Don't install bogus MMIO SPTEs if MMIO caching is disabled KVM: x86/mmu: Handle MMIO SPTEs directly in mmu_set_spte() KVM: x86/mmu: Drop redundant trace_kvm_mmu_set_spte() in the TDP MMU KVM: x86/mmu: Rename 'mask' to 'spte' in MMIO SPTE helpers KVM: x86/mmu: Stop using software available bits to denote MMIO SPTEs KVM: x86/mmu: Add module param to disable MMIO caching (for testing) KVM: x86/mmu: Rename and document A/D scheme for TDP SPTEs KVM: x86/mmu: Use MMIO SPTE bits 53 and 52 for the MMIO generation KVM: x86/mmu: Document dependency bewteen TDP A/D type and saved bits KVM: x86/mmu: Move initial kvm_mmu_set_mask_ptes() call into MMU proper KVM: x86/mmu: Co-locate code for setting various SPTE masks KVM: x86/mmu: Move logic for setting SPTE masks for EPT into the MMU proper KVM: x86/mmu: Make Host-writable and MMU-writable bit locations dynamic KVM: x86/mmu: Use high bits for host/mmu writable masks for EPT SPTEs KVM: x86/mmu: Use a dedicated bit to track shadow/MMU-present SPTEs KVM: x86/mmu: Tweak auditing WARN for A/D bits to !PRESENT (was MMIO) KVM: x86/mmu: Use is_removed_spte() instead of open coded equivalents KVM: x86/mmu: Use low available bits for removed SPTEs KVM: x86/mmu: Dump reserved bits if they're detected on non-MMIO SPTE KVM: x86: Get active PCID only when writing a CR3 value KVM: VMX: Track common EPTP for Hyper-V's paravirt TLB flush KVM: VMX: Stash kvm_vmx in a local variable for Hyper-V paravirt TLB flush KVM: VMX: Fold Hyper-V EPTP checking into it's only caller KVM: VMX: Do Hyper-V TLB flush iff vCPU's EPTP hasn't been flushed KVM: VMX: Invalidate hv_tlb_eptp to denote an EPTP mismatch KVM: VMX: Don't invalidate hv_tlb_eptp if the new EPTP matches KVM: VMX: Explicitly check for hv_remote_flush_tlb when loading pgd KVM: VMX: Define Hyper-V paravirt TLB flush fields iff Hyper-V is enabled KVM: VMX: Skip additional Hyper-V TLB EPTP flushes if one fails KVM: VMX: Track root HPA instead of EPTP for paravirt Hyper-V TLB flush KVM: x86/mmu: Use '0' as the one and only value for an invalid PAE root KVM: x86/mmu: Mark the PAE roots as decrypted for shadow paging KVM: x86/mmu: Coalesce TDP MMU TLB flushes when zapping collapsible SPTEs KVM: x86/mmu: Move flushing for "slot" handlers to caller for legacy MMU KVM: x86/mmu: Coalesce TLB flushes when zapping collapsible SPTEs KVM: x86/mmu: Coalesce TLB flushes across address spaces for gfn range zap KVM: x86/mmu: Pass address space ID to __kvm_tdp_mmu_zap_gfn_range() KVM: x86/mmu: Pass address space ID to TDP MMU root walkers KVM: x86/mmu: Use leaf-only loop for walking TDP SPTEs when changing SPTE KVM: Move prototypes for MMU notifier callbacks to generic code KVM: Move arm64's MMU notifier trace events to generic code KVM: x86/mmu: Drop trace_kvm_age_page() tracepoint KVM: x86/mmu: Remove spurious clearing of dirty bit from TDP MMU SPTE KVM: x86/mmu: Simplify code for aging SPTEs in TDP MMU KVM: SVM: Use online_vcpus, not created_vcpus, to iterate over vCPUs KVM: SVM: Do not set sev->es_active until KVM_SEV_ES_INIT completes KVM: SVM: Do not allow SEV/SEV-ES initialization after vCPUs are created KVM: x86: Account a variety of miscellaneous allocations KVM: Explicitly use GFP_KERNEL_ACCOUNT for 'struct kvm_vcpu' allocations KVM: Assert that notifier count is elevated in .change_pte() KVM: Move x86's MMU notifier memslot walkers to generic code KVM: arm64: Convert to the gfn-based MMU notifier callbacks KVM: MIPS/MMU: Convert to the gfn-based MMU notifier callbacks KVM: PPC: Convert to the gfn-based MMU notifier callbacks KVM: Kill off the old hva-based MMU notifier callbacks KVM: Move MMU notifier's mmu_lock acquisition into common helper KVM: Take mmu_lock when handling MMU notifier iff the hva hits a memslot KVM: x86/mmu: Allow yielding during MMU notifier unmap/zap, if possible KVM: SVM: Don't set current_vmcb->cpu when switching vmcb KVM: SVM: Drop vcpu_svm.vmcb_pa KVM: SVM: Add a comment to clarify what vcpu_svm.vmcb points at KVM: SVM: Enhance and clean up the vmcb tracking comment in pre_svm_run() KVM: Destroy I/O bus devices on unregister failure _after_ sync'ing SRCU KVM: Stop looking for coalesced MMIO zones if the bus is destroyed KVM: Add proper lockdep assertion in I/O bus unregister KVM: x86: Export kvm_mmu_gva_to_gpa_{read,write}() for SGX (VMX) KVM: x86: Define new #PF SGX error code bit KVM: x86: Add support for reverse CPUID lookup of scattered features KVM: x86: Add reverse-CPUID lookup support for scattered SGX features KVM: VMX: Add basic handling of VM-Exit from SGX enclave KVM: VMX: Frame in ENCLS handler for SGX virtualization KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions KVM: VMX: Add emulation of SGX Launch Control LE hash MSRs KVM: VMX: Add ENCLS[EINIT] handler to support SGX Launch Control (LC) KVM: VMX: Enable SGX virtualization for SGX1, SGX2 and LC KVM: x86: Add capability to grant VM access to privileged SGX attribute crypto: ccp: Free SEV device if SEV init fails crypto: ccp: Detect and reject "invalid" addresses destined for PSP crypto: ccp: Reject SEV commands with mismatching command buffer crypto: ccp: Play nice with vmalloc'd memory for SEV command structs crypto: ccp: Use the stack for small SEV command buffers crypto: ccp: Use the stack and common buffer for status commands crypto: ccp: Use the stack and common buffer for INIT command KVM: SVM: Allocate SEV command structures on local stack KVM: x86: Fix implicit enum conversion goof in scattered reverse CPUID code KVM: VMX: Invert the inlining of MSR interception helpers KVM: SVM: Inject #GP on guest MSR_TSC_AUX accesses if RDTSCP unsupported KVM: SVM: Clear MSR_TSC_AUX[63:32] on write KVM: SVM: Delay restoration of host MSR_TSC_AUX until return to userspace KVM: VMX: Intercept FS/GS_BASE MSR accesses for 32-bit KVM KVM: x86: Remove emulator's broken checks on CR0/CR3/CR4 loads KVM: x86: Check CR3 GPA for validity regardless of vCPU mode KVM: SVM: Truncate GPR value for DR and CR accesses in !64-bit mode KVM: VMX: Truncate GPR value for DR and CR reads in !64-bit mode KVM: nVMX: Truncate bits 63:32 of VMCS field on nested check in !64-bit KVM: nVMX: Truncate base/index GPR value on address calc in !64-bit KVM: x86/xen: Drop RAX[63:32] when processing hypercall KVM: SVM: Use default rAX size for INVLPGA emulation KVM: x86: Rename GPR accessors to make mode-aware variants the defaults x86/sev: Drop redundant and potentially misleading 'sev_enabled' KVM: SVM: Zero out the VMCB array used to track SEV ASID association KVM: SVM: Free sev_asid_bitmap during init if SEV setup fails KVM: SVM: Disable SEV/SEV-ES if NPT is disabled KVM: SVM: Move SEV module params/variables to sev.c KVM: SVM: Append "_enabled" to module-scoped SEV/SEV-ES control variables KVM: SVM: Condition sev_enabled and sev_es_enabled on CONFIG_KVM_AMD_SEV=y KVM: SVM: Enable SEV/SEV-ES functionality by default (when supported) KVM: SVM: Unconditionally invoke sev_hardware_teardown() KVM: SVM: Explicitly check max SEV ASID during sev_hardware_setup() KVM: SVM: Move SEV VMCB tracking allocation to sev.c KVM: SVM: Drop redundant svm_sev_enabled() helper KVM: SVM: Remove an unnecessary prototype declaration of sev_flush_asids() KVM: SVM: Skip SEV cache flush if no ASIDs have been used Shenming Lu (4): irqchip/gic-v3-its: Drop the setting of PTZ altogether KVM: arm64: GICv4.1: Add function to get VLPI state KVM: arm64: GICv4.1: Try to save VLPI state in save_pending_tables KVM: arm64: GICv4.1: Give a chance to save VLPI state Steve Rutherford (1): KVM: SVM: Add support for KVM_SEV_SEND_CANCEL command Suzuki K Poulose (15): perf: aux: Add flags for the buffer format perf: aux: Add CoreSight PMU buffer formats arm64: Add support for trace synchronization barrier KVM: arm64: Handle access to TRFCR_EL1 KVM: arm64: Move SPE availability check to VCPU load arm64: KVM: Enable access to TRBE support for host coresight: etm4x: Move ETM to prohibited region for disable coresight: etm-perf: Allow an event to use different sinks coresight: Do not scan for graph if none is present coresight: etm4x: Add support for PE OS lock coresight: ete: Add support for ETE sysreg access coresight: ete: Add support for ETE tracing dts: bindings: Document device tree bindings for ETE coresight: etm-perf: Handle stale output handles dts: bindings: Document device tree bindings for Arm TRBE Thomas Gleixner (1): time: Add mechanism to recognize clocksource in time_get_snapshot Tom Lendacky (1): KVM: SVM: Make sure GHCB is mapped before updating Uros Bizjak (1): KVM/SVM: Move vmenter.S exception fixups out of line Vitaly Kuznetsov (3): KVM: x86/vPMU: Forbid reading from MSR_F15H_PERF MSRs when guest doesn't have X86_FEATURE_PERFCTR_CORE KVM: x86: Properly handle APF vs disabled LAPIC situation KVM: selftests: Speed up set_memory_region_test Wang Wensheng (1): KVM: arm64: Fix error return code in init_hyp_mode() Wanpeng Li (5): x86/kvm: Don't bother __pv_cpu_mask when !CONFIG_SMP KVM: X86: Count attempted/successful directed yield KVM: X86: Do not yield to self KVM: Boost vCPU candidate in user mode which is delivering interrupt KVM: X86: Fix failure to boost kernel lock holder candidate in SEV-ES guests Wei Yongjun (2): coresight: core: Make symbol 'csdev_sink' static coresight: trbe: Fix return value check in arm_trbe_register_coresight_cpu() Will Deacon (5): arm64: lib: Annotate {clear, copy}_page() as position-independent KVM: arm64: Link position-independent string routines into .hyp.text arm64: kvm: Add standalone ticket spinlock implementation for use at hyp arm/arm64: Probe for the presence of KVM hypervisor KVM: arm64: Advertise KVM UID to guests via SMCCC Xiaofei Tan (1): arm64: sve: Provide sve_cond_update_zcr_vq fallback when !ARM64_SVE Xu Jia (1): KVM: arm64: Make symbol '_kvm_host_prot_finalize' static Yanan Wang (8): tools/headers: sync headers of asm-generic/hugetlb_encode.h KVM: selftests: Print the errno besides error-string in TEST_ASSERT KVM: selftests: Make a generic helper to get vm guest mode strings KVM: selftests: Add a helper to get system configured THP page size KVM: selftests: Add a helper to get system default hugetlb page size KVM: selftests: List all hugetlb src types specified with page sizes KVM: selftests: Adapt vm_userspace_mem_region_add to new helpers KVM: selftests: Add a test for kvm page table code Yang Yingliang (1): KVM: selftests: remove redundant semi-colon Zenghui Yu (2): KVM: arm64: GICv4.1: Restore VLPI pending state to physical side KVM: arm64: Fix Function ID typo for PTP_KVM service Zhenzhong Duan (1): selftests: kvm: Fix the check of return value .../ABI/testing/sysfs-bus-coresight-devices-trbe | 14 + Documentation/devicetree/bindings/arm/ete.yaml | 75 ++ Documentation/devicetree/bindings/arm/trbe.yaml | 49 + Documentation/trace/coresight/coresight-trbe.rst | 38 + Documentation/virt/kvm/amd-memory-encryption.rst | 143 +++ Documentation/virt/kvm/api.rst | 214 +++- Documentation/virt/kvm/arm/index.rst | 1 + Documentation/virt/kvm/arm/ptp_kvm.rst | 25 + Documentation/virt/kvm/devices/arm-vgic-its.rst | 2 +- Documentation/virt/kvm/devices/arm-vgic-v3.rst | 2 +- Documentation/virt/kvm/locking.rst | 49 +- Documentation/virt/kvm/s390-diag.rst | 33 + MAINTAINERS | 6 +- arch/arm/include/asm/hypervisor.h | 3 + arch/arm64/include/asm/assembler.h | 27 +- arch/arm64/include/asm/barrier.h | 1 + arch/arm64/include/asm/el2_setup.h | 13 + arch/arm64/include/asm/fpsimd.h | 11 + arch/arm64/include/asm/fpsimdmacros.h | 10 +- arch/arm64/include/asm/hyp_image.h | 7 + arch/arm64/include/asm/hypervisor.h | 3 + arch/arm64/include/asm/kvm_arm.h | 2 + arch/arm64/include/asm/kvm_asm.h | 9 + arch/arm64/include/asm/kvm_host.h | 55 +- arch/arm64/include/asm/kvm_hyp.h | 14 +- arch/arm64/include/asm/kvm_mmu.h | 25 +- arch/arm64/include/asm/kvm_pgtable.h | 164 ++- arch/arm64/include/asm/pgtable-prot.h | 4 +- arch/arm64/include/asm/sections.h | 1 + arch/arm64/include/asm/sysreg.h | 59 +- arch/arm64/kernel/asm-offsets.c | 3 + arch/arm64/kernel/cpu-reset.S | 5 +- arch/arm64/kernel/hyp-stub.S | 3 +- arch/arm64/kernel/image-vars.h | 34 +- arch/arm64/kernel/vmlinux.lds.S | 74 +- arch/arm64/kvm/arm.c | 220 +++- arch/arm64/kvm/debug.c | 118 +- arch/arm64/kvm/fpsimd.c | 26 +- arch/arm64/kvm/guest.c | 11 +- arch/arm64/kvm/handle_exit.c | 45 + arch/arm64/kvm/hyp/Makefile | 2 +- arch/arm64/kvm/hyp/fpsimd.S | 10 + arch/arm64/kvm/hyp/include/hyp/switch.h | 107 +- arch/arm64/kvm/hyp/include/nvhe/early_alloc.h | 14 + arch/arm64/kvm/hyp/include/nvhe/gfp.h | 68 ++ arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 36 + arch/arm64/kvm/hyp/include/nvhe/memory.h | 51 + arch/arm64/kvm/hyp/include/nvhe/mm.h | 96 ++ arch/arm64/kvm/hyp/include/nvhe/spinlock.h | 92 ++ arch/arm64/kvm/hyp/nvhe/Makefile | 9 +- arch/arm64/kvm/hyp/nvhe/cache.S | 13 + arch/arm64/kvm/hyp/nvhe/debug-sr.c | 56 +- arch/arm64/kvm/hyp/nvhe/early_alloc.c | 54 + arch/arm64/kvm/hyp/nvhe/gen-hyprel.c | 18 + arch/arm64/kvm/hyp/nvhe/host.S | 18 +- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 54 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 75 +- arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 6 +- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 279 +++++ arch/arm64/kvm/hyp/nvhe/mm.c | 173 +++ arch/arm64/kvm/hyp/nvhe/page_alloc.c | 195 ++++ arch/arm64/kvm/hyp/nvhe/psci-relay.c | 4 +- arch/arm64/kvm/hyp/nvhe/setup.c | 214 ++++ arch/arm64/kvm/hyp/nvhe/stub.c | 22 + arch/arm64/kvm/hyp/nvhe/switch.c | 26 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 4 +- arch/arm64/kvm/hyp/pgtable.c | 410 +++++-- arch/arm64/kvm/hyp/reserved_mem.c | 113 ++ arch/arm64/kvm/hyp/vhe/switch.c | 4 +- arch/arm64/kvm/hypercalls.c | 80 +- arch/arm64/kvm/mmu.c | 254 +++-- arch/arm64/kvm/perf.c | 7 +- arch/arm64/kvm/pmu-emul.c | 2 +- arch/arm64/kvm/pmu.c | 8 +- arch/arm64/kvm/reset.c | 51 +- arch/arm64/kvm/sys_regs.c | 16 + arch/arm64/kvm/trace_arm.h | 66 -- arch/arm64/kvm/va_layout.c | 7 + arch/arm64/kvm/vgic/vgic-init.c | 12 +- arch/arm64/kvm/vgic/vgic-its.c | 6 +- arch/arm64/kvm/vgic/vgic-kvm-device.c | 7 +- arch/arm64/kvm/vgic/vgic-mmio-v3.c | 81 +- arch/arm64/kvm/vgic/vgic-mmio.c | 10 +- arch/arm64/kvm/vgic/vgic-v3.c | 66 +- arch/arm64/kvm/vgic/vgic-v4.c | 38 + arch/arm64/kvm/vgic/vgic.h | 2 + arch/arm64/lib/clear_page.S | 4 +- arch/arm64/lib/copy_page.S | 4 +- arch/arm64/mm/init.c | 3 + arch/mips/include/asm/kvm_host.h | 17 +- arch/mips/kvm/mips.c | 21 +- arch/mips/kvm/mmu.c | 100 +- arch/mips/kvm/trap_emul.c | 13 +- arch/mips/kvm/vz.c | 19 +- arch/powerpc/include/asm/kvm_book3s.h | 12 +- arch/powerpc/include/asm/kvm_host.h | 7 - arch/powerpc/include/asm/kvm_ppc.h | 9 +- arch/powerpc/kvm/book3s.c | 18 +- arch/powerpc/kvm/book3s.h | 10 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 98 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 25 +- arch/powerpc/kvm/book3s_hv.c | 12 +- arch/powerpc/kvm/book3s_pr.c | 56 +- arch/powerpc/kvm/e500_mmu_host.c | 29 +- arch/powerpc/kvm/trace_booke.h | 15 - arch/s390/include/asm/kvm_host.h | 5 + arch/s390/include/asm/smp.h | 1 + arch/s390/kernel/smp.c | 1 + arch/s390/kvm/diag.c | 31 +- arch/s390/kvm/gaccess.c | 30 +- arch/s390/kvm/gaccess.h | 60 +- arch/s390/kvm/kvm-s390.c | 15 +- arch/s390/kvm/kvm-s390.h | 8 + arch/s390/kvm/vsie.c | 109 +- arch/sh/kernel/perf_event.c | 18 - arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/kvm_host.h | 67 +- arch/x86/include/asm/mem_encrypt.h | 1 - arch/x86/include/asm/svm.h | 4 +- arch/x86/include/asm/vmx.h | 1 + arch/x86/include/uapi/asm/vmx.h | 1 + arch/x86/kernel/kvm.c | 118 +- arch/x86/kvm/Makefile | 2 + arch/x86/kvm/cpuid.c | 98 +- arch/x86/kvm/cpuid.h | 155 +-- arch/x86/kvm/emulate.c | 80 +- arch/x86/kvm/kvm_cache_regs.h | 19 +- arch/x86/kvm/lapic.c | 8 +- arch/x86/kvm/mmu.h | 23 +- arch/x86/kvm/mmu/mmu.c | 637 ++++++----- arch/x86/kvm/mmu/mmu_audit.c | 2 +- arch/x86/kvm/mmu/mmu_internal.h | 44 +- arch/x86/kvm/mmu/paging_tmpl.h | 3 +- arch/x86/kvm/mmu/spte.c | 159 ++- arch/x86/kvm/mmu/spte.h | 141 ++- arch/x86/kvm/mmu/tdp_mmu.c | 740 +++++++------ arch/x86/kvm/mmu/tdp_mmu.h | 51 +- arch/x86/kvm/reverse_cpuid.h | 186 ++++ arch/x86/kvm/svm/avic.c | 24 +- arch/x86/kvm/svm/nested.c | 573 ++++++---- arch/x86/kvm/svm/sev.c | 922 ++++++++++++---- arch/x86/kvm/svm/svm.c | 1107 +++++++++---------- arch/x86/kvm/svm/svm.h | 91 +- arch/x86/kvm/svm/vmenter.S | 47 +- arch/x86/kvm/vmx/nested.c | 83 +- arch/x86/kvm/vmx/nested.h | 5 + arch/x86/kvm/vmx/sgx.c | 502 +++++++++ arch/x86/kvm/vmx/sgx.h | 34 + arch/x86/kvm/vmx/vmcs12.c | 1 + arch/x86/kvm/vmx/vmcs12.h | 4 +- arch/x86/kvm/vmx/vmx.c | 432 ++++---- arch/x86/kvm/vmx/vmx.h | 39 +- arch/x86/kvm/vmx/vmx_ops.h | 4 + arch/x86/kvm/x86.c | 214 +++- arch/x86/kvm/x86.h | 18 +- arch/x86/mm/mem_encrypt.c | 10 +- arch/x86/mm/mem_encrypt_identity.c | 1 - drivers/clocksource/arm_arch_timer.c | 36 + drivers/crypto/ccp/sev-dev.c | 193 ++-- drivers/crypto/ccp/sev-dev.h | 4 +- drivers/firmware/psci/psci.c | 2 + drivers/firmware/smccc/Makefile | 2 +- drivers/firmware/smccc/kvm_guest.c | 50 + drivers/firmware/smccc/smccc.c | 1 + drivers/hwtracing/coresight/Kconfig | 24 +- drivers/hwtracing/coresight/Makefile | 1 + drivers/hwtracing/coresight/coresight-core.c | 29 +- drivers/hwtracing/coresight/coresight-etm-perf.c | 119 +- drivers/hwtracing/coresight/coresight-etm4x-core.c | 161 ++- .../hwtracing/coresight/coresight-etm4x-sysfs.c | 19 +- drivers/hwtracing/coresight/coresight-etm4x.h | 83 +- drivers/hwtracing/coresight/coresight-platform.c | 6 + drivers/hwtracing/coresight/coresight-priv.h | 3 + drivers/hwtracing/coresight/coresight-trbe.c | 1157 ++++++++++++++++++++ drivers/hwtracing/coresight/coresight-trbe.h | 152 +++ drivers/irqchip/irq-gic-v3-its.c | 18 +- drivers/perf/arm_pmu.c | 30 - drivers/ptp/Kconfig | 2 +- drivers/ptp/Makefile | 2 + drivers/ptp/ptp_kvm_arm.c | 28 + drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 85 +- drivers/ptp/ptp_kvm_x86.c | 97 ++ include/kvm/arm_pmu.h | 4 + include/kvm/arm_vgic.h | 1 + include/linux/arm-smccc.h | 41 + include/linux/bug.h | 10 + include/linux/clocksource.h | 6 + include/linux/clocksource_ids.h | 12 + include/linux/coresight.h | 13 + include/linux/kvm_host.h | 24 +- include/linux/perf_event.h | 2 - include/linux/psp-sev.h | 18 +- include/linux/ptp_kvm.h | 19 + include/linux/timekeeping.h | 12 +- include/trace/events/kvm.h | 90 +- include/uapi/linux/kvm.h | 45 + include/uapi/linux/perf_event.h | 13 +- kernel/events/core.c | 5 - kernel/time/clocksource.c | 2 + kernel/time/timekeeping.c | 1 + lib/bug.c | 54 +- tools/include/asm-generic/hugetlb_encode.h | 3 + tools/testing/selftests/kvm/.gitignore | 2 + tools/testing/selftests/kvm/Makefile | 4 + tools/testing/selftests/kvm/aarch64/vgic_init.c | 551 ++++++++++ tools/testing/selftests/kvm/dirty_log_test.c | 69 +- tools/testing/selftests/kvm/include/kvm_util.h | 13 +- tools/testing/selftests/kvm/include/test_util.h | 21 +- tools/testing/selftests/kvm/kvm_page_table_test.c | 506 +++++++++ tools/testing/selftests/kvm/lib/assert.c | 4 +- tools/testing/selftests/kvm/lib/kvm_util.c | 138 ++- tools/testing/selftests/kvm/lib/test_util.c | 163 ++- .../testing/selftests/kvm/set_memory_region_test.c | 61 +- .../testing/selftests/kvm/x86_64/xen_shinfo_test.c | 2 +- virt/kvm/coalesced_mmio.c | 19 +- virt/kvm/kvm_main.c | 303 +++-- 217 files changed, 12444 insertions(+), 4028 deletions(-) diff --combined arch/x86/kernel/kvm.c index 5d32fa477a62,bd01a6131edf..000000000000 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@@ -574,6 -574,49 +574,54 @@@ static void kvm_smp_send_call_func_ipi( } } -static void kvm_flush_tlb_others(const struct cpumask *cpumask, ++static void kvm_flush_tlb_multi(const struct cpumask *cpumask, + const struct flush_tlb_info *info) + { + u8 state; + int cpu; + struct kvm_steal_time *src; + struct cpumask *flushmask = this_cpu_cpumask_var_ptr(__pv_cpu_mask); + + cpumask_copy(flushmask, cpumask); + /* + * We have to call flush only on online vCPUs. And + * queue flush_on_enter for pre-empted vCPUs + */ + for_each_cpu(cpu, flushmask) { ++ /* ++ * The local vCPU is never preempted, so we do not explicitly ++ * skip check for local vCPU - it will never be cleared from ++ * flushmask. ++ */ + src = &per_cpu(steal_time, cpu); + state = READ_ONCE(src->preempted); + if ((state & KVM_VCPU_PREEMPTED)) { + if (try_cmpxchg(&src->preempted, &state, + state | KVM_VCPU_FLUSH_TLB)) + __cpumask_clear_cpu(cpu, flushmask); + } + } + - native_flush_tlb_others(flushmask, info); ++ native_flush_tlb_multi(flushmask, info); + } + + static __init int kvm_alloc_cpumask(void) + { + int cpu; + + if (!kvm_para_available() || nopv) + return 0; + + if (pv_tlb_flush_supported() || pv_ipi_supported()) + for_each_possible_cpu(cpu) { + zalloc_cpumask_var_node(per_cpu_ptr(&__pv_cpu_mask, cpu), + GFP_KERNEL, cpu_to_node(cpu)); + } + + return 0; + } + arch_initcall(kvm_alloc_cpumask); + static void __init kvm_smp_prepare_boot_cpu(void) { /* @@@ -611,38 -654,8 +659,8 @@@ static int kvm_cpu_down_prepare(unsigne local_irq_enable(); return 0; } - #endif - - static void kvm_flush_tlb_multi(const struct cpumask *cpumask, - const struct flush_tlb_info *info) - { - u8 state; - int cpu; - struct kvm_steal_time *src; - struct cpumask *flushmask = this_cpu_cpumask_var_ptr(__pv_cpu_mask); - - cpumask_copy(flushmask, cpumask); - /* - * We have to call flush only on online vCPUs. And - * queue flush_on_enter for pre-empted vCPUs - */ - for_each_cpu(cpu, flushmask) { - /* - * The local vCPU is never preempted, so we do not explicitly - * skip check for local vCPU - it will never be cleared from - * flushmask. - */ - src = &per_cpu(steal_time, cpu); - state = READ_ONCE(src->preempted); - if ((state & KVM_VCPU_PREEMPTED)) { - if (try_cmpxchg(&src->preempted, &state, - state | KVM_VCPU_FLUSH_TLB)) - __cpumask_clear_cpu(cpu, flushmask); - } - } - native_flush_tlb_multi(flushmask, info); - } + #endif static void __init kvm_guest_init(void) { @@@ -655,15 -668,9 +673,9 @@@ if (kvm_para_has_feature(KVM_FEATURE_STEAL_TIME)) { has_steal_clock = 1; - pv_ops.time.steal_clock = kvm_steal_clock; + static_call_update(pv_steal_clock, kvm_steal_clock); } - if (pv_tlb_flush_supported()) { - pv_ops.mmu.flush_tlb_multi = kvm_flush_tlb_multi; - pv_ops.mmu.tlb_remove_table = tlb_remove_table; - pr_info("KVM setup pv remote TLB flush\n"); - } - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) apic_set_eoi_write(kvm_guest_apic_eoi_write); @@@ -673,6 -680,12 +685,12 @@@ } #ifdef CONFIG_SMP + if (pv_tlb_flush_supported()) { - pv_ops.mmu.flush_tlb_others = kvm_flush_tlb_others; ++ pv_ops.mmu.flush_tlb_multi = kvm_flush_tlb_multi; + pv_ops.mmu.tlb_remove_table = tlb_remove_table; + pr_info("KVM setup pv remote TLB flush\n"); + } + smp_ops.smp_prepare_boot_cpu = kvm_smp_prepare_boot_cpu; if (pv_sched_yield_supported()) { smp_ops.send_call_func_ipi = kvm_smp_send_call_func_ipi;
