The TDP MMU has a performance regression compared to the legacy MMU when CR0 changes often. This was reported for the grsecurity kernel, which uses CR0.WP to implement kernel W^X. In that case, each change to CR0.WP unloads the MMU and causes a lot of unnecessary work. When running nested, this can even cause the L1 to hardly make progress, as the L0 hypervisor it is overwhelmed by the amount of MMU work that is needed. Initially, my plan for this was to pull kvm_mmu_unload from kvm_mmu_reset_context into kvm_init_mmu. Therefore I started by separating the CPU setup (CR0/CR4/EFER, SMM, guest mode, etc.) from the shadow page table format. Right now the "MMU role" is a messy mix of the two and, whenever something is different between the MMU and the CPU, it is stored as an extra field in struct kvm_mmu; for extra bonus complication, sometimes the same thing is stored in both the role and an extra field. The aim was to keep kvm_mmu_unload only if the MMU role changed, and drop it if the CPU role changed. I even posted that cleanup, but it occurred to me later that even a conditional kvm_mmu_unload in kvm_init_mmu would be overkill. kvm_mmu_unload is only needed in the rare cases where a TLB flush is needed (e.g. CR0.PG changing from 1 to 0) or where the guest page table interpretation changes in way not captured by the role (that is, CPUID changes). But the implementation of fast PGD switching is subtle and requires a call to kvm_mmu_new_pgd (and therefore knowing the new MMU role) before kvm_init_mmu, therefore kvm_mmu_reset_context chickens and drops all the roots. Therefore, the meat of this series is a reorganization of fast PGD switching; it makes it possible to call kvm_mmu_new_pgd *after* the MMU has been set up, just using the MMU role instead of kvm_mmu_calc_root_page_role. Patches 1 to 3 are bugfixes found while working on the series. Patches 4 to 5 add more sanity checks that triggered a lot during development. Patches 6 and 7 are related cleanups. In particular patch 7 makes the cache lookup code a bit more pleasant. Patches 8 to 9 rework the fast PGD switching. Patches 10 and 11 are cleanups enabled by the rework, and the only survivors of the CPU role patchset. Finally, patch 12 optimizes kvm_mmu_reset_context. Paolo Paolo Bonzini (12): KVM: x86: host-initiated EFER.LME write affects the MMU KVM: MMU: move MMU role accessors to header KVM: x86: do not deliver asynchronous page faults if CR0.PG=0 KVM: MMU: WARN if PAE roots linger after kvm_mmu_unload KVM: MMU: avoid NULL-pointer dereference on page freeing bugs KVM: MMU: rename kvm_mmu_reload KVM: x86: use struct kvm_mmu_root_info for mmu->root KVM: MMU: do not consult levels when freeing roots KVM: MMU: look for a cached PGD when going from 32-bit to 64-bit KVM: MMU: load new PGD after the shadow MMU is initialized KVM: MMU: remove kvm_mmu_calc_root_page_role KVM: x86: do not unload MMU roots on all role changes arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/kvm/mmu.h | 28 +++- arch/x86/kvm/mmu/mmu.c | 253 ++++++++++++++++---------------- arch/x86/kvm/mmu/mmu_audit.c | 4 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 2 +- arch/x86/kvm/mmu/tdp_mmu.h | 2 +- arch/x86/kvm/svm/nested.c | 6 +- arch/x86/kvm/vmx/nested.c | 8 +- arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/x86.c | 39 +++-- 11 files changed, 190 insertions(+), 159 deletions(-) -- 2.31.1
