Currently, on POWER9 machines using HV KVM, the host kernel will use one or other of the two MMU modes available, HPT (hashed page table) or radix, and all KVM guests must use the same MMU mode. This is inconvenient because it means that a guest on a radix host can't run a legacy OS that doesn't know about POWER9. On the other hand, if the host is booted in HPT mode so that legacy guest kernels can be run, then neither the host nor any other guest can take advantage of the radix MMU mode. To further complicate matters, current POWER9 chips have the restriction that all SMT threads in each core must use the same MMU mode (HPT or radix), though they can be in different partitions. This patch series adds the ability to run HPT guests on a host in radix mode. In order to comply with the restriction of not mixing HPT and radix threads on the same core, we take a similar approach to that already used on POWER8, namely, run the host in single-threaded mode (only thread 0 of each core online), and have KVM be able to wake up the other threads when a KVM guest is to be run, and use the other threads for running guest VCPUs. So that KVM knows whether it should be managing individual threads on POWER9 (as it does currently) or whole cores, we add a new module parameter called "indep_threads_mode". This is normally Y on POWER9 but must be set to N before any HPT guests can be run on a radix host. The value of indep_threads_mode is sampled at the time of creation of each guest, and the value at the time of creation is what applies to that guest for the whole of its lifetime. It is preferable to change indep_threads_mode only when no guests are running, in order to avoid confusion or unnecessarily reduced performance. In order to allow HPT guests to run on a radix host, the following commands (or their equivalents) are required first: # echo N >/sys/module/kvm_hv/parameters/indep_threads_mode # ppc64_cpu --smt=off KVM will run up to four vcpus on each core using the existing dynamic micro-threading code. This can be controlled using the target_smt_mode module parameter. This patch series is against my kvm-ppc-next branch with the topic/ppc-kvm branch from the powerpc merged in. In this version, I have dropped the patch 'Revert "KVM: PPC: Book3S HV: POWER9 does not require secondary thread management"' since it has been merged in the powerpc tree, and I have added a patch to unify the dirty bit management between HPT and radix, so now the dirty bits do not get lost when switching between HPT and radix modes. Paul. arch/powerpc/include/asm/kvm_book3s.h | 3 +- arch/powerpc/include/asm/kvm_book3s_64.h | 140 +++++++++++-- arch/powerpc/include/asm/kvm_book3s_asm.h | 13 +- arch/powerpc/include/asm/kvm_host.h | 6 +- arch/powerpc/include/asm/kvm_ppc.h | 3 + arch/powerpc/kernel/asm-offsets.c | 3 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 110 +++++----- arch/powerpc/kvm/book3s_64_mmu_radix.c | 51 +---- arch/powerpc/kvm/book3s_hv.c | 321 +++++++++++++++++++----------- arch/powerpc/kvm/book3s_hv_builtin.c | 99 ++++++++- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 65 +++--- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 65 +++++- arch/powerpc/kvm/powerpc.c | 3 +- 13 files changed, 605 insertions(+), 277 deletions(-)
