As we progress towards being able to keep guest state private to the
host running nVHE hypervisor, this series allows the hypervisor to
install itself on newly booted CPUs before the host is allowed to run
on them.
All functionality described below is opt-in, guarded by an early param
'kvm-arm.mode=protected'. Future patches specific to the new protected
mode should be hidden behind the same param.
The hypervisor starts trapping host SMCs and intercepting host's PSCI
calls which boot CPUs. It replaces the host's entry point with its own,
initializes the EL2 state of the new CPU and installs the nVHE hyp vector
before ERETing to the host's entry point.
The kernel checks new cores' features against the finalized system
capabilities. To avoid the need to move this code/data to EL2, the
implementation only allows to boot cores that were online at the time of
KVM initialization and therefore had been checked already.
Other PSCI SMCs are forwarded to EL3, though only the known set of SMCs
implemented in the kernel is allowed. Non-PSCI SMCs are also forwarded
to EL3. Future changes will need to ensure the safety of all SMCs wrt.
protected guests.
The host is still allowed to reset EL2 back to the stub vector, eg. for
hibernation or kexec, but will not disable nVHE when there are no VMs.
Tested on Rock Pi 4B, based on kvmarm/queue, itself on top of 5.10-rc4.
Patches also available at:
https://android-kvm.googlesource.com/linux topic/psci-on-master_v4
changes since v3:
* generic 'kvm-arm.mode' kernel param instead of 'kvm-arm.protected'
* implement SYSTEM_SUSPEND
* refactor PSCI driver to expose fn IDs more cleanly
* init MAIR_EL2, TCR_EL2 from nVHE params struct
changes since v2:
* avoid non-spec error in CPU_SUSPEND
* refuse to init without PSCI
* compute hyp VA args of hyp-init in hyp instead of using params struct
* use hyp_symbol_addr in per-cpu calls
* simplify memory.h/sysreg.h includes
* rebase on kvmarm/queue, use trap handler args macros
changes since v1:
* early param sets a capability instead of a static key
* assume SMCCC v1.2 for host SMC forwarding
* fix reserved SMC ID range for PSCI
* split init_el2_state into smaller macros, move to el2_setup.h
* many small cleanups
changes since RFC:
* add early param to make features opt-in
* simplify CPU_ON/SUSPEND implementation
* replace spinlocks with CAS atomic
* make cpu_logical_map ro_after_init
David Brazdil (26):
kvm: arm64: Add kvm-arm.mode early kernel parameter
kvm: arm64: Add ARM64_KVM_PROTECTED_MODE CPU capability
psci: Support psci_ops.get_version for v0.1
psci: Split functions to v0.1 and v0.2+ variants
psci: Replace psci_function_id array with a struct
psci: Add accessor for psci_0_1_function_ids
arm64: Make cpu_logical_map() take unsigned int
arm64: Extract parts of el2_setup into a macro
kvm: arm64: Remove vector_ptr param of hyp-init
kvm: arm64: Move hyp-init params to a per-CPU struct
kvm: arm64: Init MAIR/TCR_EL2 from params struct
kvm: arm64: Add .hyp.data..ro_after_init ELF section
kvm: arm64: Support per_cpu_ptr in nVHE hyp code
kvm: arm64: Create nVHE copy of cpu_logical_map
kvm: arm64: Add SMC handler in nVHE EL2
kvm: arm64: Bootstrap PSCI SMC handler in nVHE EL2
kvm: arm64: Add offset for hyp VA <-> PA conversion
kvm: arm64: Forward safe PSCI SMCs coming from host
kvm: arm64: Extract __do_hyp_init into a helper function
kvm: arm64: Add function to enter host from KVM nVHE hyp code
kvm: arm64: Intercept host's CPU_ON SMCs
kvm: arm64: Intercept host's CPU_SUSPEND PSCI SMCs
kvm: arm64: Intercept host's SYSTEM_SUSPEND PSCI SMCs
kvm: arm64: Keep nVHE EL2 vector installed
kvm: arm64: Trap host SMCs in protected mode
kvm: arm64: Fix EL2 mode availability checks
.../admin-guide/kernel-parameters.txt | 10 +
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/el2_setup.h | 182 ++++++++++
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/kvm_asm.h | 10 +-
arch/arm64/include/asm/kvm_host.h | 10 +
arch/arm64/include/asm/kvm_hyp.h | 4 +-
arch/arm64/include/asm/kvm_mmu.h | 24 ++
arch/arm64/include/asm/percpu.h | 6 +
arch/arm64/include/asm/sections.h | 1 +
arch/arm64/include/asm/smp.h | 4 +-
arch/arm64/include/asm/virt.h | 26 ++
arch/arm64/kernel/asm-offsets.c | 5 +
arch/arm64/kernel/cpufeature.c | 22 ++
arch/arm64/kernel/head.S | 144 +-------
arch/arm64/kernel/image-vars.h | 6 +-
arch/arm64/kernel/setup.c | 2 +-
arch/arm64/kernel/vmlinux.lds.S | 10 +
arch/arm64/kvm/arm.c | 139 +++++++-
.../arm64/kvm/hyp/include/nvhe/trap_handler.h | 18 +
arch/arm64/kvm/hyp/nvhe/Makefile | 3 +-
arch/arm64/kvm/hyp/nvhe/host.S | 47 +++
arch/arm64/kvm/hyp/nvhe/hyp-init.S | 152 +++++---
arch/arm64/kvm/hyp/nvhe/hyp-main.c | 45 ++-
arch/arm64/kvm/hyp/nvhe/hyp-smp.c | 40 +++
arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 1 +
arch/arm64/kvm/hyp/nvhe/psci-relay.c | 324 ++++++++++++++++++
arch/arm64/kvm/hyp/nvhe/switch.c | 5 +-
arch/arm64/kvm/va_layout.c | 30 +-
drivers/firmware/psci/psci.c | 126 ++++---
include/linux/psci.h | 9 +
31 files changed, 1150 insertions(+), 259 deletions(-)
create mode 100644 arch/arm64/include/asm/el2_setup.h
create mode 100644 arch/arm64/kvm/hyp/include/nvhe/trap_handler.h
create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp-smp.c
create mode 100644 arch/arm64/kvm/hyp/nvhe/psci-relay.c
--
2.29.2.454.gaff20da3a2-goog
