Hi,
On 6/21/19 10:37 AM, Marc Zyngier wrote:
> I've taken over the maintenance of this series originally written by
> Jintack and Christoffer. Since then, the series has been substantially
> reworked, new features (and most probably bugs) have been added, and
> the whole thing rebased multiple times. If anything breaks, please
> blame me, and nobody else.
>
> As you can tell, this is quite big. It is also remarkably incomplete
> (we're missing many critical bits for fully emulate EL2), but the idea
> is to start merging things early in order to reduce the maintenance
> headache. What we want to achieve is that with NV disabled, there is
> no performance overhead and no regression. The only thing I intend to
> merge ASAP is the first patch in the series, because it should have
> zero effect and is a reasonable cleanup.
>
> The series is roughly divided in 4 parts: exception handling, memory
> virtualization, interrupts and timers. There are of course some
> dependencies, but you'll hopefully get the gist of it.
>
> For the most courageous of you, I've put out a branch[1] containing this
> and a bit more. Of course, you'll need some userspace. Andre maintains
> a hacked version of kvmtool[1] that takes a --nested option, allowing
> the guest to be started at EL2. You can run the whole stack in the
> Foundation model. Don't be in a hurry ;-).
>
> [1] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/nv-wip-5.2-rc5
> [2] git://linux-arm.org/kvmtool.git nv/nv-wip-5.2-rc5
>
> Andre Przywara (4):
> KVM: arm64: nv: Handle virtual EL2 registers in
> vcpu_read/write_sys_reg()
> KVM: arm64: nv: Save/Restore vEL2 sysregs
> KVM: arm64: nv: Handle traps for timer _EL02 and _EL2 sysregs
> accessors
> KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ
>
> Christoffer Dall (16):
> KVM: arm64: nv: Introduce nested virtualization VCPU feature
> KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
> KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
> KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
> KVM: arm64: nv: Handle trapped ERET from virtual EL2
> KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
> KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
> KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
> changes
> KVM: arm/arm64: nv: Support multiple nested stage 2 mmu structures
> KVM: arm64: nv: Implement nested Stage-2 page table walk logic
> KVM: arm64: nv: Handle shadow stage 2 page faults
> KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
> KVM: arm64: nv: arch_timer: Support hyp timer emulation
> KVM: arm64: nv: vgic-v3: Take cpu_if pointer directly instead of vcpu
> KVM: arm64: nv: vgic: Emulate the HW bit in software
> KVM: arm64: nv: Add nested GICv3 tracepoints
>
> Dave Martin (1):
> KVM: arm64: Migrate _elx sysreg accessors to msr_s/mrs_s
>
> Jintack Lim (21):
> arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
> KVM: arm64: nv: Add EL2 system registers to vcpu context
> KVM: arm64: nv: Support virtual EL2 exceptions
> KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
> KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
> KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
> KVM: arm64: nv: Set a handler for the system instruction traps
> KVM: arm64: nv: Handle PSCI call via smc from the guest
> KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
> KVM: arm64: nv: Respect virtual CPTR_EL2.TFP setting
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
> KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
> KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
> KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
> KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
> KVM: arm64: nv: Pretend we only support larger-than-host page sizes
> KVM: arm64: nv: Introduce sys_reg_desc.forward_trap
> KVM: arm64: nv: Rework the system instruction emulation framework
> KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
> KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
> KVM: arm64: nv: Nested GICv3 Support
>
> Marc Zyngier (17):
> KVM: arm64: Move __load_guest_stage2 to kvm_mmu.h
> KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
> KVM: arm64: nv: Handle SPSR_EL2 specially
> KVM: arm64: nv: Refactor vcpu_{read,write}_sys_reg
> KVM: arm64: nv: Don't expose SVE to nested guests
> KVM: arm64: nv: Hide RAS from nested guests
> KVM: arm/arm64: nv: Factor out stage 2 page table data from struct kvm
> KVM: arm64: nv: Move last_vcpu_ran to be per s2 mmu
> KVM: arm64: nv: Don't always start an S2 MMU search from the beginning
> KVM: arm64: nv: Propagate CNTVOFF_EL2 to the virtual EL1 timer
> KVM: arm64: nv: Load timer before the GIC
> KVM: arm64: nv: Implement maintenance interrupt forwarding
> arm64: KVM: nv: Add handling of EL2-specific timer registers
> arm64: KVM: nv: Honor SCTLR_EL2.SPAN on entering vEL2
> arm64: KVM: nv: Handle SCTLR_EL2 RES0/RES1 bits
> arm64: KVM: nv: Restrict S2 RD/WR permissions to match the guest's
> arm64: KVM: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
>
> .../admin-guide/kernel-parameters.txt | 4 +
> .../virtual/kvm/devices/arm-vgic-v3.txt | 9 +
> arch/arm/include/asm/kvm_asm.h | 5 +-
> arch/arm/include/asm/kvm_emulate.h | 3 +
> arch/arm/include/asm/kvm_host.h | 31 +-
> arch/arm/include/asm/kvm_hyp.h | 25 +-
> arch/arm/include/asm/kvm_mmu.h | 83 +-
> arch/arm/include/asm/kvm_nested.h | 9 +
> arch/arm/include/uapi/asm/kvm.h | 1 +
> arch/arm/kvm/hyp/switch.c | 11 +-
> arch/arm/kvm/hyp/tlb.c | 13 +-
> arch/arm64/include/asm/cpucaps.h | 3 +-
> arch/arm64/include/asm/esr.h | 4 +-
> arch/arm64/include/asm/kvm_arm.h | 28 +-
> arch/arm64/include/asm/kvm_asm.h | 9 +-
> arch/arm64/include/asm/kvm_coproc.h | 2 +-
> arch/arm64/include/asm/kvm_emulate.h | 157 +-
> arch/arm64/include/asm/kvm_host.h | 105 +-
> arch/arm64/include/asm/kvm_hyp.h | 82 +-
> arch/arm64/include/asm/kvm_mmu.h | 62 +-
> arch/arm64/include/asm/kvm_nested.h | 68 +
> arch/arm64/include/asm/sysreg.h | 143 +-
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kernel/cpufeature.c | 26 +
> arch/arm64/kvm/Makefile | 4 +
> arch/arm64/kvm/emulate-nested.c | 223 +++
> arch/arm64/kvm/guest.c | 6 +
> arch/arm64/kvm/handle_exit.c | 76 +-
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/at.c | 217 +++
> arch/arm64/kvm/hyp/switch.c | 86 +-
> arch/arm64/kvm/hyp/sysreg-sr.c | 267 ++-
> arch/arm64/kvm/hyp/tlb.c | 129 +-
> arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c | 2 +-
> arch/arm64/kvm/inject_fault.c | 12 -
> arch/arm64/kvm/nested.c | 551 +++++++
> arch/arm64/kvm/regmap.c | 4 +-
> arch/arm64/kvm/reset.c | 7 +
> arch/arm64/kvm/sys_regs.c | 1460 +++++++++++++++--
> arch/arm64/kvm/sys_regs.h | 6 +
> arch/arm64/kvm/trace.h | 58 +-
> include/kvm/arm_arch_timer.h | 6 +
> include/kvm/arm_vgic.h | 28 +-
> virt/kvm/arm/arch_timer.c | 158 +-
> virt/kvm/arm/arm.c | 62 +-
> virt/kvm/arm/hyp/vgic-v3-sr.c | 35 +-
> virt/kvm/arm/mmio.c | 12 +-
> virt/kvm/arm/mmu.c | 445 +++--
> virt/kvm/arm/trace.h | 6 +-
> virt/kvm/arm/vgic/vgic-init.c | 30 +
> virt/kvm/arm/vgic/vgic-kvm-device.c | 22 +
> virt/kvm/arm/vgic/vgic-nested-trace.h | 137 ++
> virt/kvm/arm/vgic/vgic-v2.c | 10 +-
> virt/kvm/arm/vgic/vgic-v3-nested.c | 236 +++
> virt/kvm/arm/vgic/vgic-v3.c | 40 +-
> virt/kvm/arm/vgic/vgic.c | 74 +-
> 56 files changed, 4683 insertions(+), 612 deletions(-)
> create mode 100644 arch/arm/include/asm/kvm_nested.h
> create mode 100644 arch/arm64/include/asm/kvm_nested.h
> create mode 100644 arch/arm64/kvm/emulate-nested.c
> create mode 100644 arch/arm64/kvm/hyp/at.c
> create mode 100644 arch/arm64/kvm/nested.c
> create mode 100644 virt/kvm/arm/vgic/vgic-nested-trace.h
> create mode 100644 virt/kvm/arm/vgic/vgic-v3-nested.c
>
When working on adding support for EL2 to kvm-unit-tests I was able to trigger
the following warning:
# ./lkvm run -f psci.flat -m 128 -c 8 --console serial --irqchip gicv3 --nested
# lkvm run --firmware psci.flat -m 128 -c 8 --name guest-151
Info: Placing fdt at 0x80200000 - 0x80210000
# Warning: The maximum recommended amount of VCPUs is 4
chr_testdev_init: chr-testdev: can't find a virtio-console
INFO: PSCI version 1.0
PASS: invalid-function
PASS: affinity-info-on
PASS: affinity-info-off
[ 24.381266] WARNING: CPU: 3 PID: 160 at
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:170
kvm_timer_irq_can_fire+0xc/0x30
[ 24.381366] Modules linked in:
[ 24.381466] CPU: 3 PID: 160 Comm: kvm-vcpu-1 Not tainted
5.2.0-rc5-00060-g7dbce63bd1c7 #145
[ 24.381566] Hardware name: Foundation-v8A (DT)
[ 24.381566] pstate: 40400009 (nZcv daif +PAN -UAO)
[ 24.381666] pc : kvm_timer_irq_can_fire+0xc/0x30
[ 24.381766] lr : timer_emulate+0x24/0x98
[ 24.381766] sp : ffff000013d8b780
[ 24.381866] x29: ffff000013d8b780 x28: ffff80087a639b80
[ 24.381966] x27: ffff000010ba8648 x26: ffff000010b71b40
[ 24.382066] x25: ffff80087a63a100 x24: 0000000000000000
[ 24.382111] x23: 000080086ca54000 x22: ffff0000100ce260
[ 24.382166] x21: ffff800875e7c918 x20: ffff800875e7a800
[ 24.382275] x19: ffff800875e7ca08 x18: 0000000000000000
[ 24.382366] x17: 0000000000000000 x16: 0000000000000000
[ 24.382466] x15: 0000000000000000 x14: 0000000000002118
[ 24.382566] x13: 0000000000002190 x12: 0000000000002280
[ 24.382566] x11: 0000000000002208 x10: 0000000000000040
[ 24.382666] x9 : ffff000012dc3b38 x8 : 0000000000000000
[ 24.382766] x7 : 0000000000000000 x6 : ffff80087ac00248
[ 24.382866] x5 : 000080086ca54000 x4 : 0000000000002118
[ 24.382966] x3 : eeeeeeeeeeeeeeef x2 : ffff800875e7c918
[ 24.383066] x1 : 0000000000000001 x0 : ffff800875e7ca08
[ 24.383066] Call trace:
[ 24.383166] kvm_timer_irq_can_fire+0xc/0x30
[ 24.383266] kvm_timer_vcpu_load+0x9c/0x1a0
[ 24.383366] kvm_arch_vcpu_load+0xb0/0x1f0
[ 24.383366] kvm_sched_in+0x1c/0x28
[ 24.383466] finish_task_switch+0xd8/0x1d8
[ 24.383566] __schedule+0x248/0x4a0
[ 24.383666] preempt_schedule_irq+0x60/0x90
[ 24.383666] el1_irq+0xd0/0x180
[ 24.383766] kvm_handle_guest_abort+0x0/0x3a0
[ 24.383866] kvm_arch_vcpu_ioctl_run+0x41c/0x688
[ 24.383866] kvm_vcpu_ioctl+0x4c0/0x838
[ 24.383966] do_vfs_ioctl+0xb8/0x878
[ 24.384077] ksys_ioctl+0x84/0x90
[ 24.384166] __arm64_sys_ioctl+0x18/0x28
[ 24.384166] el0_svc_common.constprop.0+0xb0/0x168
[ 24.384266] el0_svc_handler+0x28/0x78
[ 24.384366] el0_svc+0x8/0xc
[ 24.384366] ---[ end trace 37a32293e43ac12c ]---
[ 24.384666] WARNING: CPU: 3 PID: 160 at
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:170
kvm_timer_irq_can_fire+0xc/0x30
[ 24.384766] Modules linked in:
[ 24.384866] CPU: 3 PID: 160 Comm: kvm-vcpu-1 Tainted: G W
5.2.0-rc5-00060-g7dbce63bd1c7 #145
[ 24.384966] Hardware name: Foundation-v8A (DT)
[ 24.384966] pstate: 40400009 (nZcv daif +PAN -UAO)
[ 24.385066] pc : kvm_timer_irq_can_fire+0xc/0x30
[ 24.385166] lr : timer_emulate+0x24/0x98
[ 24.385166] sp : ffff000013d8b780
[ 24.385266] x29: ffff000013d8b780 x28: ffff80087a639b80
[ 24.385366] x27: ffff000010ba8648 x26: ffff000010b71b40
[ 24.385466] x25: ffff80087a63a100 x24: 0000000000000000
[ 24.385466] x23: 000080086ca54000 x22: ffff0000100ce260
[ 24.385566] x21: ffff800875e7c918 x20: ffff800875e7a800
[ 24.385666] x19: ffff800875e7ca80 x18: 0000000000000000
[ 24.385766] x17: 0000000000000000 x16: 0000000000000000
[ 24.385866] x15: 0000000000000000 x14: 0000000000002118
[ 24.385966] x13: 0000000000002190 x12: 0000000000002280
[ 24.385966] x11: 0000000000002208 x10: 0000000000000040
[ 24.386066] x9 : ffff000012dc3b38 x8 : 0000000000000000
[ 24.386166] x7 : 0000000000000000 x6 : ffff80087ac00248
[ 24.386266] x5 : 000080086ca54000 x4 : 0000000000002118
[ 24.386366] x3 : eeeeeeeeeeeeeeef x2 : ffff800875e7c918
[ 24.386466] x1 : 0000000000000001 x0 : ffff800875e7ca80
[ 24.386466] Call trace:
[ 24.386566] kvm_timer_irq_can_fire+0xc/0x30
[ 24.386666] kvm_timer_vcpu_load+0xa8/0x1a0
[ 24.386666] kvm_arch_vcpu_load+0xb0/0x1f0
[ 24.386898] kvm_sched_in+0x1c/0x28
[ 24.386966] finish_task_switch+0xd8/0x1d8
[ 24.387166] __schedule+0x248/0x4a0
[ 24.387354] preempt_schedule_irq+0x60/0x90
[ 24.387366] el1_irq+0xd0/0x180
[ 24.387466] kvm_handle_guest_abort+0x0/0x3a0
[ 24.387566] kvm_arch_vcpu_ioctl_run+0x41c/0x688
[ 24.387566] kvm_vcpu_ioctl+0x4c0/0x838
[ 24.387666] do_vfs_ioctl+0xb8/0x878
[ 24.387766] ksys_ioctl+0x84/0x90
[ 24.387866] __arm64_sys_ioctl+0x18/0x28
[ 24.387866] el0_svc_common.constprop.0+0xb0/0x168
[ 24.387966] el0_svc_handler+0x28/0x78
[ 24.388066] el0_svc+0x8/0xc
[ 24.388066] ---[ end trace 37a32293e43ac12d ]---
PASS: cpu-on
SUMMARY: 4 te[ 24.390266] WARNING: CPU: 3 PID: 160 at
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:170
kvm_timer_irq_can_fire+0xc/0x30
s[ 24.390366] Modules linked in:
ts[ 24.390366] CPU: 3 PID: 160 Comm: kvm-vcpu-1 Tainted: G W
5.2.0-rc5-00060-g7dbce63bd1c7 #145
[ 24.390566] Hardware name: Foundation-v8A (DT)
[ 24.390795] pstate: 40400009 (nZcv daif +PAN -UAO)
[ 24.390866] pc : kvm_timer_irq_can_fire+0xc/0x30
[ 24.390966] lr : timer_emulate+0x24/0x98
[ 24.391066] sp : ffff000013d8b780
[ 24.391066] x29: ffff000013d8b780 x28: ffff80087a639b80
[ 24.391166] x27: ffff000010ba8648 x26: ffff000010b71b40
[ 24.391266] x25: ffff80087a63a100 x24: 0000000000000000
[ 24.391366] x23: 000080086ca54000 x22: 0000000000000003
[ 24.391466] x21: ffff800875e7c918 x20: ffff800875e7a800
[ 24.391466] x19: ffff800875e7ca08 x18: 0000000000000000
[ 24.391566] x17: 0000000000000000 x16: 0000000000000000
[ 24.391666] x15: 0000000000000000 x14: 0000000000002118
[ 24.391766] x13: 0000000000002190 x12: 0000000000002280
[ 24.391866] x11: 0000000000002208 x10: 0000000000000040
[ 24.391942] x9 : ffff000012dc3b38 x8 : 0000000000000000
[ 24.391966] x7 : 0000000000000000 x6 : ffff80087ac00248
[ 24.392066] x5 : 000080086ca54000 x4 : 0000000000002118
[ 24.392166] x3 : eeeeeeeeeeeeeeef x2 : ffff800875e7c918
[ 24.392269] x1 : 0000000000000001 x0 : ffff800875e7ca08
[ 24.392366] Call trace:
[ 24.392433] kvm_timer_irq_can_fire+0xc/0x30
[ 24.392466] kvm_timer_vcpu_load+0x9c/0x1a0
[ 24.392597] kvm_arch_vcpu_load+0xb0/0x1f0
[ 24.392666] kvm_sched_in+0x1c/0x28
[ 24.392766] finish_task_switch+0xd8/0x1d8
[ 24.392766] __schedule+0x248/0x4a0
[ 24.392866] preempt_schedule_irq+0x60/0x90
[ 24.392966] el1_irq+0xd0/0x180
[ 24.392966] kvm_handle_guest_abort+0x0/0x3a0
[ 24.393066] kvm_arch_vcpu_ioctl_run+0x41c/0x688
[ 24.393166] kvm_vcpu_ioctl+0x4c0/0x838
[ 24.393266] do_vfs_ioctl+0xb8/0x878
[ 24.393266] ksys_ioctl+0x84/0x90
[ 24.393366] __arm64_sys_ioctl+0x18/0x28
[ 24.393466] el0_svc_common.constprop.0+0xb0/0x168
[ 24.393566] el0_svc_handler+0x28/0x78
[ 24.393566] el0_svc+0x8/0xc
[ 24.393666] ---[ end trace 37a32293e43ac12e ]---
[ 24.393866] WARNING: CPU: 3 PID: 160 at
arch/arm64/kvm/../../../virt/kvm/arm/arch_timer.c:170
kvm_timer_irq_can_fire+0xc/0x30
[ 24.394066] Modules linked in:
[ 24.394266] CPU: 3 PID: 160 Comm: kvm-vcpu-1 Tainted: G W
5.2.0-rc5-00060-g7dbce63bd1c7 #145
[ 24.394366] Hardware name: Foundation-v8A (DT)
[ 24.394466] pstate: 40400009 (nZcv daif +PAN -UAO)
[ 24.394466] pc : kvm_timer_irq_can_fire+0xc/0x30
[ 24.394566] lr : timer_emulate+0x24/0x98
[ 24.394666] sp : ffff000013d8b780
[ 24.394727] x29: ffff000013d8b780 x28: ffff80087a639b80
[ 24.394766] x27: ffff000010ba8648 x26: ffff000010b71b40
[ 24.394866] x25: ffff80087a63a100 x24: 0000000000000000
[ 24.394966] x23: 000080086ca54000 x22: 0000000000000003
[ 24.394966] x21: ffff800875e7c918 x20: ffff800875e7a800
[ 24.395066] x19: ffff800875e7ca80 x18: 0000000000000000
[ 24.395166] x17: 0000000000000000 x16: 0000000000000000
[ 24.395266] x15: 0000000000000000 x14: 0000000000002118
[ 24.395383] x13: 0000000000002190 x12: 0000000000002280
[ 24.395466] x11: 0000000000002208 x10: 0000000000000040
[ 24.395547] x9 : ffff000012dc3b38 x8 : 0000000000000000
[ 24.395666] x7 : 0000000000000000 x6 : ffff80087ac00248
[ 24.395866] x5 : 000080086ca54000 x4 : 0000000000002118
[ 24.395966] x3 : eeeeeeeeeeeeeeef x2 : ffff800875e7c918
[ 24.396066] x1 : 0000000000000001 x0 : ffff800875e7ca80
[ 24.396066] Call trace:
[ 24.396166] kvm_timer_irq_can_fire+0xc/0x30
[ 24.396266] kvm_timer_vcpu_load+0xa8/0x1a0
[ 24.396366] kvm_arch_vcpu_load+0xb0/0x1f0
[ 24.396366] kvm_sched_in+0x1c/0x28
[ 24.396466] finish_task_switch+0xd8/0x1d8
[ 24.396566] __schedule+0x248/0x4a0
[ 24.396666] preempt_schedule_irq+0x60/0x90
[ 24.396666] el1_irq+0xd0/0x180
[ 24.396766] kvm_handle_guest_abort+0x0/0x3a0
[ 24.396866] kvm_arch_vcpu_ioctl_run+0x41c/0x688
[ 24.396866] kvm_vcpu_ioctl+0x4c0/0x838
[ 24.397021] do_vfs_ioctl+0xb8/0x878
[ 24.397066] ksys_ioctl+0x84/0x90
[ 24.397166] __arm64_sys_ioctl+0x18/0x28
[ 24.397348] el0_svc_common.constprop.0+0xb0/0x168
[ 24.397366] el0_svc_handler+0x28/0x78
[ 24.397566] el0_svc+0x8/0xc
[ 24.397676] ---[ end trace 37a32293e43ac12f ]---
# KVM compatibility warning.
virtio-9p device was not detected.
While you have requested a virtio-9p device, the guest kernel did not
initialize it.
Please make sure that the guest kernel was compiled with
CONFIG_NET_9P_VIRTIO=y enabled in .config.
# KVM compatibility warning.
virtio-net device was not detected.
While you have requested a virtio-net device, the guest kernel did not
initialize it.
Please make sure that the guest kernel was compiled with CONFIG_VIRTIO_NET=y
enabled in .config.
These are the changes that I made to kvm-unit-tests (the diff can be applied on
top of upstream master, 2130fd4154ad ("tscdeadline_latency: Check condition
first before loop")):
diff --git a/arm/cstart64.S b/arm/cstart64.S
index b0e8baa1a23a..a7631b5a1801 100644
--- a/arm/cstart64.S
+++ b/arm/cstart64.S
@@ -51,6 +51,17 @@ start:
b 1b
1:
+ mrs x4, CurrentEL
+ cmp x4, CurrentEL_EL2
+ b.ne 1f
+ mrs x4, mpidr_el1
+ msr vmpidr_el2, x4
+ mrs x4, midr_el1
+ msr vpidr_el2, x4
+ ldr x4, =(HCR_EL2_TGE | HCR_EL2_E2H)
+ msr hcr_el2, x4
+ isb
+1:
/* set up stack */
mov x4, #1
msr spsel, x4
@@ -101,6 +112,17 @@ get_mmu_off:
.globl secondary_entry
secondary_entry:
+ mrs x0, CurrentEL
+ cmp x0, CurrentEL_EL2
+ b.ne 1f
+ mrs x0, mpidr_el1
+ msr vmpidr_el2, x0
+ mrs x0, midr_el1
+ msr vpidr_el2, x0
+ ldr x0, =(HCR_EL2_TGE | HCR_EL2_E2H)
+ msr hcr_el2, x0
+ isb
+1:
/* Enable FP/ASIMD */
mov x0, #(3 << 20)
msr cpacr_el1, x0
diff --git a/lib/arm/asm/psci.h b/lib/arm/asm/psci.h
index 7b956bf5987d..07297a27e0ce 100644
--- a/lib/arm/asm/psci.h
+++ b/lib/arm/asm/psci.h
@@ -3,6 +3,15 @@
#include <libcflat.h>
#include <linux/psci.h>
+enum psci_conduit {
+ PSCI_CONDUIT_HVC,
+ PSCI_CONDUIT_SMC,
+};
+
+extern void psci_init(void);
+extern void psci_set_conduit(enum psci_conduit conduit);
+extern enum psci_conduit psci_get_conduit(void);
+
extern int psci_invoke(unsigned long function_id, unsigned long arg0,
unsigned long arg1, unsigned long arg2);
extern int psci_cpu_on(unsigned long cpuid, unsigned long entry_point);
diff --git a/lib/arm/psci.c b/lib/arm/psci.c
index c3d399064ae3..20ad4b944738 100644
--- a/lib/arm/psci.c
+++ b/lib/arm/psci.c
@@ -6,13 +6,14 @@
*
* This work is licensed under the terms of the GNU LGPL, version 2.
*/
+#include <devicetree.h>
+#include <string.h>
#include <asm/psci.h>
#include <asm/setup.h>
#include <asm/page.h>
#include <asm/smp.h>
-__attribute__((noinline))
-int psci_invoke(unsigned long function_id, unsigned long arg0,
+static int psci_invoke_hvc(unsigned long function_id, unsigned long arg0,
unsigned long arg1, unsigned long arg2)
{
asm volatile(
@@ -22,6 +23,63 @@ int psci_invoke(unsigned long function_id, unsigned long arg0,
return function_id;
}
+static int psci_invoke_smc(unsigned long function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2)
+{
+ asm volatile(
+ "smc #0"
+ : "+r" (function_id)
+ : "r" (arg0), "r" (arg1), "r" (arg2));
+ return function_id;
+}
+
+/*
+ * Initialize to something sensible, so the exit fallback psci_system_off still
+ * works before calling psci_init when booted at EL1.
+ */
+static enum psci_conduit psci_conduit = PSCI_CONDUIT_HVC;
+static int (*psci_fn)(unsigned long, unsigned long, unsigned long,
+ unsigned long) = &psci_invoke_hvc;
+
+void psci_set_conduit(enum psci_conduit conduit)
+{
+ psci_conduit = conduit;
+ if (conduit == PSCI_CONDUIT_HVC)
+ psci_fn = &psci_invoke_hvc;
+ else
+ psci_fn = &psci_invoke_smc;
+}
+
+enum psci_conduit psci_get_conduit(void)
+{
+ return psci_conduit;
+}
+
+int psci_invoke(unsigned long function_id, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2)
+{
+ return psci_fn(function_id, arg0, arg1, arg2);
+}
+
+void psci_init(void)
+{
+ const char *conduit;
+ int ret;
+
+ ret = dt_get_psci_conduit(&conduit);
+ assert(ret == 0 || ret == -FDT_ERR_NOTFOUND);
+
+ if (ret == -FDT_ERR_NOTFOUND)
+ conduit = "hvc";
+
+ assert(strcmp(conduit, "hvc") == 0 || strcmp(conduit, "smc") == 0);
+
+ if (strcmp(conduit, "hvc") == 0)
+ psci_set_conduit(PSCI_CONDUIT_HVC);
+ else
+ psci_set_conduit(PSCI_CONDUIT_SMC);
+}
+
int psci_cpu_on(unsigned long cpuid, unsigned long entry_point)
{
#ifdef __arm__
diff --git a/lib/arm/setup.c b/lib/arm/setup.c
index 4f02fca85607..e0dc9e4801b0 100644
--- a/lib/arm/setup.c
+++ b/lib/arm/setup.c
@@ -21,6 +21,7 @@
#include <asm/setup.h>
#include <asm/page.h>
#include <asm/smp.h>
+#include <asm/psci.h>
#include "io.h"
@@ -164,7 +165,11 @@ void setup(const void *fdt)
freemem += initrd_size;
}
- /* call init functions */
+ /*
+ * call init functions. psci_init goes first so psci_system_off fallback
+ * works in case of an assert failure
+ */
+ psci_init();
mem_init(PAGE_ALIGN((unsigned long)freemem));
cpu_init();
diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
index 1d9223f728a5..18c5d29ddd1f 100644
--- a/lib/arm64/asm/processor.h
+++ b/lib/arm64/asm/processor.h
@@ -16,6 +16,9 @@
#define SCTLR_EL1_A (1 << 1)
#define SCTLR_EL1_M (1 << 0)
+#define HCR_EL2_TGE (1 << 27)
+#define HCR_EL2_E2H (1 << 34)
+
#ifndef __ASSEMBLY__
#include <asm/ptrace.h>
#include <asm/esr.h>
diff --git a/lib/devicetree.c b/lib/devicetree.c
index 2b89178a109b..4e684c7100b2 100644
--- a/lib/devicetree.c
+++ b/lib/devicetree.c
@@ -263,6 +263,27 @@ int dt_get_bootargs(const char **bootargs)
return 0;
}
+int dt_get_psci_conduit(const char **conduit)
+{
+ const struct fdt_property *prop;
+ int node, len;
+
+ *conduit = NULL;
+
+ node = fdt_node_offset_by_compatible(fdt, -1, "arm,psci-0.2");
+ if (node < 0)
+ return node;
+
+ prop = fdt_get_property(fdt, node, "method", &len);
+ if (!prop)
+ return len;
+ if (len < 4)
+ return -FDT_ERR_NOTFOUND;
+
+ *conduit = prop->data;
+ return 0;
+}
+
int dt_get_default_console_node(void)
{
const struct fdt_property *prop;
diff --git a/lib/devicetree.h b/lib/devicetree.h
index 93c7ebc63bd8..236035eb777d 100644
--- a/lib/devicetree.h
+++ b/lib/devicetree.h
@@ -211,6 +211,15 @@ extern int dt_get_reg(int fdtnode, int regidx, struct
dt_reg *reg);
extern int dt_get_bootargs(const char **bootargs);
/*
+ * dt_get_psci_conduit gets the conduit for PSCI function invocations from
+ * /psci/method
+ * returns
+ * - zero on success
+ * - a negative FDT_ERR_* value on failure, and @conduit will be set to null
+ */
+extern int dt_get_psci_conduit(const char **conduit);
+
+/*
* dt_get_default_console_node gets the node of the path stored in
* /chosen/stdout-path (or the deprecated /chosen/linux,stdout-path)
* returns
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
