This series intends to virtualize Software Delegated Exception Interface (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to deliver NMI-alike SDEI event to guest and it's needed by Async PF to deliver page-not-present notification from hypervisor to guest. The code and the required qemu changes can be found from: https://developer.arm.com/documentation/den0054/c https://github.com/gwshan/linux ("kvm/arm64_sdei") https://github.com/gwshan/qemu ("kvm/arm64_sdei") The design is quite strightforward by following the specification. The (SDEI) events are classified into the shared and private ones according to their scope. The shared event is system or VM scoped, but the private event is vcpu scoped. This implementation doesn't support the shared event because all the needed events are private. Besides, the critial events aren't supported by the implementation either. It means all events are normal in terms of priority. There are several objects (data structures) introduced to help on the event registration, enablement, disablement, unregistration, reset, delivery and handling. * kvm_sdei_event_handler SDEI event handler, which is provided through EVENT_REGISTER hypercall, is called when the SDEI event is delivered from host to guest. * kvm_sdei_event_context The saved (preempted) context when SDEI event is delivered for handling. * kvm_sdei_vcpu SDEI events and their states. The patches are organized as below: PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor hypercall routing mechanism PATCH[03] Adds SDEI virtualization infrastructure PATCH[04-16] Supports various SDEI hypercalls and event handling PATCH[17] Exposes SDEI capability PATCH[18-19] Support SDEI migration PATCH[20] Adds document about SDEI PATCH[21-22] SDEI related selftest cases The previous revisions can be found: v6: https://lore.kernel.org/lkml/20220403153911.12332-4-gshan@xxxxxxxxxx/T/ v5: https://lore.kernel.org/kvmarm/20220322080710.51727-1-gshan@xxxxxxxxxx/ v4: https://lore.kernel.org/kvmarm/20210815001352.81927-1-gshan@xxxxxxxxxx/ v3: https://lore.kernel.org/kvmarm/20210507083124.43347-1-gshan@xxxxxxxxxx/ v2: https://lore.kernel.org/kvmarm/20210209032733.99996-1-gshan@xxxxxxxxxx/ v1: https://lore.kernel.org/kvmarm/20200817100531.83045-1-gshan@xxxxxxxxxx/ Testing ======= [1] The selftest case included in this series works fine. The default SDEI event, whose number is zero, can be registered, enabled, raised. The SDEI event handler can be invoked. [host]# pwd /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm [root@virtlab-arm01 kvm]# ./aarch64/sdei NR_VCPUS: 2 SDEI Event: 0x00000000 --- VERSION Version: 1.1 (vendor: 0x4b564d) --- FEATURES Shared event slots: 0 Private event slots: 0 Relative mode: No --- PRIVATE_RESET --- SHARED_RESET --- PE_UNMASK --- EVENT_GET_INFO Type: Private Priority: Normal Signaled: Yes --- EVENT_REGISTER --- EVENT_ENABLE --- EVENT_SIGNAL Handled: Yes IRQ: No Status: Registered-Enabled-Running PC/PSTATE: 000000000040232c 00000000600003c5 Regs: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 --- PE_MASK --- EVENT_DISABLE --- EVENT_UNREGISTER Result: OK [2] There are additional patches in the following repositories to create procfs entries, allowing to inject SDEI event from host side. The SDEI client in the guest side registers the SDEI default event, whose number is zero. Also, the QEMU exports SDEI ACPI table and supports migration for SDEI. https://github.com/gwshan/linux ("kvm/arm64_sdei") https://github.com/gwshan/qemu ("kvm/arm64_sdei") [2.1] Start the guests and migrate the source VM to the destination VM. [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \ -accel kvm -machine virt,gic-version=host \ -cpu host -smp 6,sockets=2,cores=3,threads=1 \ -m 1024M,slots=16,maxmem=64G \ : \ -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \ -initrd /home/gavin/sandbox/images/rootfs.cpio.xz \ -append earlycon=pl011,mmio,0x9000000 \ : [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \ -accel kvm -machine virt,gic-version=host \ -cpu host -smp 6,sockets=2,cores=3,threads=1 \ -m 1024M,slots=16,maxmem=64G \ : \ -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \ -initrd /home/gavin/sandbox/images/rootfs.cpio.xz \ -append earlycon=pl011,mmio,0x9000000 \ -incoming tcp:0:4444 \ : [2.2] Check kernel log on the source VM. The SDEI service is enabled and the default SDEI event (0x0) is enabled. [guest-src]# dmesg | grep -i sdei ACPI: SDEI 0x000000005BC80000 000024 \ (v00 BOCHS BXPC 00000001 BXPC 00000001) sdei: SDEIv1.1 (0x4b564d) detected in firmware. SDEI TEST: Version 1.1, Vendor 0x4b564d sdei_init: SDEI event (0x0) registered sdei_init: SDEI event (0x0) enabled (qemu) migrate -d tcp:localhost:4444 [2.3] Migrate the source VM to the destination VM. Inject SDEI event to the destination VM. The event is raised and handled. (qemu) migrate -d tcp:localhost:4444 [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1 [guest-dst]# =========== SDEI Event (CPU#1) =========== Event: 0000000000000000 Parameter: 00000000dabfdabf PC: ffff800008cbb554 PSTATE: 00000000604000c5 SP: ffff800009c7bde0 Regs: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 ffff800016c28000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 ffff800009399008 ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18 0000000000000000 0000000000000000 ffff000000339d00 0000000000000000 0000000000000000 ffff800009c7bde0 ffff800008cbb5c4 Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001 ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190 ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000 Changelog ========= v7: * Rebased to v5.19.rc1 (Gavin) * Add hypercall ranges for routing (Oliver) * Remove support to the critical event and redesigned data structures. Function names are also modified as Oliver suggested (Oliver) * Deliver event when it's enabled or the specific PE is unmasked (Oliver) * Improve EVENT_COMPLETE_AND_RESUME hypercall to resume from the specified address (Oliver) * Add patches for SDEI migration and documentation (Gavin) * Misc comments from Oliver Upon (Oliver) v6: * Rebased to v5.18.rc1 (Gavin) * Pass additional argument to smccc_get_arg() (Oliver) * Add preparatory patch to route hypercalls based on their owners (Oliver) * Remove the support for shared event. (Oliver/Gavin) * Remove the support for migration and add-on patches to support it in future (Oliver) * The events are exposed by KVM instead of VMM (Oliver) * kvm_sdei_state.h is dropped and all the structures are folded into the corresponding ones in kvm_sdei.h (Oliver) * Rename 'struct kvm_sdei_registered_event' to 'struct kvm_sdei_event' (Oliver) * Misc comments from Oliver Upon (Oliver) v5/v4/v3/v2/v1: * Skipped here and please visit the history by https://lore.kernel.org/lkml/20220403153911.12332-4-gshan@xxxxxxxxxx/T/ Gavin Shan (22): KVM: arm64: Extend smccc_get_argx() KVM: arm64: Route hypercalls based on their owner KVM: arm64: Add SDEI virtualization infrastructure KVM: arm64: Support EVENT_REGISTER hypercall KVM: arm64: Support EVENT_{ENABLE, DISABLE} hypercall KVM: arm64: Support EVENT_CONTEXT hypercall KVM: arm64: Support EVENT_UNREGISTER hypercall KVM: arm64: Support EVENT_STATUS hypercall KVM: arm64: Support EVENT_GET_INFO hypercall KVM: arm64: Support PE_{MASK, UNMASK} hypercall KVM: arm64: Support {PRIVATE, SHARED}_RESET hypercall KVM: arm64: Support event injection and delivery KVM: arm64: Support EVENT_{COMPLETE, COMPLETE_AND_RESUME} hypercall KVM: arm64: Support EVENT_SIGNAL hypercall KVM: arm64: Support SDEI_FEATURES hypercall KVM: arm64: Support SDEI_VERSION hypercall KVM: arm64: Expose SDEI capbility and service KVM: arm64: Allow large sized pseudo firmware registers KVM: arm64: Support SDEI event migration KVM: arm64: Add SDEI document selftests: KVM: aarch64: Add SDEI case in hypercall tests selftests: KVM: aarch64: Add SDEI test case Documentation/virt/kvm/api.rst | 11 + Documentation/virt/kvm/arm/hypercalls.rst | 4 + Documentation/virt/kvm/arm/sdei.rst | 64 ++ arch/arm64/include/asm/kvm_host.h | 3 + arch/arm64/include/asm/kvm_sdei.h | 81 +++ arch/arm64/include/uapi/asm/kvm.h | 18 + arch/arm64/kvm/Makefile | 2 +- arch/arm64/kvm/arm.c | 8 + arch/arm64/kvm/hypercalls.c | 182 +++-- arch/arm64/kvm/psci.c | 14 +- arch/arm64/kvm/pvtime.c | 2 +- arch/arm64/kvm/sdei.c | 676 ++++++++++++++++++ arch/arm64/kvm/trng.c | 4 +- include/kvm/arm_hypercalls.h | 19 +- include/linux/arm-smccc.h | 7 + include/uapi/linux/arm_sdei.h | 8 + include/uapi/linux/kvm.h | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/aarch64/hypercalls.c | 11 +- tools/testing/selftests/kvm/aarch64/sdei.c | 450 ++++++++++++ 20 files changed, 1499 insertions(+), 67 deletions(-) create mode 100644 Documentation/virt/kvm/arm/sdei.rst create mode 100644 arch/arm64/include/asm/kvm_sdei.h create mode 100644 arch/arm64/kvm/sdei.c create mode 100644 tools/testing/selftests/kvm/aarch64/sdei.c -- 2.23.0 _______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
