On Thu, Jul 21, 2022 at 2:56 AM Fuad Tabba <tabba@xxxxxxxxxx> wrote: > > Hi Kalesh, > > > On Thu, Jul 21, 2022 at 6:57 AM Kalesh Singh <kaleshsingh@xxxxxxxxxx> wrote: > > > > Hi all, > > > > This is v5 of nVHE hypervisor stacktraces support. The series is based on > > arm64 for-next/stacktrace. > > > > The previous versions were posted at: > > v4: https://lore.kernel.org/r/20220715061027.1612149-1-kaleshsingh@xxxxxxxxxx/ > > v3: https://lore.kernel.org/r/20220607165105.639716-1-kaleshsingh@xxxxxxxxxx/ > > v2: https://lore.kernel.org/r/20220502191222.4192768-1-kaleshsingh@xxxxxxxxxx/ > > v1: https://lore.kernel.org/r/20220427184716.1949239-1-kaleshsingh@xxxxxxxxxx/ > > > > The main updates in this version are some refactoring to move stuff out of > > stacktrace/nvhe.h (leaving only the unwinder implementation in the header); > > and fixing the symbolization of the hyp stacktrace when KASLR is enabled; > > along with the addressing the other minor comments. > > > > Patch 18 (KVM: arm64: Dump nVHE hypervisor stack on panic) was also squashed > > into earlier patches. > > > > The previous cover letter is copied below for convenience. > > > > Thanks all for your feedback. > > Thank you for this. This will be very helpful. > > For the whole series: > Tested-by: Fuad Tabba <tabba@xxxxxxxxxx> Thanks for your reviews, Fuad. --Kalesh > > Cheers, > /fuad > > > > > > --Kalesh > > > > ============ > > > > KVM nVHE Stack unwinding. > > === > > > > nVHE has two modes of operation: protected (pKVM) and unprotected > > (conventional nVHE). Depending on the mode, a slightly different approach > > is used to dump the hypervisor stacktrace but the core unwinding logic > > remains the same. > > > > Protected nVHE (pKVM) stacktraces > > ==== > > > > In protected nVHE mode, the host cannot directly access hypervisor memory. > > > > The hypervisor stack unwinding happens in EL2 and is made accessible to > > the host via a shared buffer. Symbolizing and printing the stacktrace > > addresses is delegated to the host and happens in EL1. > > > > Non-protected (Conventional) nVHE stacktraces > > ==== > > > > In non-protected mode, the host is able to directly access the hypervisor > > stack pages. > > > > The hypervisor stack unwinding and dumping of the stacktrace is performed > > by the host in EL1, as this avoids the memory overhead of setting up > > shared buffers between the host and hypervisor. > > > > Resuing the Core Unwinding Logic > > ==== > > > > Since the hypervisor cannot link against the kernel code in proteced mode. > > The common stack unwinding code is moved to a shared header to allow reuse > > in the nVHE hypervisor. > > > > Reducing the memory footprint > > ==== > > > > In this version the below steps were taken to reduce the memory usage of > > nVHE stack unwinding: > > > > 1) The nVHE overflow stack is reduced from PAGE_SIZE to 4KB; benificial > > for configurations with non 4KB pages (16KB or 64KB pages). > > 2) In protected nVHE mode (pKVM), the shared stacktrace buffers with the > > host are reduced from PAGE_SIZE to the minimum size required. > > 3) In systems other than Android, conventional nVHE makes up the vast > > majority of use case. So the pKVM stack tracing is disabled by default > > (!CONFIG_PROTECTED_NVHE_STACKTRACE), which avoid the memory usage for > > setting up shared buffers. > > 4) In non-protected nVHE mode (conventional nVHE), the stack unwinding > > is done directly in EL1 by the host and no shared buffers with the > > hypervisor are needed. > > > > Sample Output > > ==== > > > > The below shows an example output from a simple stack overflow test: > > > > [ 126.862960] kvm [371]: nVHE hyp panic at: [<ffff8000090a51d0>] __kvm_nvhe_recursive_death+0x10/0x34! > > [ 126.869920] kvm [371]: Protected nVHE HYP call trace: > > [ 126.870528] kvm [371]: [<ffff8000090a5570>] __kvm_nvhe_hyp_panic+0xac/0xf8 > > [ 126.871342] kvm [371]: [<ffff8000090a55cc>] __kvm_nvhe_hyp_panic_bad_stack+0x10/0x10 > > [ 126.872174] kvm [371]: [<ffff8000090a51e4>] __kvm_nvhe_recursive_death+0x24/0x34 > > [ 126.872971] kvm [371]: [<ffff8000090a51e4>] __kvm_nvhe_recursive_death+0x24/0x34 > > . . . > > > > [ 126.927314] kvm [371]: [<ffff8000090a51e4>] __kvm_nvhe_recursive_death+0x24/0x34 > > [ 126.927727] kvm [371]: [<ffff8000090a51e4>] __kvm_nvhe_recursive_death+0x24/0x34 > > [ 126.928137] kvm [371]: [<ffff8000090a4de4>] __kvm_nvhe___kvm_vcpu_run+0x30/0x40c > > [ 126.928561] kvm [371]: [<ffff8000090a7b64>] __kvm_nvhe_handle___kvm_vcpu_run+0x30/0x48 > > [ 126.928984] kvm [371]: [<ffff8000090a78b8>] __kvm_nvhe_handle_trap+0xc4/0x128 > > [ 126.929385] kvm [371]: [<ffff8000090a6864>] __kvm_nvhe___host_exit+0x64/0x64 > > [ 126.929804] kvm [371]: ---- End of Protected nVHE HYP call trace ---- > > > > ============ > > > > Kalesh Singh (17): > > arm64: stacktrace: Add shared header for common stack unwinding code > > arm64: stacktrace: Factor out on_accessible_stack_common() > > arm64: stacktrace: Factor out unwind_next_common() > > arm64: stacktrace: Handle frame pointer from different address spaces > > arm64: stacktrace: Factor out common unwind() > > arm64: stacktrace: Add description of stacktrace/common.h > > KVM: arm64: On stack overflow switch to hyp overflow_stack > > KVM: arm64: Add PROTECTED_NVHE_STACKTRACE Kconfig > > KVM: arm64: Allocate shared pKVM hyp stacktrace buffers > > KVM: arm64: Stub implementation of pKVM HYP stack unwinder > > KVM: arm64: Stub implementation of non-protected nVHE HYP stack > > unwinder > > KVM: arm64: Save protected-nVHE (pKVM) hyp stacktrace > > KVM: arm64: Prepare non-protected nVHE hypervisor stacktrace > > KVM: arm64: Implement protected nVHE hyp stack unwinder > > KVM: arm64: Implement non-protected nVHE hyp stack unwinder > > KVM: arm64: Introduce pkvm_dump_backtrace() > > KVM: arm64: Introduce hyp_dump_backtrace() > > > > arch/arm64/include/asm/kvm_asm.h | 16 ++ > > arch/arm64/include/asm/memory.h | 8 + > > arch/arm64/include/asm/stacktrace.h | 92 +++++---- > > arch/arm64/include/asm/stacktrace/common.h | 230 +++++++++++++++++++++ > > arch/arm64/include/asm/stacktrace/nvhe.h | 199 ++++++++++++++++++ > > arch/arm64/kernel/stacktrace.c | 157 -------------- > > arch/arm64/kvm/Kconfig | 15 ++ > > arch/arm64/kvm/arm.c | 2 +- > > arch/arm64/kvm/handle_exit.c | 101 +++++++++ > > arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- > > arch/arm64/kvm/hyp/nvhe/host.S | 9 +- > > arch/arm64/kvm/hyp/nvhe/stacktrace.c | 116 +++++++++++ > > arch/arm64/kvm/hyp/nvhe/switch.c | 6 + > > 13 files changed, 749 insertions(+), 204 deletions(-) > > create mode 100644 arch/arm64/include/asm/stacktrace/common.h > > create mode 100644 arch/arm64/include/asm/stacktrace/nvhe.h > > create mode 100644 arch/arm64/kvm/hyp/nvhe/stacktrace.c > > > > > > base-commit: 82a592c13b0aeff94d84d54183dae0b26384c95f > > -- > > 2.37.0.170.g444d1eabd0-goog > > _______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
