Several people have reported problems with rebooting ARM VMs, especially on 32-bit ARM. This is mainly due to the same reason we were seeing boot errors in the past, namely that the ram, dcache, and icache weren't coherent on guest boot with the guest (stage-1) MMU disabled. We solved this by ensuring coherency when we fault in pages, but since most memory is already mapped after a reboot, we don't do anything. The solution is to unmap the regular RAM on VCPU init, but we must take care to not unmap the GIC or other IO regions, hence the somehwat complicated solution. As part of figuring this out, it became clear that some semantics around the KVM_ARM_VCPU_INIT ABI and system event ABI was unclear (what is userspace expected to do when it receives a system event). This series also clarifies the ABI and changes the kernel functionality to do what userspace expects (turn off VCPUs on a system shutdown event). The code is avaliable here as well: http://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git vcpu_init_fixes-v2 There is an alternative version with more code-reuse for the unmapping implementation for the previous version of this patch series available in the following git repo: http://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git vcpu_init_fixes-alternative Testing ------- This has been tested on CubieBoard, Arndale, TC2, and Juno. On Arndale and TC2 it was extremely easy to reproduce the problem (just start a VM that runs reboot from /etc/rc.local or similar) and this series clearly fixes the behavior. For the previous version of this series, I was seeing some problems on Juno, but it turned out to be because I wasn't limiting my testing to one of the clusters, and since we don't support re-initing a VCPU on a different physical host CPU (big.LITTLE), it was failing. For this version of the patch series, it has been running a reboot loop on Juno for hours. Changelog --------- Changes v1->v2: - New patch to not clear the VCPU_POWER_OFF flag - Fixed spelling error in commit message - Adapted ABI texts based on Peter's feedback - Check for changed parameters to KVM_ARM_VCPU_INIT - Now unmap the Stage-2 RAM mappings at VCPU init instead of at PSCI system event time. Christoffer Dall (6): arm/arm64: KVM: Don't clear the VCPU_POWER_OFF flag arm/arm64: KVM: Correct KVM_ARM_VCPU_INIT power off option arm/arm64: KVM: Reset the HCR on each vcpu when resetting the vcpu arm/arm64: KVM: Clarify KVM_ARM_VCPU_INIT ABI arm/arm64: KVM: Turn off vcpus on PSCI shutdown/reboot arm/arm64: KVM: Introduce stage2_unmap_vm Documentation/virtual/kvm/api.txt | 17 +++++++++- arch/arm/include/asm/kvm_emulate.h | 5 +++ arch/arm/include/asm/kvm_host.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 1 + arch/arm/kvm/arm.c | 56 ++++++++++++++++++++++++++++++- arch/arm/kvm/guest.c | 26 --------------- arch/arm/kvm/mmu.c | 65 ++++++++++++++++++++++++++++++++++++ arch/arm/kvm/psci.c | 19 +++++++++++ arch/arm64/include/asm/kvm_emulate.h | 5 +++ arch/arm64/include/asm/kvm_host.h | 3 +- arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/kvm/guest.c | 26 --------------- 12 files changed, 168 insertions(+), 58 deletions(-) -- 2.1.2.330.g565301e.dirty _______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
