The following series implements KVM support for ARM processors, specifically on the Cortex A-15 platform. Work is done in collaboration between Columbia University, Virtual Open Systems and ARM/Linaro. The patch series applies to kvm/next, specifically commit: ae7a2a3fb6f8b784c2752863f4f1f20c656f76fb This is Version 9 of the patch series, but the first two versions were reviewed outside of the KVM mailing list. Changes can also be pulled from: git://github.com/virtualopensystems/linux-kvm-arm.git kvm-a15-v9 A non-flattened edition of the patch series can be found at: git://github.com/virtualopensystems/linux-kvm-arm.git kvm-a15-v9-stage The implementation is broken up into a logical set of patches, the first five are preparatory patches: 1. ARM: Add mem_type prot_pte accessor 2. ARM: ARM_VIRT_EXT config option 3. ARM: Section based HYP idmaps 4. KVM: Move KVM_IRQ_LINE to arch-generic code 5. KVM: Guard code with CONFIG_MMU_NOTIFIER (repost) KVM guys, please consider pulling the KVM generic patches as early as possible. Thanks. The main implementation is broken up into separate patches, the first containing a skeleton of files, makefile changes, the basic user space interface and KVM architecture specific stubs. Subsequent patches implement parts of the system as listed: 1. Skeleton 2. Reset values for the Cortex-A15 type processor 3. Hypervisor initialization 4. Hypervisor module unloading 5. Memory virtualization setup (hyp mode mappings and 2nd stage) 6. Inject IRQs and FIQs from userspace 7. World-switch implementation and Hyp exception vectors 8. Emulation framework and CP15 emulation 9. Handle guest user memory aborts 10. Handle guest MMIO aborts 11. Support guest wait-for-interrupt instructions Testing: Limited testing, but have run GCC inside guest, which compiled a small hello-world program, which was successfully run. For v9 both ARM/Thumb-2 kernels were tested as both host/guest and both a compiled-in version and a kernel module version of KVM was tested. Hardware still unavailable to me, so all testing has been done on ARM Fast Models. For a guide on how to set up a testing environment and try out these patches, see: http://www.virtualopensystems.com/media/pdf/kvm-arm-guide.pdf There is an issue list available using the issue tracker on: https://github.com/virtualopensystems/linux-kvm-arm Additionally a few major milestones are coming up shortly: - Support Thumb MMIO emulation and test MMIO emulation code (under way) - Merge Marc Zyngier's patch series for VGIC and timers (review in progress) - Change from SMC based install to relying on booting the kernel in Hyp mode. This requires some larger changes, but will allow a guest kernel to boot with KVM configured. Changes since v8: - Support cache maintenance on SMP through set/way - Hyp mode idmaps are now section based and happen at kernel init - Handle aborts in Hyp mode - Inject undefined exceptions into the guest on error - Kernel-side reset of all crucial registers - Specifically state which target CPU is being virtualized - Exit statistics in debugfs - Some L2CTLR cp15 emulation cleanups - Support spte_hva for MMU notifiers and take write faults - FIX: Race condition in VMID generation - BUG: Run exit handling code with disabled preemption - Save/Restore abort fault register during world switch Changes since v7: - Traps accesses to ACTLR - Do not trap WFE execution - Upgrade barriers and TLB operations to inner-shareable domain - Restrucure hyp_pgd related code to be more opaque - Random SMP fixes - Random BUG fixes - Improve commenting - Support module loading/unloading of KVM/ARM - Thumb-2 support for host kernel and KVM - Unaligned cross-page wide guest Thumb instruction fetching - Support ITSTATE fields in CPSR for Thumb guests - Document HCR settings Changes since v6: - Support for MMU notifiers to not pin user pages in memory - Suport build with log debugging - Bugfix: v6 clobbered r7 in init code - Simplify hyp code mapping - Cleanup of register access code - Table-based CP15 emulation from Rusty Russell - Various other bug fixes and cleanups Changes since v5: - General bugfixes and nit fixes from reviews - Implemented re-use of VMIDs - Cleaned up the Hyp-mapping code to be readable by non-mm hackers (including myself) - Integrated preliminary SMP support in base patches - Lock-less interrupt injection and WFI support - Fixed signal-handling in while in guest (increases overall stability) Changes since v4: - Addressed reviewer comments from v4 * cleanup debug and trace code * remove printks * fixup kvm_arch_vcpu_ioctl_run * add trace details to mmio emulation - Fix from Marc Zyngier: Move kvm_guest_enter/exit into non-preemptible section (squashed into world-switch patch) - Cleanup create_hyp_mappings/remove_hyp_mappings from Marc Zyngier (squashed into hypervisor initialization patch) - Removed the remove_hyp_mappings feature. Removing hypervisor mappings could potentially unmap other important data shared in the same page. - Removed the arm_ prefix from the arch-specific files. - Initial SMP host/guest support Changes since v3: - v4 actually works, fully boots a guest - Support compiling as a module - Use static inlines instead of macros for vcpu_reg and friends - Optimize kvm_vcpu_reg function - Use Ftrace for trace capabilities - Updated documentation and commenting - Use KVM_IRQ_LINE instead of KVM_INTERRUPT - Emulates load/store instructions not supported through HSR syndrome information. - Frees 2nd stage translation tables on VM teardown - Handles IRQ/FIQ instructions - Handles more CP15 accesses - Support guest WFI calls - Uses debugfs instead of /proc - Support compiling in Thumb mode Changes since v2: - Performs world-switch code - Maps guest memory using 2nd stage translation - Emulates co-processor 15 instructions - Forwards I/O faults to QEMU. --- Christoffer Dall (14): ARM: Add config option ARM_VIRT_EXT ARM: Section based HYP idmap KVM: Move KVM_IRQ_LINE to arch-generic code ARM: KVM: Initial skeleton to compile KVM support ARM: KVM: Support Cortex-A15 VCPUs reset ARM: KVM: Hypervisor inititalization ARM: KVM: Module unloading support ARM: KVM: Memory virtualization setup ARM: KVM: Inject IRQs and FIQs from userspace ARM: KVM: World-switch implementation ARM: KVM: Emulation framework and CP15 emulation ARM: KVM: Handle guest faults in KVM ARM: KVM: Handle I/O aborts ARM: KVM: Guest wait-for-interrupts (WFI) support Marc Zyngier (2): ARM: add mem_type prot_pte accessor KVM: Guard mmu_notifier specific code with CONFIG_MMU_NOTIFIER Documentation/virtual/kvm/api.txt | 12 arch/arm/Kconfig | 2 arch/arm/Makefile | 1 arch/arm/include/asm/idmap.h | 7 arch/arm/include/asm/kvm.h | 89 +++ arch/arm/include/asm/kvm_arm.h | 184 ++++++ arch/arm/include/asm/kvm_asm.h | 58 ++ arch/arm/include/asm/kvm_emulate.h | 120 ++++ arch/arm/include/asm/kvm_host.h | 159 +++++ arch/arm/include/asm/kvm_mmu.h | 41 + arch/arm/include/asm/mach/map.h | 1 arch/arm/include/asm/pgtable-3level-hwdef.h | 5 arch/arm/include/asm/pgtable-3level.h | 13 arch/arm/include/asm/pgtable.h | 5 arch/arm/include/asm/unified.h | 12 arch/arm/kernel/armksyms.c | 7 arch/arm/kernel/asm-offsets.c | 43 + arch/arm/kernel/entry-armv.S | 1 arch/arm/kernel/vmlinux.lds.S | 6 arch/arm/kvm/Kconfig | 45 + arch/arm/kvm/Makefile | 17 + arch/arm/kvm/arm.c | 881 +++++++++++++++++++++++++++ arch/arm/kvm/emulate.c | 873 +++++++++++++++++++++++++++ arch/arm/kvm/exports.c | 38 + arch/arm/kvm/guest.c | 165 +++++ arch/arm/kvm/init.S | 149 +++++ arch/arm/kvm/interrupts.S | 698 +++++++++++++++++++++ arch/arm/kvm/mmu.c | 644 ++++++++++++++++++++ arch/arm/kvm/reset.c | 133 ++++ arch/arm/kvm/trace.h | 117 ++++ arch/arm/mm/Kconfig | 10 arch/arm/mm/idmap.c | 88 ++- arch/arm/mm/mmu.c | 9 arch/ia64/kvm/kvm-ia64.c | 33 - arch/x86/kvm/x86.c | 33 - include/linux/kvm.h | 1 include/linux/kvm_host.h | 5 mm/memory.c | 2 virt/kvm/kvm_main.c | 19 + 39 files changed, 4661 insertions(+), 65 deletions(-) create mode 100644 arch/arm/include/asm/kvm.h create mode 100644 arch/arm/include/asm/kvm_arm.h create mode 100644 arch/arm/include/asm/kvm_asm.h create mode 100644 arch/arm/include/asm/kvm_emulate.h create mode 100644 arch/arm/include/asm/kvm_host.h create mode 100644 arch/arm/include/asm/kvm_mmu.h create mode 100644 arch/arm/kvm/Kconfig create mode 100644 arch/arm/kvm/Makefile create mode 100644 arch/arm/kvm/arm.c create mode 100644 arch/arm/kvm/emulate.c create mode 100644 arch/arm/kvm/exports.c create mode 100644 arch/arm/kvm/guest.c create mode 100644 arch/arm/kvm/init.S create mode 100644 arch/arm/kvm/interrupts.S create mode 100644 arch/arm/kvm/mmu.c create mode 100644 arch/arm/kvm/reset.c create mode 100644 arch/arm/kvm/trace.h --
