This patch series add support for stage2 hardware DBM, and it is only used for dirty log for now. It works well under some migration test cases, including VM with 4K pages or 2M THP. I checked the SHA256 hash digest of all memory and they keep same for source VM and destination VM, which means no dirty pages is missed under hardware DBM. However, there are some known issues not solved. 1. Some mechanisms that rely on "write permission fault" become invalid, such as kvm_set_pfn_dirty and "mmap page sharing". kvm_set_pfn_dirty is called in user_mem_abort when guest issues write fault. This guarantees physical page will not be dropped directly when host kernel recycle memory. After using hardware dirty management, we have no chance to call kvm_set_pfn_dirty. For "mmap page sharing" mechanism, host kernel will allocate a new physical page when guest writes a page that is shared with other page table entries. After using hardware dirty management, we have no chance to do this too. I need to do some survey on how stage1 hardware DBM solve these problems. It helps if anyone can figure it out. 2. Page Table Modification Races: Though I have found and solved some data races when kernel changes page table entries, I still doubt that there are data races I am not aware of. It's great if anyone can figure them out. 3. Performance: Under Kunpeng 920 platform, for every 64GB memory, KVM consumes about 40ms to traverse all PTEs to collect dirty log. It will cause unbearable downtime for migration if memory size is too big. I will try to solve this problem in Patch v1. Keqian Zhu (7): KVM: arm64: Add some basic functions for hw DBM KVM: arm64: Set DBM bit of PTEs if hw DBM enabled KVM: arm64: Traverse page table entries when sync dirty log KVM: arm64: Steply write protect page table by mask bit kvm: arm64: Modify stage2 young mechanism to support hw DBM kvm: arm64: Save stage2 PTE dirty info if it is coverred KVM: arm64: Enable stage2 hardware DBM arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/kvm_mmu.h | 44 +++++- arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/sysreg.h | 2 + arch/arm64/kvm/reset.c | 9 +- virt/kvm/arm/arm.c | 6 +- virt/kvm/arm/mmu.c | 202 ++++++++++++++++++++++++-- 7 files changed, 246 insertions(+), 19 deletions(-) -- 2.19.1
