On Thu, 2018-04-12 at 16:59 +0200, Paolo Bonzini wrote: > On 21/02/2018 18:47, KarimAllah Ahmed wrote: > > > > For the most part, KVM can handle guest memory that does not have a struct > > page (i.e. not directly managed by the kernel). However, There are a few places > > in the code, specially in the nested code, that does not support that. > > > > Patch 1, 2, and 3 avoid the mapping and unmapping all together and just > > directly use kvm_guest_read and kvm_guest_write. > > > > Patch 4 introduces a new guest mapping interface that encapsulate all the > > bioler plate code that is needed to map and unmap guest memory. It also > > supports guest memory without "struct page". > > > > Patch 5, 6, 7, 8, 9, and 10 switch most of the offending code in VMX and hyperv > > to use the new guest mapping API. > > > > This patch series is the first set of fixes. Handling SVM and APIC-access page > > will be handled in a different patch series. > > I like the patches and the new API. However, I'm a bit less convinced > about the caching aspect; keeping a page pinned is not the nicest thing > with respect (for example) to memory hot-unplug. > > Since you're basically reinventing kmap_high, or alternatively > (depending on your background) xc_map_foreign_pages, it's not surprising > that memremap is slow. How slow is it really (as seen e.g. with > vmexit.flat running in L1, on EC2 compared to vanilla KVM)? I have not actually compared EC2 vs vanilla but I compared between the version with cached vs no-cache (both in EC2 setup). The one that cached the mappings was an order of magnitude better. For example, booting an Ubuntu L2 guest with QEMU took around 10-13 seconds with this caching and it took over 5 minutes without the caching. I will test with vanilla KVM and post the results. > > Perhaps you can keep some kind of per-CPU cache of the last N remapped > pfns? This cache would sit between memremap and __kvm_map_gfn and it > would be completely transparent to the layer below since it takes raw > pfns. This removes the need to store the memslots generation etc. (If > you go this way please place it in virt/kvm/pfncache.[ch], since > kvm_main.c is already way too big). Yup, that sounds like a good idea. I actually already implemented some sort of a per-CPU mapping pool in order to reduce the overhead when the vCPU is over-committed. I will clean this and post as you suggested. > > Thanks, > > Paolo > > > > > KarimAllah Ahmed (10): > > X86/nVMX: handle_vmon: Read 4 bytes from guest memory instead of > > map->read->unmap sequence > > X86/nVMX: handle_vmptrld: Copy the VMCS12 directly from guest memory > > instead of map->copy->unmap sequence. > > X86/nVMX: Update the PML table without mapping and unmapping the page > > KVM: Introduce a new guest mapping API > > KVM/nVMX: Use kvm_vcpu_map when mapping the L1 MSR bitmap > > KVM/nVMX: Use kvm_vcpu_map when mapping the virtual APIC page > > KVM/nVMX: Use kvm_vcpu_map when mapping the posted interrupt > > descriptor table > > KVM/X86: Use kvm_vcpu_map in emulator_cmpxchg_emulated > > KVM/X86: hyperv: Use kvm_vcpu_map in synic_clear_sint_msg_pending > > KVM/X86: hyperv: Use kvm_vcpu_map in synic_deliver_msg > > > > arch/x86/kvm/hyperv.c | 28 ++++----- > > arch/x86/kvm/vmx.c | 144 +++++++++++++++-------------------------------- > > arch/x86/kvm/x86.c | 13 ++--- > > include/linux/kvm_host.h | 15 +++++ > > virt/kvm/kvm_main.c | 50 ++++++++++++++++ > > 5 files changed, 129 insertions(+), 121 deletions(-) > > > > Amazon Development Center Germany GmbH Berlin - Dresden - Aachen main office: Krausenstr. 38, 10117 Berlin Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger Ust-ID: DE289237879 Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
