On 05/08/2011 11:15 AM, Nadav Har'El wrote:
Hi,
This is the ninth iteration of the nested VMX patch set. This iteration
addresses all of the comments and requests that were raised by reviewers in
the previous rounds, with only a few exception listed below.
Some of the issues which were solved in this version include:
* Overhauled the hardware VMCS (vmcs02) allocation. Previously we had up to
256 vmcs02s, one for each L2. Now we only have one, which is reused.
We also have a compile-time option VMCS02_POOL_SIZE to keep a bigger pool
of vmcs02s. This option will be useful in the future if vmcs02 won't be
filled from scratch on each entry from L1 to L2 (currently, it is).
* The vmcs01 structure, containing a copy of all fields from L1's VMCS, was
unnecessary, as all the necessary values are either known to KVM or appear
in vmcs12. This structure is now gone for good.
* There is no longer a "vmcs_fields" sub-structure that everyone disliked.
All the VMCS fields appear directly in the vmcs12 structure, which makes
the code simpler and more readable.
* Make sure that the vmcs12 fields have fixed sizes and location, and add
some extra padding, to support live migration and improve future-proofing.
* For some fields, nested exit used to fail to return the host-state as set
by L1. Fixed that.
* nested_vmx_exit_handled (deciding if to let L1 handle an exit, or handle it
in L0 and return to L2) is now more correct, and handles more exit reasons.
* Complete overhaul of the cr0, exception bitmap, cr3 and cr4 handling code.
The code is now shorter (uses existing functions like kvm_set_cr3, etc.),
more readable, and more uniform (no pieces of code for enable_ept and not,
less special code for cr0.TS, and none of that ugly cr0.PG monkey-business).
* Use kvm_register_write(), kvm_rip_read(), etc. Got rid of new and now
unneeded function sync_cached_regs_to_vcms().
* Fix return value of the VMX msrs to be more correct, and more constant
(not to needlessly vary on different hosts).
* Added some more missing verifications to vmcs12's fields (cleanly failing
the nested entry if these verifications fail).
* Expose the MSR-bitmap feature to L1. Every MSR access still exits to L0,
but slow exits to L1 are avoided when L1's MSR bitmap doesn't want it.
* Removed or rate limited printouts which could be exploited by guests.
* Fix VM_ENTRY_LOAD_IA32_PAT feature handling.
* Fixed potential bug and verified that nested vmx now works with both
CONFIG_PREEMPT and CONFIG_SMP enabled.
* Dozens of other code cleanups and bug fixes.
Only a few issues from previous reviews remain unaddressed. These are:
* The interrupt injection and IDT_VECTORING_INFO_FIELD handling code was
still not rewritten. It works, though ;-)
* No KVM autotests for nested VMX yet.
* Merging of L0's and L1's MSR bitmaps (and IO bitmaps) is still not
supported. As explained above, the current code uses L1's MSR bitmap
to avoid costly exits to L1, but still suffers exits to L0 on each
MSR access in L2.
* Still no option for disabling some capabilities advertised to L1.
* No support for TPR_SHADOW feature for L1.
This new set of patches applies to the current KVM trunk (I checked with
082f9eced53d50c136e42d072598da4be4b9ba23).
If you wish, you can also check out an already-patched version of KVM from
branch "nvmx9" of the repository:
git://github.com/nyh/kvm-nested-vmx.git
About nested VMX:
-----------------
The following 30 patches implement nested VMX support. This feature enables
a guest to use the VMX APIs in order to run its own nested guests.
In other words, it allows running hypervisors (that use VMX) under KVM.
Multiple guest hypervisors can be run concurrently, and each of those can
in turn host multiple guests.
The theory behind this work, our implementation, and its performance
characteristics were presented in OSDI 2010 (the USENIX Symposium on
Operating Systems Design and Implementation). Our paper was titled
"The Turtles Project: Design and Implementation of Nested Virtualization",
and was awarded "Jay Lepreau Best Paper". The paper is available online, at:
http://www.usenix.org/events/osdi10/tech/full_papers/Ben-Yehuda.pdf
This patch set does not include all the features described in the paper.
In particular, this patch set is missing nested EPT (L1 can't use EPT and
must use shadow page tables). It is also missing some features required to
run VMWare hypervisors as a guest. These missing features will be sent as
follow-on patchs.
Running nested VMX:
------------------
The nested VMX feature is currently disabled by default. It must be
explicitly enabled with the "nested=1" option to the kvm-intel module.
No modifications are required to user space (qemu). However, qemu's default
emulated CPU type (qemu64) does not list the "VMX" CPU feature, so it must be
explicitly enabled, by giving qemu one of the following options:
-cpu host (emulated CPU has all features of the real CPU)
-cpu qemu64,+vmx (add just the vmx feature to a named CPU type)
This version was only tested with KVM (64-bit) as a guest hypervisor, and
Linux as a nested guest.
Okay, truly excellent. The code is now a lot more readable, and I'm
almost beginning to understand it. The code comments are also very
good, I wish we had the same quality comments in the rest of kvm. We
can probably merge the next iteration if there aren't significant
comments from others.
The only worrying thing is the issue you raise in patch 8. Is there a
simple fix you can push that addresses correctness?
--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
