Roman Kagan <rkagan@xxxxxxxxxxxxx> writes: > On Wed, Nov 13, 2019 at 10:29:00AM +0100, Vitaly Kuznetsov wrote: >> Roman Kagan <rkagan@xxxxxxxxxxxxx> writes: >> > On Tue, Nov 12, 2019 at 11:34:27AM +0800, lantianyu1986@xxxxxxxxx wrote: >> >> From: Tianyu Lan <Tianyu.Lan@xxxxxxxxxxxxx> >> >> >> >> Hyper-V direct tlb flush targets KVM on Hyper-V guest. >> >> Enable direct TLB flush for its guests meaning that TLB >> >> flush hypercalls are handled by Level 0 hypervisor (Hyper-V) >> >> bypassing KVM in Level 1. Due to the different ABI for hypercall >> >> parameters between Hyper-V and KVM, KVM capabilities should be >> >> hidden when enable Hyper-V direct tlb flush otherwise KVM >> >> hypercalls may be intercepted by Hyper-V. Add new parameter >> >> "hv-direct-tlbflush". Check expose_kvm and Hyper-V tlb flush >> >> capability status before enabling the feature. >> >> >> >> Signed-off-by: Tianyu Lan <Tianyu.Lan@xxxxxxxxxxxxx> >> >> --- >> >> Change since v3: >> >> - Fix logic of Hyper-V passthrough mode with direct >> >> tlb flush. >> >> >> >> Change sicne v2: >> >> - Update new feature description and name. >> >> - Change failure print log. >> >> >> >> Change since v1: >> >> - Add direct tlb flush's Hyper-V property and use >> >> hv_cpuid_check_and_set() to check the dependency of tlbflush >> >> feature. >> >> - Make new feature work with Hyper-V passthrough mode. >> >> --- >> >> docs/hyperv.txt | 10 ++++++++++ >> >> target/i386/cpu.c | 2 ++ >> >> target/i386/cpu.h | 1 + >> >> target/i386/kvm.c | 24 ++++++++++++++++++++++++ >> >> 4 files changed, 37 insertions(+) >> >> >> >> diff --git a/docs/hyperv.txt b/docs/hyperv.txt >> >> index 8fdf25c829..140a5c7e44 100644 >> >> --- a/docs/hyperv.txt >> >> +++ b/docs/hyperv.txt >> >> @@ -184,6 +184,16 @@ enabled. >> >> >> >> Requires: hv-vpindex, hv-synic, hv-time, hv-stimer >> >> >> >> +3.18. hv-direct-tlbflush >> >> +======================= >> >> +Enable direct TLB flush for KVM when it is running as a nested >> >> +hypervisor on top Hyper-V. When enabled, TLB flush hypercalls from L2 >> >> +guests are being passed through to L0 (Hyper-V) for handling. Due to ABI >> >> +differences between Hyper-V and KVM hypercalls, L2 guests will not be >> >> +able to issue KVM hypercalls (as those could be mishanled by L0 >> >> +Hyper-V), this requires KVM hypervisor signature to be hidden. >> > >> > On a second thought, I wonder if this is the only conflict we have. >> > >> > In KVM, kvm_emulate_hypercall, when sees Hyper-V hypercalls enabled, >> > just calls kvm_hv_hypercall and returns. I.e. once the userspace >> > enables Hyper-V hypercalls (which QEMU does when any of hv_* flags is >> > given), KVM treats *all* hypercalls as Hyper-V ones and handles *no* KVM >> > hypercalls. >> >> Yes, but only after guest enables Hyper-V hypercalls by writing to >> HV_X64_MSR_HYPERCALL. E.g. if you run a Linux guest and add a couple >> hv_* flags on the QEMU command line the guest will still be able to use >> KVM hypercalls normally becase Linux won't enable Hyper-V hypercall >> page. > > Ah, you're right. There's no conflict indeed, the guest makes > deliberate choice which hypercall ABI to use. > > Then QEMU (or KVM on its own?) should only activate this flag in evmcs > if it sees that the guest has enabled Hyper-V hypercalls. That was my suggestion as well when KVM patches were submitted, but if I remember correctly Tianyu said that if we don't enable 'direct tlb flush' flag in eVMCS on first VMLAUNCH, underlying Hyper-V won't give us a second chance so we can't enadle it after guest writes to HV_X64_MSR_HYPERCALL. This is a very unfortunate design/implementation. -- Vitaly
