Re: [Bug] AMD nested: commit broke VMware

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



У вт, 2023-05-30 у 13:34 -0700, Jim Mattson пише:
> On Tue, May 30, 2023 at 1:10 PM Jim Mattson <jmattson@xxxxxxxxxx> wrote:
> > On Mon, May 29, 2023 at 6:44 AM Maxim Levitsky <mlevitsk@xxxxxxxxxx> wrote:
> > > У пн, 2023-05-29 у 14:58 +0200, jwarren@xxxxxxxxxxxx пише:
> > > > Hello,
> > > > Since kernel 5.16 users can't start VMware VMs when it is nested under KVM on AMD CPUs.
> > > > 
> > > > User reports are here:
> > > > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2008583
> > > > https://forums.unraid.net/topic/128868-vmware-7x-will-not-start-any-vms-under-unraid-6110/
> > > > 
> > > > I've pinpointed it to commit 174a921b6975ef959dd82ee9e8844067a62e3ec1 (appeared in 5.16rc1)
> > > > "nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB"
> > > > 
> > > > I've confirmed that VMware errors out when it checks for TLB_CONTROL_FLUSH_ASID support and gets a 'false' answer.
> > > > 
> > > > First revisions of the patch in question had some support for TLB_CONTROL_FLUSH_ASID, but it was removed:
> > > > https://lore.kernel.org/kvm/f7c2d5f5-3560-8666-90be-3605220cb93c@xxxxxxxxxx/
> > > > 
> > > > I don't know what would be the best case here, maybe put a quirk there, so it doesn't break "userspace".
> > > > Committer's email is dead, so I'm writing here.
> > > > 
> > > 
> > > I have to say that I know about this for long time, because some time ago I used  to play with VMware player in a
> > > VM on AMD on my spare time, on weekends
> > > (just doing various crazy things with double nesting, running win98 nested, vfio stuff, etc, etc).
> > > 
> > > I didn't report it because its a bug in VMWARE - they set a bit in the tlb_control without checking CPUID's FLUSHBYASID
> > > which states that KVM doesn't support setting this bit.
> > 
> > I am pretty sure that bit 1 is supposed to be ignored on hardware
> > without FlushByASID, but I'll have to see if I can dig up an old APM
> > to verify that.
> 
> I couldn't find an APM that old, but even today's APM does not specify
> that any checks are performed on the TLB_CONTROL field by VMRUN.
> 
> While Intel likes to fail VM-entry for illegal VMCS state, AMD prefers
> to massage the VMCB to render any illegal VMCB state legal. For
> example, rather than fail VM-entry for a non-canonical address, AMD is
> inclined to drop the high bits and sign-extend the low bits, so that
> the address is canonical.
> 
> I'm willing to bet that modern CPUs continue to ignore the TLB_CONTROL
> bits that were noted "reserved" in version 3.22 of the manual, and
> that Krish simply manufactured the checks in commit 174a921b6975
> ("nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB"),
> without cause.
> 
> > > Supporting FLUSHBYASID would fix this, and make nesting faster too,
> > > but it is far from a trivial job.
> > > 
> > > I hope that I will find time to do this soon.
> > > 
> > > Best regards,
> > >         Maxim Levitsky
> > > 
> > > 

Yup...

After applying this horrible hack to KVM, the VM still boots just fine
on bare metal.

diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index e7c7379d6ac7b0..2e45c1b747104a 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -170,10 +170,10 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 };
 
 
-#define TLB_CONTROL_DO_NOTHING 0
-#define TLB_CONTROL_FLUSH_ALL_ASID 1
-#define TLB_CONTROL_FLUSH_ASID 3
-#define TLB_CONTROL_FLUSH_ASID_LOCAL 7
+#define TLB_CONTROL_DO_NOTHING 0xF0
+#define TLB_CONTROL_FLUSH_ALL_ASID 0xF1
+#define TLB_CONTROL_FLUSH_ASID 0xF3
+#define TLB_CONTROL_FLUSH_ASID_LOCAL 0xF7
 
 #define V_TPR_MASK 0x0f
 

Shall we revert the offending patch then?

Best regards,
	Maxim Levitsky




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux