On Tue, Oct 01, 2019 at 11:40:34AM -0700, Sean Christopherson wrote: > On Mon, Sep 30, 2019 at 06:29:52PM -0700, Nadav Amit wrote: > > > On Sep 30, 2019, at 6:23 PM, Liran Alon <liran.alon@xxxxxxxxxx> wrote: ... > > > I also remembered to verify this behaviour against some discussions made online: > > > 1) https://software.intel.com/en-us/forums/virtualization-software-development/topic/355484 > > > * "When the 16-bit guest issues an INIT IPI to itself using the APIC, I run into an infinite VMExit situation that my hypervisor cannot seem to recover from.” > > > * "In response to the VMExit with a reason of 3 (which is expected), the hypervisor resets the 16-bit guest's registers, limits, access rights, etc. to simulate starting execution from a known initialization point. However, it seems that as soon as the hypervisor resumes guest execution, the VMExit occurs again, repeatedly.” > > > 2) https://patchwork.kernel.org/patch/2244311/ > > > "I actually find it very useful. On INIT vmexit hypervisor may call vmxoff and do proper reset." > > > > > > Anyway, Sean, can you assist verifying inside Intel what should be the expected behaviour? > > > > It might always be (yet) another kvm-unit-tests bug that is only apparent on > > bare-metal. But if Sean can confirm what the expected behavior is, it would > > save time. > > > > I do not have an ITP, so debugging on bare-metal is not fun at all... > > My understanding of the architecture is that the INIT should be consumed > on VM-Exit. The only scenario where an event is not consumed/acknowledge > is when a vanilla interrupt occurs without VM_EXIT_ACK_INTR_ON_EXIT set, > in which case the VM-Exit is technically considered a "pending" interrupt. > For all other cases (NMI, SMI, INIT, and INTR w/ ACK-ON-EXIT), the VM-Exit > is the end result of delivering the event. > > INITs are indeed blocked and not dropped in VMX root mode. But entering > non-root (guest) mode should unblock INITs and cause a VM-Exit, and thus > clear the INIT that was pended while in VMX root mode. This behavior does > not conflict with the whitepaper[*] referenced by link (2) above, and in > fact the whitepaper explicitly covers guest mode behavior in a footnote: > > When the processor is in VMX guest mode, delivery of INIT causes a > normal VMEXIT, of course. > > The INIT attack described uses "VMX mode" to refer to VMX root mode, and > other than the footnote, doesn't mention VMX guest mode. My reading of it > is that they're showing a proof of concept of based on getting the OS into > VMX root mode but not actually running a guest, e.g. this can be done > in KVM by creating a VM (KVM_CREATE_VM) but not running it (KVM_RUN). > > Anyways, I'll double check that the INIT should indeed be consumed as part > of the VM-Exit. I couldn't help but run a few tests before reaching out to the architecture folks... I modified KVM to have the CPU send an INIT IPI to itself in vmx_vcpu_run(), with a bit of delay to ensure the INIT is pending prior to VM-Enter. On an INIT VM-Exit, KVM immediately resumes the guest. On Haswell client system, the INIT does indeed appear to be consumed when it's handled by VM-Exit, i.e. KVM doesn't get stuck in an infinite INIT VM-Exits loop. One possible explanation for the infinite loop observed in (1) above, is that the developer didn't properly reconfigure guest state when emulating INIT and hit a VM-Fail. Because vmcs.EXIT_REASON isn't written on VM-Fail, if the VMM isn't checking for VM-Fail it will think it's getting endless INIT VM-Exits. I did exactly this when tweaking KVM to handle INIT (forgot to mark the VMCS as launched redoing VM-Enter), so I even inadvertantly confirmed that it's plausible :-)
