On Mon, Sep 30, 2019 at 06:29:52PM -0700, Nadav Amit wrote: > > On Sep 30, 2019, at 6:23 PM, Liran Alon <liran.alon@xxxxxxxxxx> wrote: > > > > If INIT signal won’t be kept pending until exiting VMX operation, target > > CPU which was sent with INIT signal when it was running guest, basically > > lost INIT signal forever and just received an exit-reason it cannot do much > > with. That’s why I thought Intel designed this mechanism like I specified > > above. > > Well, the host can always issue an INIT using an IPI. And conversely, if the INIT persisted then the host would be forced to do VMXOFF, otherwise it effectively couldn't do VM-Enter on that logical CPU. > > I also remembered to verify this behaviour against some discussions made online: > > 1) https://software.intel.com/en-us/forums/virtualization-software-development/topic/355484 > > * "When the 16-bit guest issues an INIT IPI to itself using the APIC, I run into an infinite VMExit situation that my hypervisor cannot seem to recover from.” > > * "In response to the VMExit with a reason of 3 (which is expected), the hypervisor resets the 16-bit guest's registers, limits, access rights, etc. to simulate starting execution from a known initialization point. However, it seems that as soon as the hypervisor resumes guest execution, the VMExit occurs again, repeatedly.” > > 2) https://patchwork.kernel.org/patch/2244311/ > > "I actually find it very useful. On INIT vmexit hypervisor may call vmxoff and do proper reset." > > > > Anyway, Sean, can you assist verifying inside Intel what should be the expected behaviour? > > It might always be (yet) another kvm-unit-tests bug that is only apparent on > bare-metal. But if Sean can confirm what the expected behavior is, it would > save time. > > I do not have an ITP, so debugging on bare-metal is not fun at all... My understanding of the architecture is that the INIT should be consumed on VM-Exit. The only scenario where an event is not consumed/acknowledge is when a vanilla interrupt occurs without VM_EXIT_ACK_INTR_ON_EXIT set, in which case the VM-Exit is technically considered a "pending" interrupt. For all other cases (NMI, SMI, INIT, and INTR w/ ACK-ON-EXIT), the VM-Exit is the end result of delivering the event. INITs are indeed blocked and not dropped in VMX root mode. But entering non-root (guest) mode should unblock INITs and cause a VM-Exit, and thus clear the INIT that was pended while in VMX root mode. This behavior does not conflict with the whitepaper[*] referenced by link (2) above, and in fact the whitepaper explicitly covers guest mode behavior in a footnote: When the processor is in VMX guest mode, delivery of INIT causes a normal VMEXIT, of course. The INIT attack described uses "VMX mode" to refer to VMX root mode, and other than the footnote, doesn't mention VMX guest mode. My reading of it is that they're showing a proof of concept of based on getting the OS into VMX root mode but not actually running a guest, e.g. this can be done in KVM by creating a VM (KVM_CREATE_VM) but not running it (KVM_RUN). Anyways, I'll double check that the INIT should indeed be consumed as part of the VM-Exit. [*] https://invisiblethingslab.com/resources/2011/Software%20Attacks%20on%20Intel%20VT-d.pdf
