> On 1 Oct 2019, at 4:14, Nadav Amit <nadav.amit@xxxxxxxxx> wrote:
>
>> On Sep 30, 2019, at 5:48 PM, Liran Alon <liran.alon@xxxxxxxxxx> wrote:
>>
>>
>>
>>> On 1 Oct 2019, at 2:02, Nadav Amit <nadav.amit@xxxxxxxxx> wrote:
>>>
>>>> On Sep 19, 2019, at 5:52 AM, Liran Alon <liran.alon@xxxxxxxxxx> wrote:
>>>>
>>>> Hi,
>>>>
>>>> This patch series aims to add a vmx test to verify the functionality
>>>> introduced by KVM commit:
>>>> 4b9852f4f389 ("KVM: x86: Fix INIT signal handling in various CPU states")
>>>>
>>>> The test verifies the following functionality:
>>>> 1) An INIT signal received when CPU is in VMX operation
>>>> is latched until it exits VMX operation.
>>>> 2) If there is an INIT signal pending when CPU is in
>>>> VMX non-root mode, it result in VMExit with (reason == 3).
>>>> 3) Exit from VMX non-root mode on VMExit do not clear
>>>> pending INIT signal in LAPIC.
>>>> 4) When CPU exits VMX operation, pending INIT signal in
>>>> LAPIC is processed.
>>>>
>>>> In order to write such a complex test, the vmx tests framework was
>>>> enhanced to support using VMX in non BSP CPUs. This enhancement is
>>>> implemented in patches 1-7. The test itself is implemented at patch 8.
>>>> This enhancement to the vmx tests framework is a bit hackish, but
>>>> I believe it's OK because this functionality is rarely required by
>>>> other VMX tests.
>>>>
>>>> Regards,
>>>> -Liran
>>>
>>> Hi Liran,
>>>
>>> I ran this test on bare-metal and it fails:
>>>
>>> Test suite: vmx_init_signal_test
>>> PASS: INIT signal blocked when CPU in VMX operation
>>> PASS: INIT signal during VMX non-root mode result in exit-reason VMX_INIT (3)
>>> FAIL: INIT signal processed after exit VMX operation
>>> SUMMARY: 8 tests, 1 unexpected failures
>>>
>>> I don’t have time to debug this issue, but let me know if you want some
>>> print-outs.
>>>
>>> Nadav
>>
>> Thanks Nadav for running this on bare-metal. This is very useful!
>>
>> It seems that when CPU exited on exit-reason VMX_INIT (3), the LAPIC INIT pending event
>> was consumed instead of still being latched until CPU exits VMX operation.
>>
>> In my commit which this unit-test verifies 4b9852f4f389 ("KVM: x86: Fix INIT signal handling in various CPU states”),
>> I have assumed that such exit-reason don’t consume the LAPIC INIT pending event.
>> My assumption was based on the phrasing of Intel SDM section 25.2 OTHER CAUSES OF VM EXITS regarding INIT signals:
>> "Such exits do not modify register state or clear pending events as they would outside of VMX operation."
>> I thought Intel logic behind this is that if an INIT signal is sent to a CPU in VMX non-root mode, it would exit
>> on exit-reason 3 which would allow hypervisor to decide to exit VMX operation in order to consume INIT signal.
>
> I think this sentence can be read differently. It also reasonable not to
> bound the host to get an INIT signal the moment it disabled vmx.
If INIT signal won’t be kept pending until exiting VMX operation, target CPU which was sent with INIT signal
when it was running guest, basically lost INIT signal forever and just received an exit-reason it cannot do much with.
That’s why I thought Intel designed this mechanism like I specified above.
I also remembered to verify this behaviour against some discussions made online:
1) https://software.intel.com/en-us/forums/virtualization-software-development/topic/355484
* "When the 16-bit guest issues an INIT IPI to itself using the APIC, I run into an infinite VMExit situation that my hypervisor cannot seem to recover from.”
* "In response to the VMExit with a reason of 3 (which is expected), the hypervisor resets the 16-bit guest's registers, limits, access rights, etc. to simulate starting execution from a known initialization point. However, it seems that as soon as the hypervisor resumes guest execution, the VMExit occurs again, repeatedly.”
2) https://patchwork.kernel.org/patch/2244311/
"I actually find it very useful. On INIT vmexit hypervisor may call vmxoff and do proper reset."
Anyway, Sean, can you assist verifying inside Intel what should be the expected behaviour?
>
>> Nadav, can you attempt to just add a delay in init_signal_test_thread() between calling vmx_off() & setting init_signal_test_thread_continued to true?
>> It may be that real hardware delays a bit when the INIT signal is released from LAPIC after exit VMX operation.
>
> I added “delay(100000)” between them, but got the same result.
>
Weird. Then I have no idea why this happening…
Thanks,
-Liran
