On Mon, May 22, 2017 at 10:43 PM, Huang, Kai <kai.huang@xxxxxxxxxxxxxxx> wrote:
>
>
> On 5/21/2017 9:55 AM, Andy Lutomirski wrote:
>>
>> On Thu, May 18, 2017 at 1:14 AM, Huang, Kai <kai.huang@xxxxxxxxxxxxxxx>
>> wrote:
>>>
>>> You are making assumption that KVM will run ENCLS on behalf of guest. :)
>>>
>>> If we don't need to look into guest's SIGSTRUCT, EINITTOKEN, etc, then I
>>> actually prefer to using MTF, as with MTF we don't have to do all the
>>> remapping guest's virtual address to KVM's virtual address thing, if we
>>> don't need to look into guest's ENCLS parameter. But if we need to look
>>> into
>>> guest's ENCLS parameters, for example, to locate physical SECS page, or
>>> to
>>> update physical EPC page's info (that KVM needs to maintain), maybe we
>>> can
>>> choose running ENCLS on behalf of guest.
>>
>>
>> After thinking about this a bit, I don't see how MTF helps.
>> Currently, KVM works kind of like this:
>>
>> local_irq_disable();
>> set up stuff;
>> VMRESUME;
>> restore some host state;
>> local_irq_enable();
>>
>> If the guest is going to run with the EINIT-exiting bit clear, the
>> only way I see this working is to modify KVM along the lines of:
>>
>> local_irq_disable();
>> set up stuff;
>> if (condition here) {
>> WRMSR to SGXLEPUBKEYHASH;
>> update percpu shadow copyl
>> clear EINIT-exiting bit;
>> } else {
>> set EINIT-exiting bit;
>> }
>> VMRESUME;
>> restore some host state;
>> local_irq_enable();
>>
>> where "condition here" might be something like "the last VMRESUME
>> exited due to EINIT".
>>
>> I don't see how MTF helps much. And if I were the KVM maintainer, I
>> would probably prefer to trap EINIT instead of adding a special case
>> to the main vm entry code.
>>
>
> Hi Andy,
>
> Thanks for your comments. However I didn't intend to use MTF in your way.
> The idea of using MTF (along with ENCLS VMEXIT) is, by turning on MTF VMEXIT
> upon ENCLS VMEXIT, we are able to mark a single step VMEXIT after ENCLS so
> that ENCLS can run in guest as single step.
>
> Let me explain how the two approaches work below in general, so that we can
> decide which is better. Only trapping EINIT in order to update
> IA32_SGXLEPUBKEYHASHn is relatively simpler but I'd compare the two in more
> general way, assuming we may want to trap more ENCLS in order to, ex, track
> EPC/Enclave status/info, in the future to support, ex, EPC oversubscription
> between KVM guests.
>
> Below diagram shows the basic idea of the two approaches.
>
...
>
> --------------------------------------------------------------
> | ENCLS |
> --------------------------------------------------------------
> |/|\ |/|\
> ENCLS VMEXIT | | VMENTRY MTF VMEXIT | | VMENTRY
> | | | |
> \|/| \|/|
> 1) Turn off EMCLS VMEXIT 1) Turn off MTF VMEXIT
> 2) turn on MTF VMEXIT 2) Turn on ENCLS VMEXIT
> 3) cache ENCLS parameters 3) check whether ENCLS has run
> (ENCLS changes RAX) 4) check whether ENCLS succeeded
> 4) do whatever needed before or not.
> ENCLS 5) do whatever needed after ENCLS
>
> 2) Using MTF
>
...
> Using MTF is also a little bit tricky, as when we turn on MTF VMEXIT upon
> ENCLS VMEXIT, the MTF won't be absolutely pending at end of that ENCLS. For
> example, MTF may be pending at end of interrupt (cannot recall exactly) if
> event is pending during VMENTRY from ENCLS VMEXIT. Therefore we have to do
> additional thing to check whether this MTF VMEXIT really happens after ENCLS
> run (step 3 above). And depending on what we need to do, we may need to
> check whether ENCLS succeeded or not in guest, which is also tricky, as
> ENCLS can fail in either setting error code in RAX, or generating #GP or #UD
> (step 4 above). We may still need to do gva->gpa->hpa, ex, in order to
> locate EPC/SECS page and update status, depending on the purpose of trapping
> ENCLS.
I think there are some issues here.
First, you're making a big assumption that, when you resume the guest
with MTF set, the instruction that gets executed is still
ENCLS[EINIT]. That's not guaranteed as is -- you could race against
another vCPU that changes the instruction, the instruction could be in
IO space, host userspace could be messing with you, etc. Second, I
don't think there's any precedent at all in KVM for doing this.
Third, you still need to make sure that the MSRs retain the value you
want them to have by the time ENCLS happens. I think that, by the
time you resolve all of these issues, it'll look a lot like the
pseudocode I emailed out, and MTF won't be necessary any more.
>
> But by using MTF, we don't have to worry about ENCLS error emulation, as
> ENCLS runs in guest, thus we don't need to worry about this root-mode and
> non-root mode difference. I think this is the major reason that we want to
> use MTF.
I don't see why error emulation is hard. If the host does ENCLS on
behalf of the guest and it returns an error, can't you return exactly
the same error to the guest with no further processing? The only
tricky case is where the host rejects due to its own policy and you
have to choose an error code.
--Andy
