On 5/21/2017 9:55 AM, Andy Lutomirski wrote:
On Thu, May 18, 2017 at 1:14 AM, Huang, Kai <kai.huang@xxxxxxxxxxxxxxx> wrote:
You are making assumption that KVM will run ENCLS on behalf of guest. :)
If we don't need to look into guest's SIGSTRUCT, EINITTOKEN, etc, then I
actually prefer to using MTF, as with MTF we don't have to do all the
remapping guest's virtual address to KVM's virtual address thing, if we
don't need to look into guest's ENCLS parameter. But if we need to look into
guest's ENCLS parameters, for example, to locate physical SECS page, or to
update physical EPC page's info (that KVM needs to maintain), maybe we can
choose running ENCLS on behalf of guest.
After thinking about this a bit, I don't see how MTF helps.
Currently, KVM works kind of like this:
local_irq_disable();
set up stuff;
VMRESUME;
restore some host state;
local_irq_enable();
If the guest is going to run with the EINIT-exiting bit clear, the
only way I see this working is to modify KVM along the lines of:
local_irq_disable();
set up stuff;
if (condition here) {
WRMSR to SGXLEPUBKEYHASH;
update percpu shadow copyl
clear EINIT-exiting bit;
} else {
set EINIT-exiting bit;
}
VMRESUME;
restore some host state;
local_irq_enable();
where "condition here" might be something like "the last VMRESUME
exited due to EINIT".
I don't see how MTF helps much. And if I were the KVM maintainer, I
would probably prefer to trap EINIT instead of adding a special case
to the main vm entry code.
Hi Andy,
Thanks for your comments. However I didn't intend to use MTF in your
way. The idea of using MTF (along with ENCLS VMEXIT) is, by turning on
MTF VMEXIT upon ENCLS VMEXIT, we are able to mark a single step VMEXIT
after ENCLS so that ENCLS can run in guest as single step.
Let me explain how the two approaches work below in general, so that we
can decide which is better. Only trapping EINIT in order to update
IA32_SGXLEPUBKEYHASHn is relatively simpler but I'd compare the two in
more general way, assuming we may want to trap more ENCLS in order to,
ex, track EPC/Enclave status/info, in the future to support, ex, EPC
oversubscription between KVM guests.
Below diagram shows the basic idea of the two approaches.
--------------------------------------------------------------
| ENCLS |
--------------------------------------------------------------
| /|\
ENCLS VMEXIT | | VMENTRY
| |
\|/ |
1) identify which ENCLS leaf (RAX)
2) reconstruct/remap guest's ENCLS parameters, ex:
- remap any guest VA (virtual address) to KVM VA
- reconstruct PAGEINFO
3) do whatever needed before ENCLS, ex:
- updating MSRs before EINIT
4) run ENCLS on behalf of guest, and skip ENCLS
5) emulate ENCLS result (succeeded or not)
- update guest's RAX-RDX.
- and/or inject #GP (or #UD).
6) do whatever needed after ENCLS, ex:
- updating EPC/Enclave status/info
1) Run ENCLS on behalf of guest
--------------------------------------------------------------
| ENCLS |
--------------------------------------------------------------
|/|\ |/|\
ENCLS VMEXIT | | VMENTRY MTF VMEXIT | | VMENTRY
| | | |
\|/| \|/|
1) Turn off EMCLS VMEXIT 1) Turn off MTF VMEXIT
2) turn on MTF VMEXIT 2) Turn on ENCLS VMEXIT
3) cache ENCLS parameters 3) check whether ENCLS has run
(ENCLS changes RAX) 4) check whether ENCLS succeeded
4) do whatever needed before or not.
ENCLS 5) do whatever needed after ENCLS
2) Using MTF
The concern of running ENCLS on behalf of guest is emulating ENCLS
error. KVM needs to *correctly* emulate ENCLS error to guest so that the
error we inject to guest can reflect the right behavior as if ENCLS run
in guest. Running ENCLS in root-mode may be potentially different
running ENCLS in non-root mode, therefore we have to go through all
possible error codes to make sure we can emulate. And for some error
code, ex, SGX_LOCKFAIL, we can handle it in KVM and don't have to inject
error to guest. So the point is we have to go through all error code to
make sure KVM can emulate ENCLS error code correctly for guest.
Another argument is Intel may add new error codes in the future when
more SGX functionalities are introduced, so emulating error code may be
a burden.
Using MTF is also a little bit tricky, as when we turn on MTF VMEXIT
upon ENCLS VMEXIT, the MTF won't be absolutely pending at end of that
ENCLS. For example, MTF may be pending at end of interrupt (cannot
recall exactly) if event is pending during VMENTRY from ENCLS VMEXIT.
Therefore we have to do additional thing to check whether this MTF
VMEXIT really happens after ENCLS run (step 3 above). And depending on
what we need to do, we may need to check whether ENCLS succeeded or not
in guest, which is also tricky, as ENCLS can fail in either setting
error code in RAX, or generating #GP or #UD (step 4 above). We may still
need to do gva->gpa->hpa, ex, in order to locate EPC/SECS page and
update status, depending on the purpose of trapping ENCLS.
But by using MTF, we don't have to worry about ENCLS error emulation, as
ENCLS runs in guest, thus we don't need to worry about this root-mode
and non-root mode difference. I think this is the major reason that we
want to use MTF.
