On 23/04/2020 16:42, Paolo Bonzini wrote:
On 23/04/20 15:19, Paraschiv, Andra-Irina wrote:
2. The enclave itself - a VM running on the same host as the primary VM
that spawned it.
The enclave VM has no persistent storage or network interface attached,
it uses its own memory and CPUs + its virtio-vsock emulated device for
communication with the primary VM.
The memory and CPUs are carved out of the primary VM, they are dedicated
for the enclave. The Nitro hypervisor running on the host ensures memory
and CPU isolation between the primary VM and the enclave VM.
These two components need to reflect the same state e.g. when the
enclave abstraction process (1) is terminated, the enclave VM (2) is
terminated as well.
With regard to the communication channel, the primary VM has its own
emulated virtio-vsock PCI device. The enclave VM has its own emulated
virtio-vsock device as well. This channel is used, for example, to fetch
data in the enclave and then process it. An application that sets up the
vsock socket and connects or listens, depending on the use case, is then
developed to use this channel; this happens on both ends - primary VM
and enclave VM.
Let me know if further clarifications are needed.
Thanks, this is all useful. However can you please clarify the
low-level details here?
- the initial CPU state: CPL0 vs. CPL3, initial program counter, etc.
The enclave VM has its own kernel and follows the well-known Linux boot
protocol, in the end getting to the user application after init finishes
its work, so that's CPL3.
- the communication channel; does the enclave see the usual local APIC
and IOAPIC interfaces in order to get interrupts from virtio-vsock, and
where is the virtio-vsock device (virtio-mmio I suppose) placed in
memory?
vsock is using eventfd for signalling; wrt enclave VM, it sees the usual
interfaces to get interrupts from virtio dev.
It's placed below the typical 4GB; in general, it may depend based on arch.
- what the enclave is allowed to do: can it change privilege levels,
what happens if the enclave performs an access to nonexistent memory,
etc.
If talking about the enclave abstraction process, it is running in the
primary VM as a user space process, so it will get into primary VM guest
kernel if privileged instructions need to be executed.
Same happens with the user space application running in the enclave VM.
And the VM itself will get to the hypervisor running on the host for
privileged instructions. The Nitro hypervisor is based on core KVM
technology.
Access to nonexistent memory gets faults.
- whether there are special hypercall interfaces for the enclave
The path towards creating / setting resources / terminating an enclave
(here referring to enclave VM) is towards the ioctl interface, with the
corresponding misc device, and the emulated PCI device. That's the
interface used to manage enclaves. Once booted, the enclave resources
setup is not modified anymore. And the way to communicate with the
enclave after booting, with the application running in the enclave, is
via the vsock comm channel.
Thanks,
Andra
Thanks,
Paolo
Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
