On Wed, May 29, 2024 at 12:43:57PM GMT, Alexander Graf wrote:
On 29.05.24 10:04, Stefano Garzarella wrote:
On Tue, May 28, 2024 at 06:38:24PM GMT, Paolo Bonzini wrote:
On Tue, May 28, 2024 at 5:53 PM Stefano Garzarella
<sgarzare@xxxxxxxxxx> wrote:
On Tue, May 28, 2024 at 05:49:32PM GMT, Paolo Bonzini wrote:
On Tue, May 28, 2024 at 5:41 PM Stefano Garzarella
<sgarzare@xxxxxxxxxx> wrote:
>I think it's either that or implementing virtio-vsock in userspace
>(https://lore.kernel.org/qemu-devel/30baeb56-64d2-4ea3-8e53-6a5c50999979@xxxxxxxxxx/,
>search for "To connect host<->guest").
For in this case AF_VSOCK can't be used in the host, right?
So it's similar to vhost-user-vsock.
Not sure if I understand but in this case QEMU knows which CIDs are
forwarded to the host (either listen on vsock and connect to the host,
or vice versa), so there is no kernel and no VMADDR_FLAG_TO_HOST
involved.
I meant that the application in the host that wants to connect to the
guest cannot use AF_VSOCK in the host, but must use the one where QEMU
is listening (e.g. AF_INET, AF_UNIX), right?
I think one of Alex's requirements was that the application in the host
continue to use AF_VSOCK as in their environment.
Can the host use VMADDR_CID_LOCAL for host-to-host communication?
Yep!
If
so, the proposed "-object vsock-forward" syntax can connect to it and
it should work as long as the application on the host does not assume
that it is on CID 3.
Right, good point!
We can also support something similar in vhost-user-vsock, where instead
of using AF_UNIX and firecracker's hybrid vsock, we can redirect
everything to VMADDR_CID_LOCAL.
Alex what do you think? That would simplify things a lot to do.
The only difference is that the application in the host has to talk to
VMADDR_CID_LOCAL (1).
The application in the host would see an incoming connection from CID
1 (which is probably fine) and would still be able to establish
outgoing connections to the actual VM's CID as long as the Enclave
doesn't check for the peer CID (I haven't seen anyone check yet). So
yes, indeed, this should work.
The only case where I can see it breaking is when you run multiple
Enclave VMs in parallel. In that case, each would try to listen to CID
3 and the second that does would fail. But it's a well solvable
problem: We could (in addition to the simple in-QEMU case) build an
external daemon that does the proxying and hence owns CID3.
Well, we can modify vhost-user-vsock for that. It's already a daemon,
already supports different VMs per single daemon but as of now they have
to have different CIDs.
So the immediate plan would be to:
1) Build a new vhost-vsock-forward object model that connects to
vhost as CID 3 and then forwards every packet from CID 1 to the
Enclave-CID and every packet that arrives on to CID 3 to CID 2.
This though requires writing completely from scratch the virtio-vsock
emulation in QEMU. If you have time that would be great, otherwise if
you want to do a PoC, my advice is to start with vhost-user-vsock which
is already there.
Thanks,
Stefano
2) Create a machine option for -M nitro-enclave that automatically
spawns the vhost-vsock-forward object. (default: off)
The above may need some fiddling with object creation times to ensure
that the forward object gets CID 3, not the Enclave as auto-assigned
CID.
Thanks,
Alex
Amazon Web Services Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
