Some comments on the example protocol stuff
On 2/23/22 1:38 PM, Dov Murik wrote:
> +cc Tobin, James
>
> On 23/02/2022 19:28, Daniel P. Berrangé wrote:
>>
>>
>> What could this look like from POV of an attestation server API, if
>> we assume HTTPS REST service with a simple JSON payload .>>
>>
>> * Guest Owner: Register a new VM to be booted:
We're trying to set the API between libvirt and the AS. I would assume
that the API between the AS and the guest owner is out of scope,
although maybe this is just an example.
>>
>> POST /vm/<UUID>
Note that this is a privileged endpoint (unlike the ones below).
>>
>> Request body:
>>
>> {
>> "scheme": "amd-sev",
>> "cloud-cert": "certificate of the cloud owner that signs the PEK",
>> "policy": 0x3,
>> "cpu-count": 3,
>> "firmware-hashes": [
>> "xxxx",
>> "yyyy",
>> ],
I think we'd need to provide the full firmware binary rather than just
the hash if we plan to calculate the launch digest in the AS.
Alternatively the guest owner can calculate the launch digest themself
and pass it to the AS. This is what kbs-rs does. There are pros and cons
to both and we should probably support both (which should be easy).
>> "kernel-hash": "aaaa",
>> "initrd-hash": "bbbb",
>> "cmdline-hash": "cccc",
>> "secrets": [
>> {
>> "type": "luks-passphrase",
>> "passphrase": "<blah>"
>> }
>> ]
>> }
>>
Registering an individual VM is kind of an interesting perspective. With
kbs-rs, rather than registering an individual VM, the guest owner
registers secrets and can set a policy (which specifies launch
parameters like the SEV policy) for each secret. Then secrets are
released to VMs that meet the policy requirements. There isn't really
any tracking of an individual VM (besides the secure channel briefly
used for secret injection). In SEV(-ES) individual VMs don't really have
an identity separate from their launch parameters and launch
measurement. I guess we're not trying to design an AS here, so we can
leave for another time.
>>
>>
>> * Libvirt: Request permission to launch a VM on a host
>>
>> POST /vm/<UUID>/launch
Since I've been thinking about VM identity a little differently, our
setup for the UUID is a bit different as well. We use a UUID to track a
connection (as in TIK, TEK), but this is not known at the time of the
launch request (aka GetBundle request). Instead, the UUID is returned in
the launch response so that it can be used for the secret request.
If we have a UUID in the launch request, it creates an interesting
coordination requirement between the CSP and the AS. Imagine a situation
where we spin up a bunch of identical VMs dynamically. Here the guest
owner would have to register a new VM with a UUID for each instance and
then get all of that information over to libvirt. This seems difficult.
Shifting focus from VMs to secrets and policies and automatically
provisioning the UUID sidesteps this issue. This is especially important
for something like Confidential Containers (of course CC doesn't use
libvirt but we'd like to share the AS API).
>>
>> Request body:
>>
>> {
>> "pdh": "<blah>",
>> "cert-chain": "<blah>",
>> "cpu-id": "<CPU ID>",
There's an interesting question as to whether the CEK should be signed
by libvirt or by the AS.
>> ...other relevant bits...
>> }
>>
>> Service decides if the proposed host is acceptable
>>
>> Response body (on success)
>>
>> {
>> "session": "<blah>",
>> "owner-cert": "<blah>",
>> "policy": 3,
I've assumed that the policy would be part of the request, having been
set in the libvirt XML.
>> }
>>
>>
>>
>> * Libvirt: Request secrets to inject to launched VM
>>
>> POST /vm/<UUID>/validate
>>
>> Request body:
>>
>> {
>> "api-minor": 1,
>> "api-major": 2,
>> "build-id": 241,
>> "policy": 3,
>> "measurement": "<blah>",
>> "firmware-hash": "xxxx",
>> "cpu-count": 3,
>> ....other relevant stuff....
>> }
>>
>> Service validates the measurement...
>>
>> Response body (on success):
>>
>> {
>> "secret-header": "<blah>",
>> "secret-table": "<blah>",
Referring to secret payload format as OVMF secret table?
Looks pretty good overall. I am a bit worried about the UUID stuff.
-Tobin
>> }
>>
>>
>>
>> So we can see there are only a couple of REST API calls we need to be
>> able to define. If we could do that then creating a SEV/SEV-ES enabled
>> guest with libvirt would not involve anything more complicated for the
>> mgmt app that providing the URI of the guest owner's attestation service
>> and an identifier for the VM. ie. the XML config could be merely:
>>
>> <launchSecurity type="sev">
>> <attestation vmid="57f669c2-c427-4132-bc7a-26f56b6a718c"
>> service="http://somehost/some/url"/>
>> </launchSecurity>
>>
>> And then involve virDomainCreate as normal with any other libvirt / QEMU
>> guest. No special workflow is required by the mgmt app. There is a small
>> extra task for the guest owner to register existance of their VM with the
>> attestation service. Aside from that the only change to the way they
>> interact with the cloud mgmt app is to provide the VM ID and URI for the
>> attestation service. No need to learn custom APIs for each different
>> cloud vendor, for dealing with fetching launch measurements or injecting
>> secrets.
>>
>>
>> Finally this attestation service REST protocol doesn't have to be something
>> controlled or defined by libvirt. I feel like it could be a protocol that
>> is defined anywhere and libvirt merely be one consumer of it. Other apps
>> that directly use QEMU may also wish to avail themselves of it.
>>
>>
>> All that really matters from libvirt POV is:
>>
>> - The protocol definition exist to enable the above workflow,
>> with a long term API stability guarantee that it isn't going to
>> changed in incompatible ways
>>
>> - There exists a fully open source reference implementation of sufficient
>> quality to deploy in the real world
>>
>> I know https://github.com/slp/sev-attestation-server exists, but its current
>> design has assumptions about it being used with libkrun AFAICT. I have heard
>> of others interested in writing similar servers, but I've not seen code.
>>
>
>
> Tobin has just released kbs-rs which has similar properties to what
> you're proposing above, aiming to solve similar issues. Better talk
> with him before running into building yet another attestation server.
>
> -Dov
>
>
>> We are at a crucial stage where mgmt apps are looking to support measured
>> boot with SEV/SEV-ES and if we delay they'll all go off and do their own
>> thing, and it'll be too late, leading to https://xkcd.com/927/.
>>
>> Especially for apps using libvirt to manage QEMU, I feel we have got a
>> few months window of opportunity to get such a service available, before
>> they all end up building out APIs for the tedious manual workflow,
>> reinventing the wheel.
>>
>> Regards,
>> Daniel
