On 2/24/22 7:26 AM, Daniel P. Berrangé wrote: > On Wed, Feb 23, 2022 at 03:33:22PM -0500, Tobin Feldman-Fitzthum wrote: >> >> >> On 2/23/22 1:38 PM, Dov Murik wrote: >>> +cc Tobin, James >>> >>> On 23/02/2022 19:28, Daniel P. Berrangé wrote: >>>> Extending management apps using libvirt to support measured launch of >>>> QEMU guests with SEV/SEV-ES is unreasonably complicated today, both for >>>> the guest owner and for the cloud management apps. We have APIs for >>>> exposing info about the SEV host, the SEV guest, guest measurements >>>> and secret injections. This is a "bags of bits" solution. We expect >>>> apps to them turn this into a user facting solution. It is possible >>>> but we're heading to a place where every cloud mgmt app essentially >>>> needs to reinvent the same wheel and the guest owner will need to >>>> learn custom APIs for dealing with SEV/SEV-ES for each cloud mgmt >>>> app. This is pretty awful. We need to do a better job at providing >>>> a solution that is more general purpose IMHO. >>>> > >> Note in particular that we provide a client script called LaunchVM.py >> that uses libvirt to start an SEV VM in conjunction with the attestation >> server. This is basically a stand in for a management app or cloud >> control plane. The modifications needed to launch an SEV VM are not >> particularly extensive. I agree with some of your comments though. In >> some ways it might be nice to have libvirt take care of things and hide >> the complexity from the management app. > > LaunchVM.py nicely illustrates my concerns. Every application that > uses libvirt that knows how to start VMs, now needs to be changed > to support the series of operations shown in LaunchVM.py. THe guest > owner probably can't use LaunchVM.py except for demoware, as they'll > need a equivalent that talks to the API of their cloud mgmt app, > of which there are many. > >> When we started working on our attestation server, our initial plan was >> to make PRs to libvirt that would add one end of the attestation API to >> libvirt, which would directly query the KBS. This is basically what you >> are proposing. We decided against this for a couple of reasons. >> >> First, we were concerned that libvirt might not have network >> connectivity to an arbitrary attestation server in a cloud environment. >> We had envisioned that the guest owner would provide a URI for their >> attestation server as part of the XML. This assumes that the node where >> the VM is going to run can connect to an attestation server living >> somewhere on the internet. I think that this might be challenging in >> some cloud environments. By having the management app connect to libvirt >> and the attestation server, we add some flexibility. > > Agreed, we can't assume that libvirt will always have ability to > connect to an arbitrary service on the internet. > > That said, it does not neccessarily need this ability. If the user > gives a URL of 'https://myhost.com/attest', the cloud doesn't have > to give that straight to libvirt. The cloud software could have a > attestation proxy server. So they could tell libvirt to use the > URI https://10.0.0.5/attest, and then libvirt connects to that, > it will proxy the calls through the guest owner's real attestation > server. This might slightly contradict the idea of the management app being out of the loop, but I guess setting up a proxy isn't very difficult. I think CSPs already do this kind of thing to enable other features. > > If even that isn't possible though, there's still the fallback > option of ignoring libvirt's native support for talking to an > attestation server, and doing it manually as per LaunchVM.py > illustration. > >> Second, we were worried that it would be difficult to settle on and >> maintain a standard. Fortunately this discussion is only relevant for >> SEV(-ES), given that SNP measurements are reported from inside the >> guest, but nonetheless there are already a number of approaches for >> handling things. By using a management app, each CSP can easily adapt >> the standard libvirt api into whatever attestation API they want. This >> does put a burden on the management apps, but it might sidestep a tricky >> problem for libvirt and like I said, we found it pretty easy to write >> our LaunchVM script (except for the CEK issue mentioned elsewhere). > > The attestation server is ultimately something that the guest owner > needs to control / use. Whether the cloud mgmt app conects to it, or > if libvirt connects to it, it feels like we would benefit from having > a standard that can be used from either approach. > > I don't think we want to end up with IBM's cloud requiring one attestation > server design and OpenStack requiring another and KubeVirt requiring yet > another, etc. Guest owners souldn't be given the burden of using different > services depending on which cloud they deploy on each time, as that would > effectively become a form of vendor lockin. Yes. We're probably going to end up with a bunch of attestation servers no matter, but it would be great if they were interoperable. > > The needs of all the apps look similar enough, because they are all > ultimately constrained by the functionality made available by SEV(ES). > Thus at least wrt apps doing traditional VM management using QEMU, > it feels like it should be practical to come up with a common solution. > I am all for common ground here. I had basically given up on it, but maybe libvirt has enough influence to set a standard. In your first email you said that there is a relatively short window to come up with something and I think that is probably correct. > I can understand if it is harder to achieve commonality with tech > like libkrun though, since that's consuming virt in quite a different > way at the userspace level. Yeah, extending the focus beyond SEV(-ES) with QEMU might make things more difficult. There is some discussion right now about trying to find common ground between SEV-SNP and TDX attestation, but I assume that is all out of scope since libvirt isn't really involved. -Tobin > > >> Finally, we didn't think that there would be any interest in the libvirt >> community. It seems like we might have been wrong about this. Like I >> said, our first instinct was to extend libvirt, and if there is interest >> in doing this, we could dust off those ideas. I certainly have a lot of >> ideas about how to design an API for attestation. Of course we now have >> an API for attestation that we think is pretty good. It is gRPC, but we >> are thinking about also supporting a REST interface. If an attestation >> api is added to libvirt, I will definitely try to be involved, although >> honestly I think it's fine, and in some ways maybe better, to have the >> management app take care of things. > > Thanks for the feedback so far. > > Regards, > Daniel
