On Fri, Jul 24, 2009 at 07:25:54AM -0400, Miloslav Trmac wrote:
>
> ----- "Daniel P. Berrange" <berrange@xxxxxxxxxx> wrote:
> > > Not quite: the main case of a "dumb" client would be a large-scale
> > > virtualization management software that contains a primary store
> > > of encryption information, and gives each node access only to those
> > > keys that are currently necessary by the node to run its domains;
> > > the fact that each node has access to only a limited set of keys
> > > prevents an attacker that compromises a single node from reading
> > > disk images of all domains managed in the entire site, even if
> > > the disk image storage (e.g. unencrypted NFS) does not allow
> > > managing access by each node separately.
> > >
> > > Such a client must be able to transfer the actual secrets, not only
> > > identifiers, to libvirt. (The idea of a "dumb" client, that does
> > > not know the specifics of the format, is an additional feature on
> > > top, but one that implies that the client does send the actual
> > secrets.)
> >
> > This implies a flow of secrets
> >
> > Key server --\
> > +-> libvirt client -> libvirt daemon -> qemu
> > MGMT server --/
> >
> > This does not in fact guarentee that secrets for a particular
> > node are only used on the node for which they are intended,
> > because the key server cannot be sure of what libvirt daemons
> > the libvirt client is connected to.
>
> A client in this case is the central, fully trusted, management
> system (e.g. oVirt), there is no need to protect against it.
> A more likely flow is
>
> MGMT client (no knowledge of secrets)
> |
> v
> MGMT server + key server (integrated or separate but cooperating)
> |
> v
> libvirt daemon
> |
> v
> qemu
>
> > What I am suggesting is that libvirt daemon should communicate
> > with the key server directly in all cases, and take the client
> > out of the loop. The client should merely indicate whether it
> > wants encryption or not, and never be able to directly access
> > any key material itself. With a direct trust relationship
> > between the key server and each libvirtd daemon, you do now
> > have a guarentee that keys are only ever used on the node for
> > which they are intended. You also have the additional guarentee
> > that no libvirt client can ever see any key secrets or passphrases
> > since it has been taken completely out of the loop.
>
> As far as I understand it, the whole point of virtual machine
> encryption is that the nodes are _not_ trusted, and different
> encryption keys protect data on different nodes.
I did not mean to imply that libvirtd on a node should have
access to *all* secrets. Each libvirtd daemon has its own
identity, and when talking to a keys server it would authenticate
and the key server would only allow it access to some sub-set
of keys. ie, only the keys neccessary for VMs it needs to run.
If you include the secrets in the XML for a guest, that does
not ensure those secrets are only accessible to that host.
For example, when performing migration, the XML doc for a guest
is passed directly from the source libvirtd to the destination
libvirtd.
So an admin could ssh into a node, run 'virsh migrate' and the
guest & secrets would be transferred to another host.
If you have each libvirtd requesting secrets directly from the
keystore, at the time it starts a guest, then should an admin
issue a migrate command manually, the destination libvirtd
would still be unable to access the secrets of the incoming
VM, since the keystore will not have been configured to allow
it access.
We also have to bear in mind how the MGMT server communicates
with the libvirt daemon. One likely option is using a messaging
service, which offers authenticity but not secrecy. ie, libvirt
receiving a request off the message bus can be sure the request
came from the mgmtm server, but cannot be sure it wasn't seen
by other users during transport. Thus by including secrets in
the XML, you could be exposing them to the message bus adminstrator.
Taking your diagram, I think I would generalize it still further
to allow for mgmt server to be optional separate from key server,
and to introduce a "message bus" between MGMT server & libvirt.
Really pushing the limits of ASCII art...
MGMT client
|
V
MGMT server <---> Key server
| ^
V |
message bus |
| |
V |
libvirt daemon <----/
|
V
QEMU
To start a VM, the sequence of steps would be:
1. MGMT client says 'boot vm X on node Y' to MGMT server
2. MGMT server says 'allow node Y access to secrets for VM X' to key server
3. MGMT server puts 'start vm X' message o nmessage bus to node Y
4. libvirt on node Y says 'give secrets for vm X' to key server
5. libvirt spawns QEMU, passing secrets
If node E intercepts the 'start vm X' message from the bus
it cannot see the secrets directly. If node 'E' asks the
key server for secrets for VM X, it will be refused, since
the MGMT server has not authorized it to see them.
If sysadmin on node N, tries to migrate the VM to node E,
node E will not be able to start the VM, since it again has
not been authorized to fetch the secrets.
To actually allow a migration, the sequence of steps would be
1. MGMT client says 'migrate vm X from node Y to node Z' to MGMT server
2. MGMT server says 'allow node Z access to secrets S' to key server
3. MGMT server puts 'migrate vm X to node Z' message on message bus to node Y
4. libvirt on node Y issues the migration operation, passing XML
for the VM to node Z
5. libvirt on node Z says 'give secrets for vm X' to key server
6. libvirt on node Z spawns QEMU, passing secrets
7. MGMT server says 'deny node Y access to secrets for vm X' to key server
I believe this deals with all the use cases, specifically
- A node can only see secrets for VMs it is configured to run
- MGMT server does not need to ever see the secrets itself. It
merely controls access for which nodes can see them, and can
request generation of new secrets
- Messages between the MGMT server & libvirtd do not need to
be encrypted, since they don't include secrets
- Other users who authenticate to libvirt on a node cannot
move a VM to an unauthorized node, since it can't see secrets
- VM save memory images (which include the fully XML) do not ever
expose the secrets. So VM save image cannot be restored on an
unauthorized node.
> If all nodes are trusted, what additional functionality does
> volume encryption with per-volume keys provide? If the nodes
> are trusted to read only data from the domains they currently
> host, the nodes could just as well use an encrypted local hard
> drive to store all images, or share a single key to encrypt
> all images stored on a NFS/SAN.
I think the outline above should address your concern here.
> > Yes, libvirt will grow the concept of a user in the future, as well as
> > access controls on (user, object, operation) tuples
>
> Overall, it seems this all boils down to one thing - is libvirt intended
> to be a simple "virtualization server" that locally performs requested
> operations, managed by a separate "virtualization management" client
> that has full knowledge about the site (nodes, storage, secrets, access
> rights), with clients connecting to the "virtualization management"
> software instead of libvirtd, or a complete "virtualization management
> solution" that integrates all site-wide knowedge, accessed by dumb
> clients using the libvirt protocol?
libvirt is intended to deal with resources on a single node, and provide
a means for command & control of that node. I don't think anything I
have said suggests / requires that libvirt needs full knowledge of the
whole site. What we're dealing with here is a layered architecture,
that can be deployed in a variety of different scenarios.
The MGMT server authenticates & authorizes connections from MGMT clients.
Libvirtd can be made to authenticate & authorize connections from the
MGMT server and possibly other clients apps - that's a deployment
decision of the specific MGMT application / adminstrator. libvirtd does
not need to know the secrets of all VMs for all nodes, it only needs the
ability to use key service which can provide secrets it needs. That
key service may be local to the node, it may be on storage shared by
several nodes, or it may be over the network on a remote node. A remote
key service should of course authenticate / authorize each libvirtd
that connects and requests access to keys, to ensure a node cannot access
keys for VMs on another node. By providing a pluggable key service
backend for libvirt I believe we can satisfy all these different deployment
scenarios that have been discussed thus far, without needing to favour
one. I think it is very important that all these deployment scenarios
can be supported without including the secrets in the XML config, since
there are faaaar too many ways in which the XML config itself may be
exposed to undesirable places.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
--
Libvir-list mailing list
Libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list