> On May 18, 2017, at 9:34 AM, Stefan Hajnoczi <stefanha@xxxxxxxxxx> wrote: > > On Tue, May 16, 2017 at 09:11:42AM -0400, J. Bruce Fields wrote: >> I think you explained this before, perhaps you could just offer a >> pointer: remind us what your requirements or use cases are especially >> for VM migration? > > The NFS over AF_VSOCK configuration is: > > A guest running on host mounts an NFS export from the host. The NFS > server may be kernel nfsd or an NFS frontend to a distributed storage > system like Ceph. A little more about these cases below. > > Kernel nfsd is useful for sharing files. For example, the guest may > read some files from the host when it launches and/or it may write out > result files to the host when it shuts down. The user may also wish to > share their home directory between the guest and the host. > > NFS frontends are a different use case. They hide distributed storage > systems from guests in cloud environments. This way guests don't see > the details of the Ceph, Gluster, etc nodes. Besides benefiting > security it also allows NFS-capable guests to run without installing > specific drivers for the distributed storage system. This use case is > "filesystem as a service". > > The reason for using AF_VSOCK instead of TCP/IP is that traditional > networking configuration is fragile. Automatically adding a dedicated > NIC to the guest and choosing an IP subnet has a high chance of > conflicts (subnet collisions, network interface naming, firewall rules, > network management tools). AF_VSOCK is a zero-configuration > communications channel so it avoids these problems. > > On to migration. For the most part, guests can be live migrated between > hosts without significant downtime or manual steps. PCI passthrough is > an example of a feature that makes it very hard to live migrate. I hope > we can allow migration with NFS, although some limitations may be > necessary to make it feasible. > > There are two NFS over AF_VSOCK migration scenarios: > > 1. The files live on host H1 and host H2 cannot access the files > directly. There is no way for an NFS server on H2 to access those > same files unless the directory is copied along with the guest or H2 > proxies to the NFS server on H1. Having managed (and shared) storage on the physical host is awkward. I know some cloud providers might do this today by copying guest disk images down to the host's local disk, but generally it's not a flexible primary deployment choice. There's no good way to expand or replicate this pool of storage. A backup scheme would need to access all physical hosts. And the files are visible only on specific hosts. IMO you want to treat local storage on each physical host as a cache tier rather than as a back-end tier. > 2. The files are accessible from both host H1 and host H2 because they > are on shared storage or distributed storage system. Here the > problem is "just" migrating the state from H1's NFS server to H2 so > that file handles remain valid. Essentially this is the re-export case, and this makes a lot more sense to me from a storage administration point of view. The pool of administered storage is not local to the physical hosts running the guests, which is how I think cloud providers would prefer to operate. User storage would be accessible via an NFS share, but managed in a Ceph object (with redundancy, a common high throughput backup facility, and secure central management of user identities). Each host's NFS server could be configured to expose only the the cloud storage resources for the tenants on that host. The back-end storage (ie, Ceph) could operate on a private storage area network for better security. The only missing piece here is support in Linux-based NFS servers for transparent state migration. -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
