On Fri, May 12, 2017 at 01:00:47PM -0400, Chuck Lever wrote: > > > On May 12, 2017, at 11:01 AM, Trond Myklebust <trondmy@xxxxxxxxxxxxxxx> wrote: > > > > On Fri, 2017-05-12 at 10:34 -0400, J. Bruce Fields wrote: > >> On Fri, May 12, 2017 at 09:27:21AM -0400, Stefan Hajnoczi wrote: > >>> Hi, > >>> I've been working on NFS over the AF_VSOCK transport > >>> (https://www.spinics.net/lists/linux-nfs/msg60292.html). AF_VSOCK > >>> resets established network connections when the virtual machine is > >>> migrated to a new host. > >>> > >>> The NFS client expects file handles and other state to remain valid > >>> upon > >>> reconnecting. This is not the case after VM live migration since > >>> the > >>> new host does not have the NFS server state from the old host. > >>> > >>> Volatile file handles have been suggested as a way to reflect that > >>> state > >>> does not persist across reconnect, but the Linux NFS client does > >>> not > >>> support volatile file handles. > >> > >> That's unlikely to change; the protocol allows the server to > >> advertise > >> volatile filehandles, but doesn't really give any tools to implement > >> them reliably. > >> > >>> I saw NFS 4.1 has a way for a new server running with the same > >>> network > >>> address of an old server to communicate that it is indeed a new > >>> server > >>> instance. If the server owner/scope in the EXCHANGE_ID response > >>> does > >>> not match the previous server's values then the server is a new > >>> instance. > >>> > >>> The implications of encountering a new server owner/scope upon > >>> reconnect > >>> aren't clear to me and I'm not sure to what extent the Linux > >>> implementation handles this case. Can anyone explain what happens > >>> if > >>> the NFS client finds a new server owner/scope after reconnecting? > >> > >> I haven't tested it, but if it reconnects to the same IP address and > >> finds out it's no longer talking to the same server, I think the only > >> correct thing it could do would be to just fail all further access. > >> > >> There's no easy solution. > >> > >> To migrate between NFS servers you need some sort of clustered NFS > >> service with shared storage. We can't currently support concurrent > >> access to shared storage from multiple NFS servers, so all that's > >> possible active/passive failover. Also, people that set that up > >> normally depend on a floating IP address--I'm not sure if there's an > >> equivalent for VSOCK. > >> > > > > Actually, this might be a use case for re-exporting NFS. If the host > > could re-export a NFS mount to the guests, then you don't necessarily > > need a clustered filesystem. > > > > OTOH, this would not solve the problem of migrating locks, which is not > > really easy to support in the current state model for NFSv4.x. > > Some alternatives: > > - Make the local NFS server's exports read-only, NFSv3 > only, and do not support locking. Ensure that the > filehandles and namespace are the same on every NFS > server. > > - As Trond suggested, all the local NFS servers accessed > via AF_SOCK should re-export NFS filesystems that > are located elsewhere and are visible everywhere. > > - Ensure there is an accompanying NFSv4 FS migration event > that moves the client's files (and possibly its open and > lock state) from the local NFS server to the destination > NFS server concurrent with the live migration. > > If the client is aware of the FS migration, it will expect > the filehandles to be the same, but it can reconstruct > the open and lock state on the destination server (if that > server allows GRACEful recovery for that client). > > This is possible in the protocol and implemented in the > Linux NFS client, but none of it is implemented in the > Linux NFS server. Great, thanks for the pointers everyone. It's clear to me that AF_VSOCK won't get NFS migration for free. Initially live migration will not be supported. Re-exporting sounds interesting - perhaps the new host could re-export the old host's file systems. I'll look into the spec and code. Stefan
Attachment:
signature.asc
Description: PGP signature
