On Fri, Jul 22, 2011 at 07:41:08AM -0400, Jeff Layton wrote: > On Fri, 22 Jul 2011 13:24:22 +0200 > Michael Schwartzkopff <misch@xxxxxxxxxxxxxxxxx> wrote: > > > > On Thu, 21 Jul 2011 15:40:05 +0200 > > > > > > Michael Schwartzkopff <misch@xxxxxxxxxxxxxxxxx> wrote: > > > > Hi, > > > > > > > > setting up a linux NFS server cluster I experience a strange problem > > > > during a failover. A client can access a file only after 60 to 90 > > > > seconds. On the line I see a NFS4ERR_GRACE from the server to the > > > > client. > > > > > > > > I already set the /proc/fs/nfsd/nfsv4leastime to 10. > > > > > > > > Any other idea? > > > > > > > > For the tests I have a simple setup: > > > > > > > > One host with a simple script that simulates failover: > > > > > > > > #!/bin/bash > > > > > > > > ip a d 1.2.3.4/24 dev eth0 > > > > exportfs -u *:/srv/nfs/home > > > > exportfs -u *:/srv/nfs > > > > /etc/init.d/nfsserver stop > > > > # > > > > /etc/init.d/nfsserver start > > > > exportfs -o fsid=0,rw,crossmnt,no_root_squash *:/srv/nfs > > > > exportfs -o fsid=1000,rw,mountpoint,no_root_squash *:/srv/nfs/home > > > > ip a a 1.2.3.4/24 dev eth0 > > > > > > This is normal. The grace period is there to allow clients to reclaim > > > their state without other clients racing in and grabbing their locks on > > > the new server, etc. > > > > Ok. But this is the same client that cannot its own data. > > > > That may be the case in your situation, but the protocol has to allow for that. We could be a bit smarter here: - Use the NFSv4.1 RECLAIM_COMPLETE to identify when all clients have reclaimed state, if possible. - Skip the grace period when we *know* no clients held open state on the previous boot. (But we'd have to think about v2/v3 clients: the kernel doesn't know about them. Possibly we can assume they don't use share locks, hence don't need to wait on them for v4 opens. But we *do* actually have share lock procedures in NLM--I wonder if anyone uses them? They aren't consistent with v4 opens, argh.) > > > You can play with /proc/fs/nfsd/nfsv4gracetime too but I'd be very > > > leery of setting that too low. It should never be lower than the > > > previous leasetime (see the comments on write_gracetime in the kernel). > > > > I did already set this to 10 seconds without success. The client still has to > > wait 60 seconds to access its data. > > > > You may need to restart the server and remount the clients in order for it to work. Note /proc/sys/fs/nfs/nlm_grace_period needs to be shrunk too. --b. > > > By the way: Is there a nice way to set this during startup of the nfsserver, > > i.e. a mount option for nfsd? > > > > Not currently, but a module parm for this would seem to make more sense > than this file in /proc. > > -- > Jeff Layton <jlayton@xxxxxxxxxx> > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
