Re: when rpc.mountd flushes auth.unix.gid

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, 17 Oct 2014 21:21:18 -0500
Tom Haynes <thomas.haynes@xxxxxxxxxxxxxxx> wrote:

> 
> On Oct 17, 2014, at 6:37 PM, Jeff Layton <jlayton@xxxxxxxxxxxxxxx> wrote:
> 
> > On Fri, 17 Oct 2014 17:24:14 -0500
> > Tom Haynes <thomas.haynes@xxxxxxxxxxxxxxx> wrote:
> > 
> >> 
> >> 
> >> 
> >>> On Oct 17, 2014, at 4:06 PM, Jeff Layton <jeff.layton@xxxxxxxxxxxxxxx> wrote:
> >>> 
> >>> On Fri, 17 Oct 2014 09:42:14 -0500
> >>> Colin Hudler <chudler@xxxxxxxxxxxxxxx> wrote:
> >>> 
> >>>> We have a few hundred computers mounting an NFS server in a typical 
> >>>> LDAP-based users (nss) setup. We frequently add and remove exports and 
> >>>> use exportfs -r to update etab. Every time we do so, the clients report 
> >>>> "NFS server not responding" and start backing off their requests. After 
> >>>> a painful 3-5 minutes, they recover and life is normal again.
> >>>> 
> >>>> We discovered that when the rpc.mountd cache flushing occurs, our NIS 
> >>>> system is overwhelmed with grouplist requests and this obviously blocks 
> >>>> things. We are working on that problem separately, and I admit this to 
> >>>> be a weakness in our setup. My question is simple.
> >>>> 
> >>>> Why does it flush auth.unix.gid when the etab changed? I think it makes 
> >>>> unnecessary work for rpc.mountd because the gids are unlikely to have 
> >>>> changed, and they already have a reasonable expiration policy.
> >>> 
> >>> Most likely because no one really cared until now.
> >>> 
> >>> When exports change, cache_flush() is called and that function flushes
> >>> out all of the kernel caches.
> >>> 
> >>> I expect that could be made to do something a bit more granular, but
> >>> you may need to do some archaeology in mountd/exportfs (and the kernel)
> >>> to ensure that you're not missing anything.
> >>> 
> >> 
> >> One thing would be to not remove the exports which are going to be added back in.
> >> 
> >> The catch here is that you have to account for new entries which need to be added.
> >> 
> >> 
> > 
> > I'm not sure that flushing the uid or gid caches is really necessary on
> > an exports change at all. I don't think we expect that info to change.
> 
> Is there a manual way to flush these caches?
> 
> Bump down the default TTL?
> 
> 

The manual way is to write to /proc/net/rpc/*/flush (which is what
cache_flush() in nfs-utils does). The comments over it say:

/* flush the kNFSd caches.
 * Set the flush time to the mtime of _PATH_ETAB or
 * if force, to now.
 * the caches to flush are:
 *  auth.unix.ip nfsd.export nfsd.fh
 */

...but it looks like auth.unix.gid was added in 2007 and the
comment wasn't updated.

> > 
> > In practical terms, we might be able to change exportfs to just flush
> > the nfsd.fh and nfsd.export caches instead of a full cache_flush() ?
> > 
> > -- 
> > Jeff Layton <jlayton@xxxxxxxxxxxxxxx>
> 


-- 
Jeff Layton <jlayton@xxxxxxxxxxxxxxx>
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux