On Sat, Sep 26, 2015 at 8:03 PM, Matt Benjamin <mbenjamin@xxxxxxxxxx> wrote: > Hi John, > > I prototyped an invalidate upcall for libcephfs and the Gasesha Ceph fsal, building on the Client invalidation callback registrations. > > As you suggested, NFS (or AFS, or DCE) minimally expect a more generic "cached vnode may have changed" trigger than the current inode and dentry invalidates, so I extended the model slightly to hook cap revocation, feedback appreciated. In cap_release, we probably need to be a bit more discriminating about when to drop, e.g. if we've only lost our exclusive write caps, the rest of our metadata might all still be fine to cache. Is ganesha in general doing any data caching? I think I had implicitly assumed that we were only worrying about metadata here but now I realise I never checked that. The awkward part is Client::trim_caps. In the Client::trim_caps case, the lru_is_expirable part won't be true until something has already been invalidated, so there needs to be an explicit hook there -- rather than invalidating in response to cap release, we need to invalidate in order to get ganesha to drop its handle, which will render something expirable, and finally when we expire it, the cap gets released. In that case maybe we need a hook in ganesha to say "invalidate everything you can" so that we don't have to make a very large number of function calls to invalidate things. In the fuse/kernel case we can only sometimes invalidate a piece of metadata (e.g. we can't if its flocked or whatever), so we ask it to invalidate everything. But perhaps in the NFS case we can always expect our invalidate calls to be respected, so we could just invalidate a smaller number of things (the difference between actual cache size and desired)? John > > git@xxxxxxxxxx:linuxbox2/ceph.git , branch invalidate > git@xxxxxxxxxx:linuxbox2/nfs-ganesha.git , branch ceph-invalidates > > thanks, > > Matt > > -- > Matt Benjamin > Red Hat, Inc. > 315 West Huron Street, Suite 140A > Ann Arbor, Michigan 48103 > > http://www.redhat.com/en/technologies/storage > > tel. 734-761-4689 > fax. 734-769-8938 > cel. 734-216-5309 > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
