On Mon, 2019-03-11 at 21:59 +0000, Sage Weil wrote:
> On Mon, 11 Mar 2019, Jeff Layton wrote:
> > This may, on its face, sound like a stupid question, but I'm going over
> > the rules to ensure that our cap handling rules will be able to properly
> > support buffered creates/unlinks.
> >
> > Dentry leases seem to be really poorly documented. The "rules" for them
> > are unclear, but also the rationale. What was the supposed benefit of
> > the dentry leases in ceph over just relying on appropriate caps on the
> > parent directory?
>
> The dentry leases are granular, while the dir cap covers the whole
> directory.
>
> You get a lease on an individual dentry via a lookup, without knowing
> anything about any other sibling dentries. The direcotry might have
> thousands of files! You can satisfy a client-side lookup with a dentry
> lease. So a lookup of /home/sage/a/b/c will get a lease on /home/sage
> without having to know anything about the other zillion users, and the
> client will generally have leases on dentries connecting any active
> subtrees.
>
> OTOH, the directory Fs cap covers the entire directory. With this cap you
> can satisfy any lookup on any file in the directory if you have a positive
> hit in the client's cache. If you also have the directory's COMPLETE
> flag, you can satisfy a negative lookup locally, and also satisfy readdir
> client-side. You only get the COMPLETE flag after a full readdir without
> racing directory modifications. You could get the Fs cap on /home in teh
> example above and it would let you trust any /home/* dentries in cache,
> but you'd have to invalidate all of them on mkdir /home/newuser.
>
> The dentry leases are handed out at the MDS's discretion, so you could
> imagine the MDS deciding not to give out dentry leases on a directory it
> doesn't expect modifications on, avoiding the overhead of establishing any
> lease state.
>
Thanks Sage. So should I take this to mean that caps on the parent
directory and dentry leases should be handled (mostly) independently?
I ask specifically because Zheng seemed to indicate something different
with his comment here:
http://tracker.ceph.com/issues/24461#note-16
It may be easier to sort this out with a concrete example:
Suppose we have a directory (/foo) with a file in it (/foo/bar). A
client has Fs caps on /foo and a dentry lease on /foo/bar. Another
client then creates a new file in that directory (/foo/baz). At that
point the MDS will revoke Fs caps from the first client.
Will the dentry lease on /foo/bar also be recalled at that point? I'm
assuming not since /foo/bar wasn't touched.
> > In principle I suppose it would allow you to continue caching most of
> > your dentries when only some small subset changes. Was that it or was
> > there some other reason to add them?
> >
> > If that was the reason, what was the rationale for making them time-
> > based (they all have a lease_ttl)?
>
> A TTL seemed like a good idea at the time. It lets the dentry lease state
> expire without additional chatter on the wire, either for a release or
> revocation of a not-recently-used dentry. There wasn't any comprehensive
> analysis done to confirm this was a net win or not.
>
Makes sense, thanks.
--
Jeff Layton <jlayton@xxxxxxxxxxxxxxx>
