On Tue, Mar 12, 2019 at 8:18 PM Jeff Layton <jlayton@xxxxxxxxxxxxxxx> wrote: > > On Mon, 2019-03-11 at 21:59 +0000, Sage Weil wrote: > > On Mon, 11 Mar 2019, Jeff Layton wrote: > > > This may, on its face, sound like a stupid question, but I'm going over > > > the rules to ensure that our cap handling rules will be able to properly > > > support buffered creates/unlinks. > > > > > > Dentry leases seem to be really poorly documented. The "rules" for them > > > are unclear, but also the rationale. What was the supposed benefit of > > > the dentry leases in ceph over just relying on appropriate caps on the > > > parent directory? > > > > The dentry leases are granular, while the dir cap covers the whole > > directory. > > > > You get a lease on an individual dentry via a lookup, without knowing > > anything about any other sibling dentries. The direcotry might have > > thousands of files! You can satisfy a client-side lookup with a dentry > > lease. So a lookup of /home/sage/a/b/c will get a lease on /home/sage > > without having to know anything about the other zillion users, and the > > client will generally have leases on dentries connecting any active > > subtrees. > > > > OTOH, the directory Fs cap covers the entire directory. With this cap you > > can satisfy any lookup on any file in the directory if you have a positive > > hit in the client's cache. If you also have the directory's COMPLETE > > flag, you can satisfy a negative lookup locally, and also satisfy readdir > > client-side. You only get the COMPLETE flag after a full readdir without > > racing directory modifications. You could get the Fs cap on /home in teh > > example above and it would let you trust any /home/* dentries in cache, > > but you'd have to invalidate all of them on mkdir /home/newuser. > > > > The dentry leases are handed out at the MDS's discretion, so you could > > imagine the MDS deciding not to give out dentry leases on a directory it > > doesn't expect modifications on, avoiding the overhead of establishing any > > lease state. > > > > Thanks Sage. So should I take this to mean that caps on the parent > directory and dentry leases should be handled (mostly) independently? > yes. they are logically independent. But mds avoids issuing lease for individual dentry if it has issued Fs on directory. > I ask specifically because Zheng seemed to indicate something different > with his comment here: > > http://tracker.ceph.com/issues/24461#note-16 > > It may be easier to sort this out with a concrete example: > > Suppose we have a directory (/foo) with a file in it (/foo/bar). A > client has Fs caps on /foo and a dentry lease on /foo/bar. Another > client then creates a new file in that directory (/foo/baz). At that > point the MDS will revoke Fs caps from the first client. > > Will the dentry lease on /foo/bar also be recalled at that point? I'm > assuming not since /foo/bar wasn't touched. No, lease on /foo/bar will not be recalled. > > > > In principle I suppose it would allow you to continue caching most of > > > your dentries when only some small subset changes. Was that it or was > > > there some other reason to add them? > > > > > > If that was the reason, what was the rationale for making them time- > > > based (they all have a lease_ttl)? > > > > A TTL seemed like a good idea at the time. It lets the dentry lease state > > expire without additional chatter on the wire, either for a release or > > revocation of a not-recently-used dentry. There wasn't any comprehensive > > analysis done to confirm this was a net win or not. > > > > Makes sense, thanks. > -- > Jeff Layton <jlayton@xxxxxxxxxxxxxxx> >
