On Thu, Dec 03, 2020 at 09:57:41PM +0000, Trond Myklebust wrote: > On Thu, 2020-12-03 at 13:45 -0800, Frank Filz wrote: > > > On Thu, 2020-12-03 at 16:13 -0500, bfields@xxxxxxxxxxxx wrote: > > > > On Thu, Dec 03, 2020 at 08:27:39PM +0000, Trond Myklebust wrote: > > > > > On Thu, 2020-12-03 at 13:51 -0500, bfields wrote: > > > > > > I've been scratching my head over how to handle reboot of a > > > > > > re- > > > > > > exporting server. I think one way to fix it might be just to > > > > > > allow the re- export server to pass along reclaims to the > > > > > > original > > > > > > server as it receives them from its own clients. It might > > > > > > require > > > > > > some protocol tweaks, I'm not sure. I'll try to get my > > > > > > thoughts > > > > > > in order and propose something. > > > > > > > > > > > > > > > > It's more complicated than that. If the re-exporting server > > > > > reboots, > > > > > but the original server does not, then unless that re-exporting > > > > > server persisted its lease and a full set of stateids > > > > > somewhere, it > > > > > will not be able to atomically reclaim delegation and lock > > > > > state on > > > > > the server on behalf of its clients. > > > > > > > > By sending reclaims to the original server, I mean literally > > > > sending > > > > new open and lock requests with the RECLAIM bit set, which would > > > > get > > > > brand new stateids. > > > > > > > > So, the original server would invalidate the existing client's > > > > previous clientid and stateids--just as it normally would on > > > > reboot--but it would optionally remember the underlying locks > > > > held by > > > > the client and allow compatible lock reclaims. > > > > > > > > Rough attempt: > > > > > > > > > > > > https://wiki.linux-nfs.org/wiki/index.php/Reboot_recovery_for_re-expor > > > > t_servers > > > > > > > > Think it would fly? > > > > > > So this would be a variant of courtesy locks that can be reclaimed > > > by the client > > > using the reboot reclaim variant of OPEN/LOCK outside the grace > > > period? The > > > purpose being to allow reclaim without forcing the client to > > > persist the original > > > stateid? > > > > > > Hmm... That's doable, but how about the following alternative: Add > > > a function > > > that allows the client to request the full list of stateids that > > > the server holds on > > > its behalf? > > > > > > I've been wanting such a function for quite a while anyway in order > > > to allow the > > > client to detect state leaks (either due to soft timeouts, or due > > > to reordered > > > close/open operations). > > > > Oh, that sounds interesting. So basically the re-export server would > > re-populate it's state from the original server rather than relying > > on it's clients doing reclaims? Hmm, but how does the re-export > > server rebuild its stateids? I guess it could make the clients > > repopulate them with the same "give me a dump of all my state", using > > the state details to match up with the old state and replacing > > stateids. Or did you have something different in mind? > > > > I was thinking that the re-export server could just use that list of > stateids to figure out which locks can be reclaimed atomically, and > which ones have been irredeemably lost. The assumption is that if you > have a lock stateid or a delegation, then that means the clients can > reclaim all the locks that were represented by that stateid. I'm confused about how the re-export server uses that list. Are you assuming it persisted its own list across its own crash/reboot? I guess that's what I was trying to avoid having to do. --b.
