On Thu, 2022-04-14 at 14:52 +1000, NeilBrown wrote:
> On Thu, 14 Apr 2022, Trond Myklebust wrote:
> > On Thu, 2022-04-14 at 14:13 +1000, NeilBrown wrote:
> > > On Wed, 13 Apr 2022, Chuck Lever III wrote:
> > > >
> > > > > On Apr 12, 2022, at 3:06 AM, NeilBrown <neilb@xxxxxxx> wrote:
> > > > >
> > > > > On Tue, 12 Apr 2022, Chuck Lever wrote:
> > > > > > +
> > > > > > +If a client's "co_ownerid" string or principal are not
> > > > > > stable,
> > > > > > +state recovery after a server or client reboot is not
> > > > > > guaranteed.
> > > > > > +If a client unexpectedly restarts but presents a different
> > > > > > +"co_ownerid" string or principal to the server, the server
> > > > > > orphans
> > > > > > +the client's previous open and lock state. This blocks
> > > > > > access
> > > > > > to
> > > > > > +locked files until the server removes the orphaned state.
> > > > > > +
> > > > > > +If the server restarts and a client presents a changed
> > > > > > "co_ownerid"
> > > > > > +string or principal to the server, the server will not
> > > > > > allow
> > > > > > the
> > > > > > +client to reclaim its open and lock state, and may give
> > > > > > those
> > > > > > locks
> > > > > > +to other clients in the mean time. This is referred to as
> > > > > > "lock
> > > > > > +stealing".
> > > > >
> > > > > This is not a possible scenario with Linux NFS client. The
> > > > > client
> > > > > assembles the string once from various sources, then uses it
> > > > > consistently at least until unmount or reboot. Is it worth
> > > > > mentioning?
> > > >
> > > > Neil, thanks for the eyes-on. I've integrated the other
> > > > suggestions
> > > > in your reply. However there are some corner cases here that
> > > > I'd
> > > > like to consider before proceeding.
> > > >
> > > > Generally, preserving the cl_owner_id string is good defense
> > > > against
> > > > lock stealing. Looks like the Linux NFS client didn't do that
> > > > before
> > > > ceb3a16c070c ("NFSv4: Cache the NFSv4/v4.1 client owner_id in
> > > > the
> > > > struct nfs_client").
> > > >
> > > > If a server filesystem is migrated to a server that the client
> > > > hasn't
> > > > contacted before, and the client's uniquifier or hostname has
> > > > changed
> > > > since the client established its lease with the first server,
> > > > there
> > > > is the possibility of lock stealing during transparent state
> > > > migration.
> > > >
> > > > I'm also not certain how the Linux NFS client preserves the
> > > > principal
> > > > that was used when a lease is first established. It's going to
> > > > use
> > > > Kerberos if possible, but what if the kernel's cred cache
> > > > expires
> > > > and
> > > > the keytab has been altered in the meantime? I haven't walked
> > > > through
> > > > that code carefully enough to understand whether there is still
> > > > a
> > > > vulnerability.
> > > >
> > >
> > > I don't think id stability relates to lock stealing.
> > >
> > > - global uniqueness guards against stealing
> > > - stability guards against misplacing locks during migration
> > > ("stolen"
> > > - seems an inappropriate term as no entity an be blamed for
> > > stealing).
> > >
> > > Certainly stability of both the identity and the credential are
> > > important. If that stability is not provided then that is a
> > > kernel
> > > bug.
> > > As you say, ceb3a16c070c fixed a bug in this area.
> > > I don't think this document is an appropriate place to warn
> > > against
> > > kernel bugs - that doesn't fit with the purpose given in the
> > > intro.
> > >
> > > I don't know know if the credential can change either - I hope
> > > not.
> > > IF the credential can actually change, that would be a kernel
> > > bug.
> > > IF the same credential cannot be renewed, that is a separate
> > > problem,
> > > but should be treated like any other credential that cannot be
> > > renewed.
> > >
> > > I won't argue strongly against this text - I just don't see how
> > > it is
> > > appropriate and think it could be confusing.
> > >
> >
> > See RFC 5661, Section 2.4.3.
> >
> > It is up to the server to enforce the correspondence between the
> > lease
> > and the principal that owns that lease.
> >
>
> I'm sorry but I don't understand the relevance of that statement.
>
> I think we are talking about the client, and the importance of using
> a
> stable identity and principal. Obvious if the client changes either
> then the correct action of the server will cause problems for the
> client.
> I'm saying that the sys admin shouldn't need to worry about this as
> the
> NFS kernel module should ensure these things don't change - and I
> think
> that it does, though I'm not certain about the principal as I haven't
> checked.
The use of the phrase "lock stealing" implies that there is a security
aspect. I'm saying that the lease is associated with a principal that
is supposed to be unique to the client in the ideal world of strong
authentication. The server enforces that association.
On its side, the kernel client doesn't know or care about principals.
It doesn't know or care about keytabs and their contents. It just knows
that it presents the same credential to the RPC layer, and then
whatever happens, happens. If the user wants to shoot themselves in the
foot by changing the principal associated with the credential, then the
kernel client will happily pull that trigger and watch the
NFS4ERR_CLID_INUSE bullets come raining down until that lease expires.
--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@xxxxxxxxxxxxxxx
