On Mon, Nov 04, 2024 at 01:05:27PM +1100, NeilBrown wrote:
> On Mon, 04 Nov 2024, cel@xxxxxxxxxx wrote:
> > From: Chuck Lever <chuck.lever@xxxxxxxxxx>
> >
> > I noticed that recently, simple operations like "make" started
> > failing on NFSv3 mounts of ext4 exports. Network capture shows that
> > READDIRPLUS operated correctly but READDIR failed with
> > NFS3ERR_INVAL. The vfs_llseek() call returned EINVAL when it is
> > passed a non-zero starting directory cookie.
> >
> > I bisected to commit c689bdd3bffa ("nfsd: further centralize
> > protocol version checks.").
> >
> > Turns out that nfsd3_proc_readdir() does not call fh_verify() before
> > it calls nfsd_readdir(), so the new fhp->fh_64bit_cookies boolean is
> > not set properly. This leaves the NFSD_MAY_64BIT_COOKIE unset when
> > the directory is opened.
> >
> > For ext4, this causes the wrong "max file size" value to be used
> > when sanity checking the incoming directory cookie (which is a seek
> > offset value).
> >
> > Both NFSv2 and NFSv3 READDIR need to call fh_verify() now to ensure
> > the new boolean fields are properly initialized.
> >
> > There is a risk that these procedures might now return a status code
> > that is not valid (by spec), or that operations that are currently
> > allowed might no longer be.
>
> This seems like the wrong fix to me.
I agree that there is more than one alternative fix.
I chose this one because it's late in the cycle, and this fix seemed
least risky. I intended that further improvement would be done after
the v6.13-rc window closes.
> Why should nfsd4_proc_readdir()
> call fh_verify() when nfsd_readdir() already does that via
> nfsd_open()??
NFSv4 COMPOUND processing calls fh_verify() already to verify the
current FH. So when nfsd4_encode_dirlist4() calls nfsd_readdir(),
that fhp is already instantiated properly.
nfsd3_proc_readdirplus() needs the fh_verify() call because it needs
to grab a handle on the FH's export to check the "nordplus" export
option. The fhp is instantiated before nfsd_readdir() is called.
So that double fh_verify() is already in the execution path of the
more complex READDIR operations. It doesn't seem harmful to add to
nfsd3_proc_readdir() -- but it's not terribly aesthetic.
> The only reason is to set NFSD_MAY_64BIT_COOKIE.
>
> Let's just get rid of
> that flag instead and use fhp->fh_64bit_cookies directly.
64BIT_COOKIE doesn't seem to fit with the semantics of the other
MAY flags, so I like this clean-up.
> Like this:
>
> diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h
> index b8470d4cbe99..51b2074d92a5 100644
> --- a/fs/nfsd/trace.h
> +++ b/fs/nfsd/trace.h
> @@ -86,7 +86,6 @@ DEFINE_NFSD_XDR_ERR_EVENT(cant_encode);
> { NFSD_MAY_NOT_BREAK_LEASE, "NOT_BREAK_LEASE" }, \
> { NFSD_MAY_BYPASS_GSS, "BYPASS_GSS" }, \
> { NFSD_MAY_READ_IF_EXEC, "READ_IF_EXEC" }, \
> - { NFSD_MAY_64BIT_COOKIE, "64BIT_COOKIE" }, \
> { NFSD_MAY_LOCALIO, "LOCALIO" })
>
> TRACE_EVENT(nfsd_compound,
> diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
> index 22325b590e17..a86b6f6a3b98 100644
> --- a/fs/nfsd/vfs.c
> +++ b/fs/nfsd/vfs.c
> @@ -903,7 +903,7 @@ __nfsd_open(struct svc_rqst *rqstp, struct svc_fh *fhp, umode_t type,
> goto out;
> }
>
> - if (may_flags & NFSD_MAY_64BIT_COOKIE)
> + if (fhp->fh_64bit_cookies)
> file->f_mode |= FMODE_64BITHASH;
> else
> file->f_mode |= FMODE_32BITHASH;
Is there any reason this logic couldn't be moved out to
nfsd_readdir(), after the nfsd_open() call?
> @@ -2174,9 +2174,6 @@ nfsd_readdir(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t *offsetp,
> loff_t offset = *offsetp;
> int may_flags = NFSD_MAY_READ;
>
> - if (fhp->fh_64bit_cookies)
> - may_flags |= NFSD_MAY_64BIT_COOKIE;
> -
> err = nfsd_open(rqstp, fhp, S_IFDIR, may_flags, &file);
> if (err)
> goto out;
> diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h
> index 3ff146522556..2ba4265a22a0 100644
> --- a/fs/nfsd/vfs.h
> +++ b/fs/nfsd/vfs.h
> @@ -31,8 +31,6 @@
> #define NFSD_MAY_BYPASS_GSS 0x400
> #define NFSD_MAY_READ_IF_EXEC 0x800
>
> -#define NFSD_MAY_64BIT_COOKIE 0x1000 /* 64 bit readdir cookies for >= NFSv3 */
> -
> #define NFSD_MAY_LOCALIO 0x2000 /* for tracing, reflects when localio used */
>
> #define NFSD_MAY_CREATE (NFSD_MAY_EXEC|NFSD_MAY_WRITE)
>
>
> If you agree I can send that as a proper patch.
Yes, please send a proper patch. If it passes testing, I will drop
the RFC patch.
> Thanks,
> NeilBrown
>
>
>
> >
> > Fixes: c689bdd3bffa ("nfsd: further centralize protocol version checks.")
> > Signed-off-by: Chuck Lever <chuck.lever@xxxxxxxxxx>
> > ---
> > fs/nfsd/nfs3proc.c | 6 ++++++
> > fs/nfsd/nfsproc.c | 8 +++++++-
> > 2 files changed, 13 insertions(+), 1 deletion(-)
> >
> > diff --git a/fs/nfsd/nfs3proc.c b/fs/nfsd/nfs3proc.c
> > index dfcc957e460d..48bcdc96b867 100644
> > --- a/fs/nfsd/nfs3proc.c
> > +++ b/fs/nfsd/nfs3proc.c
> > @@ -592,6 +592,11 @@ nfsd3_proc_readdir(struct svc_rqst *rqstp)
> > resp->cookie_offset = 0;
> > resp->rqstp = rqstp;
> > offset = argp->cookie;
> > +
> > + resp->status = fh_verify(rqstp, &resp->fh, S_IFDIR, NFSD_MAY_NOP);
> > + if (resp->status != nfs_ok)
> > + goto out;
> > +
> > resp->status = nfsd_readdir(rqstp, &resp->fh, &offset,
> > &resp->common, nfs3svc_encode_entry3);
> > memcpy(resp->verf, argp->verf, 8);
> > @@ -600,6 +605,7 @@ nfsd3_proc_readdir(struct svc_rqst *rqstp)
> > /* Recycle only pages that were part of the reply */
> > rqstp->rq_next_page = resp->xdr.page_ptr + 1;
> >
> > +out:
> > return rpc_success;
> > }
> >
> > diff --git a/fs/nfsd/nfsproc.c b/fs/nfsd/nfsproc.c
> > index 97aab34593ef..ebe8fd3c9ddd 100644
> > --- a/fs/nfsd/nfsproc.c
> > +++ b/fs/nfsd/nfsproc.c
> > @@ -586,11 +586,17 @@ nfsd_proc_readdir(struct svc_rqst *rqstp)
> > resp->common.err = nfs_ok;
> > resp->cookie_offset = 0;
> > offset = argp->cookie;
> > +
> > + resp->status = fh_verify(rqstp, &resp->fh, S_IFDIR, NFSD_MAY_NOP);
> > + if (resp->status != nfs_ok)
> > + goto out;
> > +
> > resp->status = nfsd_readdir(rqstp, &argp->fh, &offset,
> > &resp->common, nfssvc_encode_entry);
> > nfssvc_encode_nfscookie(resp, offset);
> > -
> > fh_put(&argp->fh);
> > +
> > +out:
> > return rpc_success;
> > }
> >
> > --
> > 2.47.0
> >
> >
>
--
Chuck Lever
