On Fri, 2024-12-06 at 11:43 +1100, NeilBrown wrote:
> If a client ever uses the highest available slot for a given session,
> attempt to allocate more slots so there is room for the client to use
> them if wanted. GFP_NOWAIT is used so if there is not plenty of
> free memory, failure is expected - which is what we want. It also
> allows the allocation while holding a spinlock.
>
> Each time we increase the number of slots by 20% (rounded up). This
> allows fairly quick growth while avoiding excessive over-shoot.
>
> We would expect to stablise with around 10% more slots available than
> the client actually uses.
>
> Signed-off-by: NeilBrown <neilb@xxxxxxx>
> ---
> fs/nfsd/nfs4state.c | 40 +++++++++++++++++++++++++++++++++++-----
> 1 file changed, 35 insertions(+), 5 deletions(-)
>
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 67dfc699e411..ec4468ebbd40 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -4235,11 +4235,6 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> slot = xa_load(&session->se_slots, seq->slotid);
> dprintk("%s: slotid %d\n", __func__, seq->slotid);
>
> - /* We do not negotiate the number of slots yet, so set the
> - * maxslots to the session maxreqs which is used to encode
> - * sr_highest_slotid and the sr_target_slot id to maxslots */
> - seq->maxslots = session->se_fchannel.maxreqs;
> -
> trace_nfsd_slot_seqid_sequence(clp, seq, slot);
> status = check_slot_seqid(seq->seqid, slot->sl_seqid,
> slot->sl_flags & NFSD4_SLOT_INUSE);
> @@ -4289,6 +4284,41 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> cstate->session = session;
> cstate->clp = clp;
>
> + /*
> + * If the client ever uses the highest available slot,
> + * gently try to allocate another 20%. This allows
> + * fairly quick growth without grossly over-shooting what
> + * the client might use.
> + */
20% seems like a reasonable place to start, but I do wonder if this
might need to be tunable under some workloads. Oh well, we can cross
that bridge if/when someone complains.
> + if (seq->slotid == session->se_fchannel.maxreqs - 1 &&
> + session->se_fchannel.maxreqs < NFSD_MAX_SLOTS_PER_SESSION) {
> + int s = session->se_fchannel.maxreqs;
> + int cnt = DIV_ROUND_UP(s, 5);
> +
> + do {
> + /*
> + * GFP_NOWAIT is a low-priority non-blocking
> + * allocation which can be used under
> + * client_lock and only succeeds if there is
> + * plenty of memory.
> + * Use GFP_ATOMIC which is higher priority for
> + * xa_store() so we are less likely to waste the
> + * effort of the first allocation.
> + */
I don't know here. Why not just use GFP_NOWAIT for the xa_store too? If
we're so memory constrained that that fails, we're probably better off
releasing the slot.
> + slot = kzalloc(slot_bytes(&session->se_fchannel),
> + GFP_NOWAIT);
> + if (slot &&
> + !xa_is_err(xa_store(&session->se_slots, s, slot,
> + GFP_ATOMIC | __GFP_NOWARN))) {
> + s += 1;
> + session->se_fchannel.maxreqs = s;
> + } else {
> + kfree(slot);
> + }
> + } while (slot && --cnt > 0);
> + }
> + seq->maxslots = session->se_fchannel.maxreqs;
> +
> out:
> switch (clp->cl_cb_state) {
> case NFSD4_CB_DOWN:
--
Jeff Layton <jlayton@xxxxxxxxxx>
