Hi Bruce,
In accordance with the traditional Good Friday theme of "suffering", here
is the first draft of the NFSd lock scalability patches for your perusal.
I've only done light testing so far, but since there is a total of 70
patches to review, I figured you might want to start sooner rather than
waiting for the QA folks to finish testing.
During the process, I did notice a bug or two in the existing NFSd code,
and so the first 3 patches are bugfixes that you may want to apply
irrespectively.
In general, what I've tried to do is add reference counting wherever
there is a need to carry structures around outside locks, and to
ensure that we make structures invisible to lookups before we attempt
to free them.
There are still a few optimisations that might be worth considering in
order to reduce the frequency of global locks that are taken. Suggestions
include:
1) Caching the nfs4_client in the struct nfsd4_compound_state
2) Caching the current/save_stateid as references to the actual nfs4_stid in the struct nfsd4_compound_state
3) Converting the stateid/sessionid/clientid global table lookups to use RCU.
4) Converting the file_hashtbl lookups to use RCU
Cheers
Trond
Benny Halevy (4):
nfsd4: rename recall_lock to state_lock
nfsd4: use cl_lock to synchronize all stateid idr calls
nfsd4: hash deleg stateid only on successful nfs4_set_delegation
nfsd4: use state_lock for delegation hashing
Trond Myklebust (66):
NFSd: Ensure we clear the cstate->slot in nfsd4_proc_compound
NFSd: Move default initialisers from create_client() to alloc_client()
NFSd: call rpc_destroy_wait_queue() from free_client()
NFSd: Remove 'inline' designation for free_client()
NFSd: Mark nfs4_free_lockowner and nfs4_free_openowner as static
functions
NFSd: Avoid taking state_lock while holding inode lock in
nfsd_break_one_deleg
NFSd: Ensure delegation setup is safe w.r.t. break_lease()
NFSd: Add fine grained protection for the nfs4_file->fi_stateids list
NFSd: Clean up nfs4_preprocess_stateid_op
NFSd: Add a mutex to protect the NFSv4.0 open owner replay cache
NFSd: Add locking to the nfs4_file->fi_fds[] array
NFSd: Protect the nfs4_file delegation fields using the fi_lock
NFSd: Lock owners are not per open stateid
NFSd: Clean up helper __release_lock_stateid
NFSd: Allow lockowners to hold several stateids
NFSd: NFSv4 lock-owners are not associated to a specific file
NFSd: Get rid of the lockowner_ino_hashtbl
NFSd: Cleanup nfs4svc_encode_compoundres
NFSd: Don't get a session reference without a client reference
NFSd: Move the delegation reference counter into the struct nfs4_stid
NFSd: Simplify stateid management
NFSd: Fix delegation revocation
NFSd: Don't let the laundromat reap clients that are referenced
NFSd: Add reference counting to the lock and open stateids
NFSd: Add a struct nfs4_file field to struct nfs4_stid
NFSd: Replace delegation->dl_file with the dl_stid.sc_file
NFSd: Replace nfs4_ol_stateid->st_file with the st_stid.sc_file
NFSd: Ensure stateids remain unique until they are freed
NFSd: Ensure atomicity of stateid destruction and idr tree removal
NFSd: Fix atomicity of delegation counter
NFSd: Slight cleanup of find_stateid()
NFSd: Add reference counting to find_stateid
NFSd: nfs4_preprocess_seqid_op should only set *stpp on success
NFSd: Add reference counting to lock stateids
NFSd: nfsd4_locku() must reference the lock stateid
NFSd: Ensure that nfs4_open_delegation() references the delegation
stateid
NFSd: nfsd4_process_open2() must reference the delegation stateid
NFSd: nfsd4_process_open2() must reference the open stateid
NFSd: Prepare nfsd4_close() for open stateid referencing
NFSd: nfsd4_open_confirm() must reference the open stateid
NFSd: Add reference counting to nfs4_preprocess_confirmed_seqid_op
NFSd: Migrate the stateid reference into nfs4_preprocess_seqid_op
NFSd: Migrate the stateid reference into nfs4_lookup_stateid()
NFSd: Migrate the stateid reference into nfs4_find_stateid_by_type()
NFSd: Cleanup - Let nfsd4_lookup_stateid() take a cstate argument
NFSd: Use the session->se_client in lookup_clientid()
NFSd: Convert nfsd4_process_open1() to work with lookup_clientid()
NFSd: Convert nfs4_check_open_reclaim() to work with lookup_clientid()
NFSd: Ensure struct nfs4_client is unhashed before we try to destroy
it
NFSd: Ensure that the laundromat unhashes the client before releasing
locks
NFSd: Don't require client_lock in free_client
NFSd: Move create_client() call outside the lock
NFSd: Protect unconfirmed client creation using client_lock
NFSd: Protect session creation and client confirm using client_lock
NFSd: Protect nfsd4_destroy_clientid using client_lock
NFSd: Ensure lookup_clientid() takes client_lock
NFSd: Remove nfs4_lock_state(): nfs4_preprocess_stateid_op()
NFSd: Remove nfs4_lock_state(): nfsd4_test_stateid/nfsd4_free_stateid
NFSd: Remove nfs4_lock_state(): nfsd4_release_lockowner
NFSd: Remove nfs4_lock_state(): nfsd4_lock/locku/lockt()
NFSd: Remove nfs4_lock_state(): nfsd4_open_downgrade + nfsd4_close
NFSd: Remove nfs4_lock_state(): nfsd4_delegreturn()
NFSd: Remove nfs4_lock_state(): nfsd4_open and nfsd4_open_confirm
NFSd: Remove nfs4_lock_state(): exchange_id, create/destroy_session()
NFSd: Remove nfs4_lock_state(): setclientid, setclientid_confirm,
renew
NFSd: Remove nfs4_lock_state(): reclaim_complete()
fs/nfsd/netns.h | 4 -
fs/nfsd/nfs4callback.c | 18 +-
fs/nfsd/nfs4proc.c | 28 +-
fs/nfsd/nfs4state.c | 1239 ++++++++++++++++++++++++++++++------------------
fs/nfsd/nfs4xdr.c | 15 +-
fs/nfsd/state.h | 46 +-
fs/nfsd/xdr4.h | 21 +-
7 files changed, 819 insertions(+), 552 deletions(-)
--
1.9.0
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
