On 10/13/2017 08:09 PM, Olga Kornievskaia wrote: > On Fri, Oct 13, 2017 at 5:26 PM, J. Bruce Fields <bfields@xxxxxxxxxx> wrote: >> On Fri, Oct 13, 2017 at 04:54:02PM -0400, Olga Kornievskaia wrote: >>> To do asynchronous copies, NFSD creates a new kthread to handle the request. >>> Upon receiving the COPY, it generates a unique copy stateid (stored in a >>> global list for keeping track of state for OFFLOAD_STATUS to be queried by), >>> starts the thread, and replies back to the client. nfsd4_copy arguments that >>> are allocated on the stack are copies for the kthread. >>> >>> In the async copy handler, it calls into VFS copy_file_range() (for synch >>> we keep the 4MB chunk and requested size for the async copy). If error is >>> encountered it's saved but also we save the amount of data copied so far. >>> Once done, the results are queued for the callback workqueue and sent via >>> CB_OFFLOAD. >>> >>> When the server received an OFFLOAD_CANCEL, it will find the kthread running >>> the copy and will send a SIGPENDING and kthread_stop() and it will interrupt >>> the ongoing do_splice() and once vfs returns we are choosing not to send >>> the CB_OFFLOAD back to the client. >>> >>> When the server receives an OFFLOAD_STATUS, it will find the kthread running >>> the copy and will query the i_size_read() of the associated filehandle of >>> the destination file and return the result. >> >> That assumes we're copying into a previously empty file? > > Sigh. Alright, then it's back to my original solution where I broke > everything into 4MB calls and kept track of bytes copies so far. Do they have to be 4MB calls? Assuming clients don't need a super-accurate results, you could probably use a larger copy size and still have decent copy performance. > >> >> --b. >> >>> >>> v5: >>> 1. reimplementing asynchronous copy to use kthreads instead of the workqueue >>> 2. store asynchronous copies in the list of the nfs4_client structure >>> 3. when copying nfsd4_copy datastructure for the async copy refcount the >>> nfs4_client structure as well as struct file src/dst structure >>> 4. add refcount to the nfsd4_copy structure to coordinate with offload_status >>> and offload_cancel >>> 5. offload_cancel/copy_shutdown sets the SIGPENDING in the copy's thread to >>> interrupt do_splice and calls kthread_stop to wait for the copy to stop >>> 6. offload_cancel adds MODIFIES_SOMETHING to flags >>> 7. offload_status reports the size of the destination file form i_size_read() >>> instead of before 4MB loop updated chunk size. >>> 8. for async copy call vfs_copy_file_range() with the whole copy size. still >>> keep the loop to do more the MAX_RW_COUNT that do_splice can do. >>> >>> Olga Kornievskaia (10): >>> NFSD CB_OFFLOAD xdr >>> NFSD OFFLOAD_STATUS xdr >>> NFSD OFFLOAD_CANCEL xdr >>> NFSD xdr callback stateid in async COPY reply >>> NFSD first draft of async copy >>> NFSD return nfs4_stid in nfs4_preprocess_stateid_op >>> NFSD create new stateid for async copy >>> NFSD handle OFFLOAD_CANCEL op >>> NFSD support OFFLOAD_STATUS >>> NFSD stop queued async copies on client shutdown >>> >>> fs/nfsd/netns.h | 8 ++ >>> fs/nfsd/nfs4callback.c | 97 +++++++++++++++ >>> fs/nfsd/nfs4proc.c | 321 ++++++++++++++++++++++++++++++++++++++++++++----- >>> fs/nfsd/nfs4state.c | 77 +++++++++++- >>> fs/nfsd/nfs4xdr.c | 50 ++++++-- >>> fs/nfsd/nfsctl.c | 1 + >>> fs/nfsd/state.h | 21 +++- >>> fs/nfsd/xdr4.h | 28 +++++ >>> fs/nfsd/xdr4cb.h | 10 ++ >>> 9 files changed, 576 insertions(+), 37 deletions(-) >>> >>> -- >>> 1.8.3.1 >>> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
