On Aug 12, 2010, at 11:25 AM, Benny Halevy wrote:
> On Aug. 12, 2010, 2:01 +0300, Steve Dickson <SteveD@xxxxxxxxxx> wrote:
>>
>>
>> On 08/11/2010 01:29 PM, J. Bruce Fields wrote:
>>> On Wed, Aug 11, 2010 at 09:42:21AM -0700, Marc Eshel wrote:
>>>> Meeting on Thursday 08/12/10 at 9:30 AM pacific time (12:30 PM UMICH time)
>>>
>>> I won't make it.--b.
>> ditto...
>
> Given the low attendance rate, shall we just skip the call this week?
Here is what I've been up to.
Following Fred's most excellent suggestion (at last Thursday's hackers pub meeting), I created a single patch (which I called 'the whole enchilada'!) containing all of the pnfs-submit branch changes, and then proceeded to slice/dice it into logical consumable patches for submission to Trond. This beats the heck out of trying to squash 281 patches together just to re-order all the functionality.
I've attached the battle plan which I'm mostly following, and which although has some detail, is not intended to be a complete list of functions etc. I update the battle plan as I go.
I'm currently working on patch #27 called "associate layout segment with nfs_page" in the attachment, so the previous 26 patches mI tagged the tree before I started, and I do a git diff against the tag after each extraction to ensure that the tree is unchanged, and I compile after each patch that I extract from the 'whole enchilada'. I should have the tree down to 40-some patches by this weeks end. The resultant tree will be unchanged from the current pnfs-submit tree, and the last patch will be all the code that is either unused or has no effect.
The subject of patch ownership was discussed during the pNFS conference call, but I have not added any authors/signers and would really like some help. There will be more work to do but we will be _much_ closer to a submission stream when I'm done. I think the first submission (this month) should be patches numbered 1-17 in the attachment which provides LAYOUTGET functionality.
If you want to have the call today, it's fine with me.
-->Andy
Apply first (send to Trond?)
- nfs41: prevent exchange_id from sending server-only flag
- sunrpc: define xdr_decode_opaque_fixed (when is this first used?
- decode_layoutget only user
- sunrpc: don't reset buflen twice in xdr_shrink_pagelen
- nfsd: remove duplicate NFS4_STATEID_SIZE declaration
DONE 1) pnfs Kconfig
fs/nfs/Kconfig change for both pnfs and nfslayoutdriver
DONE 2) register/unregister pnfs module
introduce nfs4_pnfs.h
struct pnfs_layoutdriver_type
struct layoutdriver_io_operations
struct layoutdriver_policy_operations
pnfs.h
------
pnfs_initialize()
pnfs.c
------
pnfs_spinlock
pnfs_modules_tbl
static int pnfs_initialized
pnfs_initialize()
pnfs_register_layoutdriver
find_pnfs
pnfs_unregister_layoutdriver
inode.c
-------
pnfs_initialize call
DONE 3) set/unset pnfs layoutdriver
client.c
-------
nfs4_init_pnfs
nfs4.h
-----
enum pnfs_layouttype
FATTR4_WORD1_FS_LAYOUT_TYPES
FATTR4_WORD2_LAYOUT_BLKSIZE ( not used )
pnfs.c
------
set_pnfs_layoutdriver
unmount_pnfs_layoutdriver
pnfs.h
------
LAYOUT_NFSV4_1_MODULE_PREFIX
macro helpers (PNFS_EXISTS_XX)
nfs_xdr.h
--------
nfs_fsinfo layouttype
nfs4proc.c
----------
nfs4_fsinfo_bitmap
add FATTR4_WORD1_FS_LAYOUT_TYPES
nfs4_xdr.c
----------
decode_attr_pnfstype
and add to decode_fsinfo
DONE 4) generic deviceid cache
pnfs.c
------
nfs4_alloc_init_deviceid_cache
nfs4_init_deviceid_node
nfs4_set_layout_deviceid
nfs4_unset_layout_deviceid
nfs4_find_deviceid
nfs4_add_deviceid
nfs4_remove_deviceid
nfs4_free_deviceid_cache
nfs4_put_deviceid_cache
nfs4_pnfs.h
-----------
NFS4_DEVICE_ID_HASH_BITS
NFS4_DEVICE_ID_HASH_SIZE
NFS4_DEVICE_ID_HASH_MASK
nfs4_deviceid_hash
struct nfs4_deviceid_cache
struct nfs4_deviceid
* all the cache function declarations *
DONE 5) filelayout init/uninit_mountpoint
nfs4filelayout.c nfs4filelayout.h introduced
nfs4filelayoutdev.c introduced
module stuff
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Dean Hildebrand <dhildebz@xxxxxxxxxxxxxx>");
MODULE_DESCRIPTION("The NFSv4 file layout driver");
module_init(nfs4filelayout_init);
module_exit(nfs4filelayout_exit);
filelayout_io_operations
filelayout_initialize_mountpoint,
filelayout_uninitialize_mountpoint,
filelayout_type
nfs4filelayout_init
nfs4filelayout_exit
filelayout_initialize_mountpoint
filelayout_uninitialize_mountpoint
DONE 6) filelayout data server cache
nfs4_ds_cache_lock
nfs4_data_server_cache
print_ds
print_ds_list
_data_server_lookup
destroy_ds
nfs4_pnfs_ds_add
DONE 7) filelayout deviceid cache
deviceid_fmt
nfs4_fl_free_deviceid
nfs4_fl_free_deviceid_callback
nfs4_pnfs_device_item_find
DONE 8) generic getdeviceinfo
pnfs.c
------
pnfs_client_operations pnfs_ops
nfs4_pnfs_getdeviceinfo
nfs4proc.c
---------
nfs4_pnfs_getdeviceinfo
pnfs.h
-------
nfs4_pnfs_getdeviceinfo
nfs4xdr.c
---------
encode_getdeviceinfo_maxsz
decode_getdeviceinfo_maxsz
NFS4_enc_getdeviceinfo_sz
NFS4_dec_getdeviceinfo_sz
encode_getdeviceinfo
nfs4_xdr_enc_getdeviceinfo
decode_getdeviceinfo
nfs4_xdr_dec_getdeviceinfo
pnfs_xdr.h
----------
nfs4_pnfs_getdeviceinfo_arg
nfs4_pnfs_getdeviceinfo_res
DONE 9) filelayout getdeviceinfo
get_device_info
decode_and_add_device
decode_device
decode_and_add_ds
10) DONE make stateid a union
(alexandros added a union to the stateid)
nfs4.h
nfs4proc.c
nfs4xdr.c
delegation.c
callback_proc.c
callback_xdr.c
DONE 11) generic layout stateid
pnfs_set_layout_stateid
pnfs_get_layout_stateid
pnfs_layout_from_open_stateid
DONE 12) generic layout cache header
pnfs.c
-----
BUG_ON_UNLOCKED_INO/LO
alloc_init_layout
pnfs_alloc_layout
get_layout (change to get_layout_locked)
put_layout_locked
put_layout
pnfs_free_layout
[remove lseg lookup]
pnfs_destroy_layout
pnfs_destroy_all_layouts
pnfs.h
------
lo_fail_bit
pnfs_destroy_all_layouts
pnfs_destroy_layout
nfs4_pnfs.h
-----------
PNFS_NFS_INODE
nfs4state.c
-----------
pnfs_destroy_all_layouts call
DONE 13) filelayout layout cache header
struct nfs4_filelayout
filelayout_alloc_layout
filelayout_free_layout
FILE_LO
add to filelayout_io_operations
DONE 14) generic layoutsegment cache
pnfs.c
------
has_matching_lseg
pnfs_has_layout
_pnfs_update_layout
pnfs_free_layout
[add back lseg lookup]
init_lseg
destroy_lseg
put_lseg_locked
put_lseg
pnfs.h
------
2 get_lseg
put_lseg
nfs4_pnfs.h
-----------
struct pnfs_layout_segment
DONE 15) generic layoutget
pnfs.h
-------
pnfs4_proc_layoutget
nfs4.h
------
NFSPROC4_CLNT_PNFS_LAYOUTGET
nfs4proc.c
----------
pnfs4_proc_layoutget
_pnfs4_proc_layoutget
note: [call to pnfs_layout_process will be a comment]
nfs4_pnfs_layoutget_call_ops
nfs4_pnfs_layoutget_prepare
nfs4_pnfs_layoutget_done
nfs4_pnfs_layoutget_release
nfs4xdr.c
---------
encode_layoutget_sz (encode_layoutget_maxsz)
decode_layoutget_maxsz
NFS4_enc_layoutget_sz
NFS4_dec_layoutget_sz
encode_layoutget
decode_layoutget
nfs4_xdr_enc_layoutget
nfs4_xdr_dec_layoutget
DONE 16) layoutget helpers
pnfs.c
------
send_layout (send_layoutget)
[ uncomment _pnfs_update_layout call to send_layout]
pnfs_layout_release
pnfs_get_layout_done
cmp_layout (cmp_lseg)
pnfs_insert_layout (pnfs_insert_lseg)
pnfs_layout_process (pnfs_process_lseg)
DONE 17) filelayout layout segment cache
struct nfs4_filelayout_segment
nfs4filelayout.c
----------------
filelayout_alloc_lseg
filelayout_free_lseg
_filelayout_free_lseg
filelayout_free_fh_array
filelayout_set_layout
filelayout_check_layout
nfs4_pnfs.h
-----------
LSEG_LD_DATA
DONE 18) generic layoutcommit helpers
pnfs.c
------
initialize/uninitialize (and all layoutcommit kmem cache)
pnfs_layoutcommit_alloc
pnfs_layoutcommit_free
pnfs_layoutcommit_mempool in pnfs_initialize
pnfs_need_layoutcommit
pnfs_layoutcommit_setup
pnfs_update_last_write
nfs4_pnfs.h
-----------
has_layout
layoutcommit_needed
DONE 19) generic layoutcommit
pnfs_layoutcommit_inode
pnfs.h
------
pnfs_layoutcommit_inode
nfs4.h
------
NFSPROC4_CLNT_PNFS_LAYOUTCOMMIT
nfs4proc.c
---------
pnfs4_proc_layoutcommit
_pnfs4_proc_layoutcommit
pnfs_layoutcommit_ops
pnfs_layoutcommit_release
pnfs_layoutcommit_done
pnfs_layoutcommit_prepare
nfs4xdr.c
--------
encode_layoutcommit_sz (encode_layoutcommit_maxsz)
decode_layoutcommit_maxsz
NFS4_enc_layoutcommit_sz
NFS4_dec_layoutcommit_sz
encode_layoutcommit
decode_layoutcommit
nfs4_xdr_enc_layoutcommit
nfs4_xdr_dec_layoutcommit
write.c
-------
nfs_write_inode call to pnfs_layoutcommit_inode
DONE 21) generic layoutreturn helpers
pnfs.c
------
has_layout_to_return
_pnfs_can_return_lseg
should_free_lseg
pnfs_return_layout_barrier
_pnfs_return_layout
DONE 22) generic layoutreturn
pnfs.c
------
return_layout
pnfs.h
-----
2 pnfs_return_layout
pnfs_layout_roc_iomode
nfs4.h
------
NFSPROC4_CLNT_PNFS_LAYOUTRETURN
nfs4proc.c
---------
pnfs4_proc_layoutreturn
_pnfs4_proc_layoutreturn
nfs4_pnfs_layoutreturn_call_ops
nfs4_pnfs_layoutreturn_release
nfs4_pnfs_layoutreturn_done
nfs4_pnfs_layoutreturn_prepare
nfs4xdr.c
--------
encode_layoutreturn_sz (encode_layoutreturn_maxsz)
decode_layoutreturn_maxsz
NFS4_enc_layoutreturn_sz
NFS4_dec_layoutreturn_sz
encode_layoutreturn
decode_layoutreturn
nfs4_xdr_enc_layoutreturn
nfs4_xdr_dec_layoutreturn
inode.c
-------
nfs4_clear_inode pnfs_return_layout call
DONE 23) add session to nfs4_sequence
fs/nfs/nfs4_fs.h
nfs4proc.c
unlink.c
DONE 24) update nfs4_async_handle_error for data server
is_ds_only_session
is_ds_only_client and calls in:
nfs4_async_handle_error
DONE 25) update state renewal for data servers
nfs4_renew_state
nfs41_setup_state_renewal
DONE 26) pageio helpers
pnfs.c
------
pnfs_set_pg_test
pnfs_getboundary
pnfs_pageio_init_read
pnfs_pageio_init_write
nfs4_pnfs.h
-----------
layoutdriver_policy_operations
add policies
nfs_page.h
----------
nfs_pageio_descriptor
27) associate layout segment with nfs_page
pagelist.c
----------
nfs_create_request changes
nfs_scan_list changes
nfs_clear_request changes
nfs_can_coalesce_requests changes
nfs_can_coalesce_requests changes
_pnfs_clear_lseg_from_pages
write.c
-------
+nfs_mark_request_nopnfs
call in nfs_redirty_request
+nfs_scan_commit
nfs_scan_list
nfs_try_to_update_request
nfs_setup_write_request
nfs_writepage_setup
nfs_flush_incompatible
nfs_updatepage
28) filelayout policy ops
filelayout_policy_operations
- add to filelayout_type
filelayout_pg_test
filelayout_get_stripesize
29) filelayout i/o helpers
nfs4_pnfs_ds_create
nfs4_fl_prepare_ds
_nfs4_fl_calc_j_index
nfs4_fl_calc_ds_index
nfs4_fl_select_ds_fh
filelayout_get_dserver_offset
30) generic read
pnfs.c
------
pnfs_try_to_read_data
pnfs.h
-------
pnfs_try_to_read_data
read.c
------
nfs_initiate_read
pnfs_initiate_read
nfs_read_rpcsetup call to pnfs_initiate_read
nfs4_read_done is_ds changes
31) filelayout read
filelayout_read_call_done
filelayout_read_release
filelayout_read_call_ops
filelayout_read_pagelist
32) generic write
pnfs.c
------
pnfs_try_to_write_data
pnfs.h
------
pnfs_try_to_write_data
write.c
-------
nfs_initiate_write
pnfs_initiate_write
nfs_write_rpcsetup call to pnfs_initiate_write
nfs_mark_request_nopnfs
- nfs_redirty_request
nfs4_proc_write_setup
nfs4_write_done
pnfs4_update_write_done
33) data server write
nfs4.h
------
add NFSPROC4_CLNT_PNFS_WRITE
nfs4proc.c
---------
nfs4_proc_write_setup sets NFSPROC4_CLNT_PNFS_WRITE
nfs4xdr.c
--------
add PNFS_WRITE to rpc_procinfo
NFS4_enc_dswrite_sz
NFS4_dec_dswrite_sz
nfs4_xdr_enc_dswrite
nfs4_xdr_dec_dswrite
34) file layout write
filelayout_write_call_done
filelayout_write_release
filelayout_write_pagelist
filelayout_write_call_ops
35) generic commit
pnfs.c
------
pnfs_try_to_commit
pnfs.h
------
pnfs_try_to_commit
write.c
-------
nfs_initiate_commit
pnfs_initiate_commit
nfs_commit_rpcsetup call to pnfs_initiate_commit
nfs_commit_release add nfs_mark_request_nopnfs calls
nfs4_commit_done ds changes
nfs4_proc_commit_setup ds changes
36) data server commit
nfs4.h
------
add NFSPROC4_CLNT_PNFS_COMMIT
nfs4proc.c
---------
nfs4_proc_commit_setup sets NFSPROC4_CLNT_PNFS_COMMIT
nfs4xdr.c
--------
add PNFS_COMMIT to rpc_procinfo
NFS4_enc_dscommit_sz
NFS4_dec_dscommit_sz
nfs4_xdr_enc_dscommit
nfs4_xdr_dec_dscommit
37) file layout commit
filelayout_clone_write_data
filelayout_commit_call_done
filelayout_commit_call_ops
filelayout_commit
38) layoutrecall
callback_proc.c
---------------
pnfs_is_next_layout_stateid
nfs_layoutrecall_find_inode
recall_layout_threadargs
pnfs_recall_layout
pnfs_async_return_layout
pnfs_recall_all_layouts
pnfs_cb_layoutrecall
callback_xdr.c
--------------
OP_CB_LAYOUTRECALL declaration
decode_pnfs_layoutrecall_args
callback.h
----------
struct cb_pnfs_layoutrecallargs
39) data server recovery
nfs4proc.c
----------
nfs4_recover_expired_lease changes and EXPORT
nfs4_async_handle_error change
nfs4_setup_sequence changes
40) unused stuff to get rid of or leave for post-submit tree
pnfs_get_write_status
pnfs_get_read_status
PNFS_LD_POLICY_OPS
struct pnfs_devicelist
struct pnfs_client_operations nfs_return_layout operation.
FATTR4_WORD2_LAYOUT_BLKSIZE in nfs4.h
[ in fs/nfs/callback.h left over from removing CB_NOTIFY_DEVICEID ]
struct cb_pnfs_devicenotifyitem
#define NFS4_DEV_NOTIFY_MAXENTRIES 10
struct cb_pnfs_devicenotifyargs {
extern unsigned pnfs_cb_devicenotify(struct cb_pnfs_devicenotifyargs *args,
- void *dummy);
include/linux/nfs4_pnfs.c
struct pnfs_devicelist
include/linux/nfs4.h
enum pnfs_layouttype {
LAYOUT_OSD2_OBJECTS = 2,
LAYOUT_BLOCK_VOLUME = 3,
};
fs/nfs/inode.c: nfs_update_inode; this does nothing.
+ * file needs layout commit, server attributes may be stale
+ */
+ if (layoutcommit_needed(nfsi) && nfsi->change_attr >= fattr->change_attr) {
+ dprintk("NFS: %s: layoutcommit is needed for file %s/%ld\n",
+ __func__, inode->i_sb->s_id, inode->i_ino);
+ return 0;
+ }
+ /*
pnfs.h:
+unsigned int pnfs_getiosize(struct nfs_server *server);
write.c:
--------
@@ -1489,6 +1612,7 @@ int nfs_write_inode(struct inode *inode, struct
writeback_control *wbc)
*/
int nfs_wb_all(struct inode *inode)
{
+ int ret;
struct writeback_control wbc = {
.sync_mode = WB_SYNC_ALL,
.nr_to_write = LONG_MAX,
@@ -1496,7 +1620,8 @@ int nfs_wb_all(struct inode *inode)
.range_end = LLONG_MAX,
};
- return sync_inode(inode, &wbc);
+ ret = sync_inode(inode, &wbc);
+ return ret;
}
>
> Benny
>
>>
>> steved.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at http://vger.kernel.org/majordomo-info.html
