Hi everyone, This is the latest patchset for the Ceph distributed file system client. I've broken it down into individual files for ease of review. Alternatively, the ceph-client.git tree includes a full history (from mid-October). A shortlog for changes since the last time I sent this out (during the 2.6.33 merge window) at the end of this email. We've been hammering on this steadily for the past few weeks and are pretty happy with the stability at this point. We would definitely benefit from the broader testing that would come from inclusion in 2.6.34. If there are no major complaints, I will send this to Linus again shortly. Questions, comments, review, etc. are all welcome. :) Thanks- sage Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus Server-side git: git://ceph.newdream.net/ceph.git --- Documentation/filesystems/ceph.txt | 139 ++ Documentation/ioctl/ioctl-number.txt | 1 + MAINTAINERS | 9 + fs/Kconfig | 1 + fs/Makefile | 1 + fs/ceph/Kconfig | 27 + fs/ceph/Makefile | 39 + fs/ceph/README | 20 + fs/ceph/addr.c | 1188 +++++++++++++ fs/ceph/armor.c | 99 ++ fs/ceph/auth.c | 257 +++ fs/ceph/auth.h | 84 + fs/ceph/auth_none.c | 121 ++ fs/ceph/auth_none.h | 28 + fs/ceph/auth_x.c | 656 ++++++++ fs/ceph/auth_x.h | 49 + fs/ceph/auth_x_protocol.h | 90 + fs/ceph/buffer.c | 78 + fs/ceph/buffer.h | 39 + fs/ceph/caps.c | 2925 ++++++++++++++++++++++++++++++++ fs/ceph/ceph_debug.h | 37 + fs/ceph/ceph_frag.c | 21 + fs/ceph/ceph_frag.h | 109 ++ fs/ceph/ceph_fs.c | 74 + fs/ceph/ceph_fs.h | 651 ++++++++ fs/ceph/ceph_hash.c | 118 ++ fs/ceph/ceph_hash.h | 13 + fs/ceph/ceph_strings.c | 176 ++ fs/ceph/crush/crush.c | 151 ++ fs/ceph/crush/crush.h | 180 ++ fs/ceph/crush/hash.c | 149 ++ fs/ceph/crush/hash.h | 17 + fs/ceph/crush/mapper.c | 596 +++++++ fs/ceph/crush/mapper.h | 20 + fs/ceph/crypto.c | 408 +++++ fs/ceph/crypto.h | 48 + fs/ceph/debugfs.c | 483 ++++++ fs/ceph/decode.h | 194 +++ fs/ceph/dir.c | 1220 ++++++++++++++ fs/ceph/export.c | 223 +++ fs/ceph/file.c | 937 +++++++++++ fs/ceph/inode.c | 1750 ++++++++++++++++++++ fs/ceph/ioctl.c | 160 ++ fs/ceph/ioctl.h | 40 + fs/ceph/mds_client.c | 3021 ++++++++++++++++++++++++++++++++++ fs/ceph/mds_client.h | 335 ++++ fs/ceph/mdsmap.c | 174 ++ fs/ceph/mdsmap.h | 54 + fs/ceph/messenger.c | 2240 +++++++++++++++++++++++++ fs/ceph/messenger.h | 254 +++ fs/ceph/mon_client.c | 834 ++++++++++ fs/ceph/mon_client.h | 119 ++ fs/ceph/msgpool.c | 186 +++ fs/ceph/msgpool.h | 27 + fs/ceph/msgr.h | 158 ++ fs/ceph/osd_client.c | 1537 +++++++++++++++++ fs/ceph/osd_client.h | 166 ++ fs/ceph/osdmap.c | 1019 ++++++++++++ fs/ceph/osdmap.h | 125 ++ fs/ceph/pagelist.c | 54 + fs/ceph/pagelist.h | 54 + fs/ceph/rados.h | 374 +++++ fs/ceph/snap.c | 904 ++++++++++ fs/ceph/super.c | 1030 ++++++++++++ fs/ceph/super.h | 900 ++++++++++ fs/ceph/types.h | 29 + fs/ceph/xattr.c | 844 ++++++++++ 67 files changed, 28064 insertions(+), 0 deletions(-) Below is a shortlog of changes since v0.18, the last patchset sent out to LKML/fsdevel for review. The main items are support for client and server authentication (auth_x), some support for easing future protocol changes, and a variety of bug fixes. The full history is available at http://ceph.newdream.net/git/?p=ceph-client.git;a=shortlog;h=unstable fs/ceph/Kconfig | 1 + fs/ceph/Makefile | 4 +- fs/ceph/addr.c | 153 ++++++++--- fs/ceph/armor.c | 99 +++++++ fs/ceph/auth.c | 70 ++++-- fs/ceph/auth.h | 7 + fs/ceph/auth_none.c | 1 + fs/ceph/auth_x.c | 656 +++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/auth_x.h | 49 ++++ fs/ceph/auth_x_protocol.h | 90 ++++++ fs/ceph/buffer.c | 54 ++++- fs/ceph/buffer.h | 30 +-- fs/ceph/caps.c | 310 +++++++++++++--------- fs/ceph/ceph_fs.h | 23 +- fs/ceph/crypto.c | 408 ++++++++++++++++++++++++++++ fs/ceph/crypto.h | 48 ++++ fs/ceph/debugfs.c | 93 +++++-- fs/ceph/decode.h | 35 +++ fs/ceph/dir.c | 14 +- fs/ceph/file.c | 95 +++++--- fs/ceph/inode.c | 188 +++++++++++--- fs/ceph/mds_client.c | 607 ++++++++++++++++++++++------------------- fs/ceph/mds_client.h | 20 +- fs/ceph/mdsmap.c | 4 + fs/ceph/messenger.c | 507 ++++++++++++++++++++++------------- fs/ceph/messenger.h | 37 ++-- fs/ceph/mon_client.c | 173 +++++++++--- fs/ceph/mon_client.h | 8 +- fs/ceph/msgpool.c | 9 +- fs/ceph/msgr.h | 23 +-- fs/ceph/osd_client.c | 525 ++++++++++++++++++++++++------------ fs/ceph/osd_client.h | 30 ++- fs/ceph/osdmap.c | 223 +++++++++++---- fs/ceph/osdmap.h | 7 +- fs/ceph/pagelist.c | 54 ++++ fs/ceph/pagelist.h | 54 ++++ fs/ceph/rados.h | 16 +- fs/ceph/snap.c | 69 +++-- fs/ceph/super.c | 76 +++++- fs/ceph/super.h | 47 ++-- fs/ceph/xattr.c | 8 +- 41 files changed, 3731 insertions(+), 1194 deletions(-) Alexander Beregalov (1): ceph: move dereference after NULL test Julia Lawall (1): ceph: remove duplicate variable initialization Sage Weil (80): ceph: use kref for ceph_buffer ceph: simplify ceph_buffer interface ceph: use kref for struct ceph_mds_request ceph: use kref for ceph_osd_request ceph: use kref for ceph_msg ceph: do not feed bad device ids to crush ceph: fix leak of monc mutex ceph: carry explicit msg reference for currently sending message ceph: plug msg leak in con_fault ceph: detect lossy state of connection ceph: don't save sent messages on lossy connections ceph: hex dump corrupt server data to KERN_DEBUG ceph: plug leak of incoming message during connection fault/close ceph: make mds ops interruptible ceph: include link to bdi in debugfs ceph: ensure rename target dentry fails revalidation ceph: do not drop lease during revalidate ceph: fix error paths for corrupt osdmap messages ceph: fix incremental osdmap pg_temp decoding bug ceph: do not touch_caps while iterating over caps list ceph: only unregister registered bdi ceph: unregister canceled/timed out osd requests ceph: use connection mutex to protect read and write stages ceph: control access to page vector for incoming data ceph: more informative msgpool errors ceph: include transaction id in ceph_msg_header (protocol change) ceph: add feature bits to connection handshake (protocol change) ceph: support ceph_pagelist for message payload ceph: use ceph_pagelist for mds reconnect message; change encoding (protocol change) ceph: remove unused erank field ceph: display pgid in debugfs osd request dump ceph: mark MDS CREATE as a write op ceph: properly handle aborted mds requests ceph: precede encoded ceph_pg_pool struct with version ceph: include type in ceph_entity_addr, filepath ceph: release all pages after successful osd write response ceph: buffer decoding helpers ceph: aes crypto and base64 encode/decode helpers ceph: allow renewal of auth credentials ceph: add struct version to auth encoding ceph: add support for auth_x authentication protocol ceph: add uid field to ceph_pg_pool ceph: cap revocation fixes ceph: do not retain caps that are being revoked ceph: fix sync read eof check deadlock ceph: cleanup async writeback, truncation, invalidate helpers ceph: invalidate pages even if truncate is pending ceph: remove bogus invalidate_mapping_pages ceph: fix msgr to keep sent messages until acked ceph: reset osd connections after fault ceph: allow connection to be reopened by fault callback ceph: cancel delayed work when closing connection ceph: use rbtree for mds requests ceph: use rbtree for snap_realms ceph: use rbtree for mon statfs requests ceph: fix authentication races, auth_none oops ceph: clean up readdir caps reservation ceph: fix iterate_caps removal race ceph: fix memory leak when destroying osdmap with pg_temp mappings ceph: use rbtree for pg pools; decode new osdmap format ceph: v0.19 release ceph: fix typo in ceph_queue_writeback debug output ceph: fix check for invalidate_mapping_pages success ceph: fix up unexpected message handling ceph: fix comments, locking in destroy_inode ceph: drop messages on unregistered mds sessions; cleanup ceph: fix client_request_forward decoding ceph: invalidate_authorizer without con->mutex held ceph: fix connection fault STANDBY check ceph: remove fragile __map_osds optimization ceph: remove bogus mds forward warning ceph: reset bits on connection close ceph: use single osd op reply msg ceph: fix snaptrace decoding on cap migration between mds ceph: reset front len on return to msgpool; BUG on mismatched front iov ceph: set osd request message front length correctly ceph: return EBADF if waiting for caps on closed file ceph: fix osdmap decoding when pools include (removed) snaps ceph: include migrating caps in issued set ceph: fix flush_dirty_caps race with caps migration Yehuda Sadeh (23): ceph: fix msgpool reservation leak ceph: remove unaccessible code ceph: writepage grabs and releases inode ceph: writeback congestion control ceph: fix copy_user_to_page_vector() ceph: change dentry offset and position after splice_dentry ceph: allocate middle of message before stating to read ceph: refactor messages data section allocation ceph: alloc message data pages and check if tid exists ceph: keep reserved replies on the request structure ceph: remove unreachable code ceph: always send truncation info with read and write osd ops ceph: remove unused variable ceph: put unused osd connections on lru ceph: fix short synchronous reads ceph: refactor ceph_write_begin, fix ceph_page_mkwrite ceph: fix truncation when not holding caps ceph: sync read/write considers page cache ceph: remove page upon writeback completion if lost cache cap ceph: don't truncate dirty pages in invalidate work thread ceph: cleanup redundant code in handle_cap_grant ceph: don't clobber write return value when using O_SYNC ceph: reset osd after relevant messages timed out -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
