On Tue, Jun 21, 2011 at 6:45 PM, Andreas Dilger <adilger@xxxxxxxxx> wrote: > On 2011-06-21, at 5:06 AM, "Amir G." <amir73il@xxxxxxxxxxxxxxxxxxxxx> wrote: >> On Tue, Jun 7, 2011 at 6:07 PM, <amir73il@xxxxxxxxxxxxxxxxxxxxx> wrote: >>> >>> I am resending the snapshots patch series as per Lukas's request. >>> This time, the snapshot*.c files have not been omitted, as in >>> the previous posting. >>> >>> The series is still based on ext4 dev branch sometime in the preparation >>> for 3.0 merge window. It was not yet rebased on 3.0-rc1, so punch holes >>> changes have not been addressed yet. >>> >>> As always, I advocate online review of the patches at: >>> https://github.com/amir73il/ext4-snapshots/commits/for-ext4-v1 >>> but if you insist on doing it the old way, I won't complain. >>> >>> >> >> To answer your question about possible diet to snapshot patches, >> following are some diffstat on groups of patches (by functionality). >> The diffsstat includes only ext4 core file (excluding new snapshot* files). >> >> As you can see, removing the shrink/merge functionality for multiple >> snapshots, will remove the need for exclude bitmap and reduce the changes >> to core files by ~300 insertions. >> >> There is some advantage of not having to add the exclude bitmap to existing >> fs, but I think the win is not so big, assuming that we will want to have the >> shrink/merge functionality eventually. > > Wouldn't that also mean if shrink/merge are added later it needs another feature flag and added complexity due to another row in the feature test matrix? > Sure, it would add another row in the matrix and we don't want that either. Shrink/merge capability could be derived from the exclude_bitmap (compatible) feature. In fact, without exclude bitmap, shrink/merge can still be done, but the shrink will not be as efficient as with exclude_bitmap, but the shrink code works just the same (i.e. it doesn't have to check for exclude_bitmap feature). BTW, I have just learned that ZFS disk space is not reclaimed at all unless deleting a complete part of the snapshots history (including the oldest). This is essentially how ext4 snapshots would work without the shrink/merge patches and exclude_bitmap, but it is not very useful for long snapshot retention policies (i.e. 1 yearly, 3 monthly, 4 weekly,...). > >> >>> [PATCH v1 01/36] ext4: EXT4 snapshots (Experimental) >>> [PATCH v1 02/36] ext4: snapshot debugging support >> >> Generic stuff. >> >> 15 files changed, 122 insertions(+), 1 deletions(-) >> >>> [PATCH v1 03/36] ext4: snapshot hooks - inside JBD hooks >>> [PATCH v1 04/36] ext4: snapshot hooks - block bitmap access >>> [PATCH v1 05/36] ext4: snapshot hooks - delete blocks >>> [PATCH v1 06/36] ext4: snapshot hooks - move data blocks >>> [PATCH v1 07/36] ext4: snapshot hooks - direct I/O >>> [PATCH v1 08/36] ext4: snapshot hooks - move extent file data blocks >> >> Most of the code in this group handles MOW. >> >> 7 files changed, 550 insertions(+), 68 deletions(-) >> >>> [PATCH v1 09/36] ext4: snapshot file >>> [PATCH v1 10/36] ext4: snapshot file - read through to block device >>> [PATCH v1 11/36] ext4: snapshot file - permissions >>> [PATCH v1 12/36] ext4: snapshot file - store on disk >>> [PATCH v1 13/36] ext4: snapshot file - increase maximum file size limit to 16TB >> >> Implementation of special snapshot file. >> >> 7 files changed, 220 insertions(+), 14 deletions(-) >> >>> [PATCH v1 14/36] ext4: snapshot block operations >>> [PATCH v1 15/36] ext4: snapshot block operation - copy blocks to snapshot >>> [PATCH v1 16/36] ext4: snapshot block operation - move blocks to snapshot >>> [PATCH v1 17/36] ext4: snapshot block operation - copy block bitmap to snapshot >> >> Copy/move to snapshot file operations. >> >> 5 files changed, 126 insertions(+), 24 deletions(-) >> >>> [PATCH v1 18/36] ext4: snapshot control >>> [PATCH v1 19/36] ext4: snapshot control - init new snapshot >>> [PATCH v1 20/36] ext4: snapshot control - fix new snapshot >>> [PATCH v1 21/36] ext4: snapshot control - reserve disk space for snapshot >> >> Mostly new code in ioctl.c. >> >> 6 files changed, 171 insertions(+), 3 deletions(-) >> >>> [PATCH v1 22/36] ext4: snapshot journaled - increase transaction credits >>> [PATCH v1 23/36] ext4: snapshot journaled - implement journal_release_buffer() >>> [PATCH v1 24/36] ext4: snapshot journaled - bypass to save credits >>> [PATCH v1 25/36] ext4: snapshot journaled - cache last COW tid in journal_head >> >> Helper functions to handle extra COW credits. >> >> 7 files changed, 284 insertions(+), 31 deletions(-) >> >>> [PATCH v1 26/36] ext4: snapshot journaled - trace COW/buffer credits >> >> Trace/debug for extra COW credits. >> >> 3 files changed, 108 insertions(+), 0 deletions(-) >> >>> [PATCH v1 27/36] ext4: snapshot list support >>> [PATCH v1 28/36] ext4: snapshot list - read through to previous snapshot >> >> Not much to gain from dropping snapshot list support... >> >> 2 files changed, 2 insertions(+), 0 deletions(-) >> >>> [PATCH v1 29/36] ext4: snapshot race conditions - concurrent COW bitmap operations >>> [PATCH v1 30/36] ext4: snapshot race conditions - concurrent COW operations >>> [PATCH v1 31/36] ext4: snapshot race conditions - tracked reads >> >> We must handle race conditions. >> >> 2 files changed, 32 insertions(+), 0 deletions(-) >> >>> [PATCH v1 32/36] ext4: snapshot exclude - the exclude bitmap >> >> We do not need the exclude bitmap if we do not shrink/merge snapshots. >> >> 5 files changed, 226 insertions(+), 3 deletions(-) >> >>> [PATCH v1 33/36] ext4: snapshot cleanup >>> [PATCH v1 34/36] ext4: snapshot cleanup - shrink deleted snapshots >>> [PATCH v1 35/36] ext4: snapshot cleanup - merge shrunk snapshots >> >> We could support multiple snapshots without shrinking/merging deleted snapshot, >> but that means that only disk space of oldest snapshot is reclaimed on delete. >> >> 2 files changed, 74 insertions(+), 22 deletions(-) >> >>> [PATCH v1 36/36] ext4: snapshot rocompat - enable rw mount >>> >>> fs/ext4/Kconfig | 11 + >>> fs/ext4/Makefile | 3 + >>> fs/ext4/balloc.c | 132 +++ >>> fs/ext4/ext4.h | 188 ++++- >>> fs/ext4/ext4_jbd2.c | 162 ++++- >>> fs/ext4/ext4_jbd2.h | 266 ++++++- >>> fs/ext4/extents.c | 157 ++++- >>> fs/ext4/file.c | 11 +- >>> fs/ext4/ialloc.c | 19 +- >>> fs/ext4/inode.c | 668 +++++++++++++-- >>> fs/ext4/ioctl.c | 120 +++ >>> fs/ext4/mballoc.c | 161 ++++- >>> fs/ext4/move_extent.c | 3 +- >>> fs/ext4/namei.c | 9 + >>> fs/ext4/resize.c | 19 +- >>> fs/ext4/snapshot.c | 1000 ++++++++++++++++++++++ >>> fs/ext4/snapshot.h | 690 ++++++++++++++++ >>> fs/ext4/snapshot_buffer.c | 393 +++++++++ >>> fs/ext4/snapshot_ctl.c | 2002 +++++++++++++++++++++++++++++++++++++++++++++ >>> fs/ext4/snapshot_debug.c | 107 +++ >>> fs/ext4/snapshot_debug.h | 105 +++ >>> fs/ext4/snapshot_inode.c | 960 ++++++++++++++++++++++ >>> fs/ext4/super.c | 157 ++++- >>> fs/ext4/xattr.c | 4 +- >>> 24 files changed, 7182 insertions(+), 165 deletions(-) >>> >>> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
