Hi all, Here's the fourth round to implement the second step of extent status tree. In the first step, extent status tree can track all delayed extents, improve bigalloc, delayed allocation, and introduce lseek(2) seek data/hole. The first step has been applied. In the second step, extent status tree will try to track all extent status, and it will be as a extent cache for extent tree. Then we can first lookup a block mapping from extent status tree before traversing extent tree. The major change in this version is to rework the patch that reclaims extent from extent status tree. Now when the extent status tree of an inode is accessing, this inode will be moved into the tail of the lru list. A counter is used to count the number of written/unwritten extents in a extent status tree. Meanwhile a normal shrinker is registered to reclaim memeory when we are under a high memory pressure. ext4_es_reclaim_extent_count() traverses the lru list to count all cached objects. ext4_es_shrink() tries to reclaim written/unwritten extent from the tree. Here we never reclaim delayed extent in the tree because bigalloc, seek data/hole, and fiemap need it. I do the following test to verify that the shrinker works well. 50 files that the length of file is 128m are created with 2048 extents, and mlock(2) is used to occupy the memory as much as possible. Then grep(1) is used to touch all extents of every files. I can see from the tracepoint in ext4_es_shrink() that extents are reclaimed. changelog: v4 <- v3: - register a normal shrinker to reclaim extent from extent status tree v3 <- v2: - use prune_super() to reclaim extents from extent status tree - stashed es_status into es_pblk - remove single extent cache - rebase against 3.8-rc4 v2 <- v1: - drop patches that try to improve unwritten extent conversion - remove EXT4_MAP_FROM_CLUSTER flag - add tracepoint for ext4_es_lookup_extent() - drop a patch, which tries to fix a warning when bigalloc and delalloc are enabled - add a shrinker to reclaim memory from extent status tree - rebase against 3.8-rc2 v3: http://lwn.net/Articles/533730/ v2: http://lwn.net/Articles/532446/ v1: http://lwn.net/Articles/531065/ Any comments or feedbacks are appreciated. Regards, - Zheng Zheng Liu (9): ext4: refine extent status tree ext4: remove EXT4_MAP_FROM_CLUSTER flag ext4: add physical block and status member into extent status tree ext4: adjust interfaces of extent status tree ext4: track all extent status in extent status tree ext4: lookup block mapping in extent status tree ext4: remove single extent cache ext4: adjust some functions for reclaiming extents from extent status tree ext4: reclaim extents from extent status tree fs/ext4/ext4.h | 34 +-- fs/ext4/ext4_extents.h | 6 - fs/ext4/extents.c | 256 ++++--------------- fs/ext4/extents_status.c | 588 ++++++++++++++++++++++++++++++++------------ fs/ext4/extents_status.h | 50 +++- fs/ext4/file.c | 14 +- fs/ext4/inode.c | 132 +++++++--- fs/ext4/move_extent.c | 3 - fs/ext4/super.c | 8 +- include/trace/events/ext4.h | 178 ++++++++++++-- 10 files changed, 802 insertions(+), 467 deletions(-) -- 1.7.12.rc2.18.g61b472e -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
