Hi list, this is another try for the inline data support for ext4. The main change in this version is that now we only store the 4 bytes for parent's inode number and both "." and ".." are removed which is suggested by Ted. I have runned all the xfstests cases and fixes the bugs it found. Hope it is stable now for the inclusion. So in genernal, I have runned several test cases and try to prove that it works. A fsstats shows the directory information for my home. (The test tools can be found here. http://www.pdsi-scidac.org/fsstats/. Thanks goes to Andreas Dilger). directory size (entries): Range of entries, count of directories in range, number of dirs in range as a % of total num of dirs, number of dirs in this range or smaller as a % total number of dirs, total entries in range, number of entries in range as a % of total number of entries, number of entries in this range or smaller as a % of total number of entries. count=13671 avg=11.58 ents min=0.00 ents max=4562.00 ents [ 0- 1 ents]: 5198 (38.02%) ( 38.02% cumulative) 3322.00 ents ( 2.10%) ( 2.10% cumulative) [ 2- 3 ents]: 2246 (16.43%) ( 54.45% cumulative) 5477.00 ents ( 3.46%) ( 5.56% cumulative) [ 4- 7 ents]: 2266 (16.58%) ( 71.03% cumulative) 12037.00 ents ( 7.60%) ( 13.16% cumulative) [ 8- 15 ents]: 1877 (13.73%) ( 84.76% cumulative) 20269.00 ents (12.80%) ( 25.96% cumulative) [ 16- 31 ents]: 1126 ( 8.24%) ( 92.99% cumulative) 24196.00 ents (15.28%) ( 41.24% cumulative) [ 32- 63 ents]: 554 ( 4.05%) ( 97.04% cumulative) 23772.00 ents (15.01%) ( 56.26% cumulative) [ 64- 127 ents]: 234 ( 1.71%) ( 98.76% cumulative) 20919.00 ents (13.21%) ( 69.47% cumulative) [ 128- 255 ents]: 119 ( 0.87%) ( 99.63% cumulative) 20654.00 ents (13.05%) ( 82.52% cumulative) [ 256- 511 ents]: 40 ( 0.29%) ( 99.92% cumulative) 12941.00 ents ( 8.17%) ( 90.69% cumulative) [ 512-1023 ents]: 6 ( 0.04%) ( 99.96% cumulative) 4147.00 ents ( 2.62%) ( 93.31% cumulative) [1024-2047 ents]: 3 ( 0.02%) ( 99.99% cumulative) 3487.00 ents ( 2.20%) ( 95.51% cumulative) [2048-4095 ents]: 1 ( 0.01%) ( 99.99% cumulative) 2544.00 ents ( 1.61%) ( 97.12% cumulative) [4096-8191 ents]: 1 ( 0.01%) (100.00% cumulative) 4562.00 ents ( 2.88%) (100.00% cumulative) So with inode size equals 256, more than 50% directory can be inlined. I have also runned several other test cases to prove the peroformance gain, And I'd like to describe the result here. For more test details, please see below. For a dir iteration test, if most of the dirs are inlined, we can see a 10 fold speed up(240ms vs. 2250ms) in dir iteration(try to find a not-exist file). Yes, that is expected. ;) With a normal kernel tar and directory iteration test, we can find a 1.5% space saving(with cluster size=4096, I would imagine a better result with bigalloc enabled). As for the directory iteration, we can see a 22.4% speedup. I ran postmark for 2 senarios, one with file size between 11 and 500 which is good for inline data test, and another with file size between 100 and 50000. We can see a 32.8%(49s vs. 73s) improvement for the first one, and 3.6%(805s vs. 835s) for the latter. The final test case is the fs_mark. I ran the different sync modes, and we can see more than 100% increase if we does the sync and this is really a huge win. 1. normal test case of taring a kernel source and find a file: time tar zxvf linux-3.4.tar.gz -C $MNT_DIR df -hm|grep $MNT_DIR umount $MNT_DIR mount -t ext4 $DEVICE $MNT_DIR time find $MNT_DIR -name a inline_data result in find real 570ms in find sys 80ms in find user 30ms normal result in find real 760ms in find sys 110ms in find user 30ms inline_data + journal result in find real 270ms in find sys 80ms in find user 30ms normal journal result in find real 590ms in find sys 120ms in find user 40ms 2. postmark test case 1: set location $MNT_DIR set subdirectories 2570 set size 11 500 set transactions 51110 set number 977770 with inline data result: Time: 56 seconds total 2 seconds of transactions (25555 per second) Files: 1003234 created (17914 per second) Creation alone: 977770 files (29629 per second) Mixed with transactions: 25464 files (12732 per second) 25561 read (12780 per second) 25548 appended (12774 per second) 1003234 deleted (17914 per second) Deletion alone: 977588 files (46551 per second) Mixed with transactions: 25646 files (12823 per second) Data: 6.25 megabytes read (114.27 kilobytes per second) 247.18 megabytes written (4.41 megabytes per second) Without inline data: Time: 71 seconds total 24 seconds of transactions (2129 per second) Files: 1003234 created (14130 per second) Creation alone: 977770 files (31540 per second) Mixed with transactions: 25464 files (1061 per second) 25561 read (1065 per second) 25548 appended (1064 per second) 1003234 deleted (14130 per second) Deletion alone: 977588 files (61099 per second) Mixed with transactions: 25646 files (1068 per second) Data: 6.25 megabytes read (90.13 kilobytes per second) 247.18 megabytes written (3.48 megabytes per second) postmark test 2: set location $MNT_DIR set subdirectories 2570 set size 100 50000 set transactions 51110 set number 977770 with inline data: Time: 805 seconds total 263 seconds of transactions (194 per second) Files: 1003213 created (1246 per second) Creation alone: 977770 files (1975 per second) Mixed with transactions: 25443 files (96 per second) 25597 read (97 per second) 25513 appended (97 per second) 1003213 deleted (1246 per second) Deletion alone: 977546 files (20798 per second) Mixed with transactions: 25667 files (97 per second) without inline data: Time: 835 seconds total 289 seconds of transactions (176 per second) Files: 1003213 created (1201 per second) Creation alone: 977770 files (2011 per second) Mixed with transactions: 25443 files (88 per second) 25597 read (88 per second) 25513 appended (88 per second) 1003213 deleted (1201 per second) Deletion alone: 977546 files (16292 per second) Mixed with transactions: 25667 files (88 per second) 3. fs_mark test: fs_mark -d $MNT_DIR -D 256 -s 256 -n 10000 -t 256 -S [0|1|2] inline+nojournal noinline+nojournal Files/sec for sync 0 69931.1 69305.7 Files/sec for sync 1 40674.0 13975.2 Files/sec for sync 2 45786.8 16154.8 inline+journal noinline+journal Files/sec for sync 0 63523.4 61092.1 Files/sec for sync 1 45325.4 20685.5 Files/sec for sync 2 41140.1 15908.6 git diff --stat fs/ext4/Makefile | 2 +- fs/ext4/dir.c | 39 +- fs/ext4/ext4.h | 90 +++- fs/ext4/extents.c | 22 +- fs/ext4/ialloc.c | 4 + fs/ext4/inline.c | 1827 +++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ext4/inode.c | 237 ++++++-- fs/ext4/namei.c | 463 +++++++++----- fs/ext4/xattr.c | 87 ++- fs/ext4/xattr.h | 277 ++++++++ 10 files changed, 2783 insertions(+), 265 deletions(-) any comments are welcomed. Thanks Tao -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
