(Added cover letter to avoid much text in patch description) LZ4 specification defines 2 byte offset length for 64 KB data. But in case of ZRAM we compress data per page and in most of architecture PAGE_SIZE is 4KB. So we can decide offset length based on actual offset value. For this we can reserve 1 bit to decide offset length (1 byte or 2 byte). 2 byte required only if ofsset is greater than 127, else 1 byte is enough. With this new implementation new offset value can be at MAX 32 KB. Thus we can save more memory for compressed data. results checked with new implementation:- comression size for same input source (LZ4_DYN < LZO < LZ4) LZO ======= orig_data_size: 78917632 compr_data_size: 15894668 mem_used_total: 17117184 LZ4 ======== orig_data_size: 78917632 compr_data_size: 16310717 mem_used_total: 17592320 LZ4_DYN ======= orig_data_size: 78917632 compr_data_size: 15520506 mem_used_total: 16748544 checked performance with below tool:- https://github.com/sergey-senozhatsky/zram-perf-test # ./fio-perf-o-meter.sh /tmp/test-fio-zram-lz4 /tmp/test-fio-zram-lz4_dyn Processing /tmp/test-fio-zram-lz4 Processing /tmp/test-fio-zram-lz4_dyn #jobs1 WRITE: 1101.7MB/s 1197.7MB/s WRITE: 799829KB/s 900838KB/s READ: 2670.2MB/s 2649.5MB/s READ: 2027.8MB/s 2039.9MB/s READ: 603703KB/s 597855KB/s WRITE: 602943KB/s 597103KB/s READ: 680438KB/s 707986KB/s WRITE: 679582KB/s 707095KB/s #jobs2 WRITE: 1993.2MB/s 2121.2MB/s WRITE: 1654.1MB/s 1700.2MB/s READ: 5038.2MB/s 4970.9MB/s READ: 3930.1MB/s 3908.5MB/s READ: 1113.2MB/s 1117.4MB/s WRITE: 1111.8MB/s 1115.2MB/s READ: 1255.8MB/s 1286.5MB/s WRITE: 1254.2MB/s 1284.9MB/s #jobs3 WRITE: 2875.6MB/s 3010.3MB/s WRITE: 2394.4MB/s 2363.2MB/s READ: 7384.7MB/s 7314.3MB/s READ: 5389.5MB/s 5427.6MB/s READ: 1570.8MB/s 1557.3MB/s WRITE: 1568.8MB/s 1555.3MB/s READ: 1848.5MB/s 1854.0MB/s WRITE: 1846.2MB/s 1851.7MB/s #jobs4 WRITE: 3720.3MB/s 3077.4MB/s WRITE: 3027.4MB/s 3072.8MB/s READ: 9694.7MB/s 9822.6MB/s READ: 6606.5MB/s 6617.2MB/s READ: 1941.6MB/s 1966.8MB/s WRITE: 1939.1MB/s 1964.3MB/s READ: 2405.3MB/s 2347.5MB/s WRITE: 2402.3MB/s 2344.5MB/s #jobs5 WRITE: 3335.6MB/s 3360.7MB/s WRITE: 2670.2MB/s 2677.9MB/s READ: 9455.3MB/s 8782.2MB/s READ: 6534.8MB/s 6501.7MB/s READ: 1848.9MB/s 1858.3MB/s WRITE: 1846.6MB/s 1855.1MB/s READ: 2232.4MB/s 2223.7MB/s WRITE: 2229.6MB/s 2220.9MB/s #jobs6 WRITE: 3896.5MB/s 3772.9MB/s WRITE: 3171.1MB/s 3109.4MB/s READ: 11060MB/s 11120MB/s READ: 7375.8MB/s 7384.7MB/s READ: 2132.5MB/s 2133.1MB/s WRITE: 2129.8MB/s 2131.3MB/s READ: 2608.4MB/s 2627.3MB/s WRITE: 2605.7MB/s 2623.2MB/s #jobs7 WRITE: 4129.4MB/s 4083.2MB/s WRITE: 3364.5MB/s 3384.4MB/s READ: 12088MB/s 11062MB/s READ: 7868.3MB/s 7851.5MB/s READ: 2277.8MB/s 2291.6MB/s WRITE: 2274.9MB/s 2288.7MB/s READ: 2798.5MB/s 2890.1MB/s WRITE: 2794.1MB/s 2887.4MB/s #jobs8 WRITE: 4623.3MB/s 4794.9MB/s WRITE: 3749.3MB/s 3676.9MB/s READ: 12337MB/s 14076MB/s READ: 8320.1MB/s 8229.4MB/s READ: 2496.9MB/s 2486.3MB/s WRITE: 2493.8MB/s 2483.2MB/s READ: 3340.4MB/s 3370.6MB/s WRITE: 3336.2MB/s 3366.4MB/s #jobs9 WRITE: 4427.6MB/s 4341.3MB/s WRITE: 3542.6MB/s 3597.2MB/s READ: 10094MB/s 9888.5MB/s READ: 7863.5MB/s 8119.9MB/s READ: 2357.1MB/s 2382.1MB/s WRITE: 2354.1MB/s 2379.1MB/s READ: 2828.8MB/s 2826.2MB/s WRITE: 2825.3MB/s 2822.7MB/s #jobs10 WRITE: 4463.9MB/s 4327.7MB/s WRITE: 3637.7MB/s 3592.4MB/s READ: 10020MB/s 11118MB/s READ: 7837.8MB/s 8098.7MB/s READ: 2459.6MB/s 2406.5MB/s WRITE: 2456.5MB/s 2403.4MB/s READ: 2804.2MB/s 2829.8MB/s WRITE: 2800.7MB/s 2826.2MB/s jobs1 perfstat stalled-cycles-frontend 20,23,52,25,317 ( 54.32%) 19,29,10,49,608 ( 54.50%) instructions 44,62,30,88,401 ( 1.20) 42,50,67,71,907 ( 1.20) branches 7,12,44,77,233 ( 738.975) 6,64,52,15,491 ( 725.584) branch-misses 2,38,66,520 ( 0.33%) 2,04,33,819 ( 0.31%) jobs2 perfstat stalled-cycles-frontend 42,82,90,69,149 ( 56.63%) 41,58,70,01,387 ( 56.01%) instructions 85,33,18,31,411 ( 1.13) 85,32,92,28,973 ( 1.15) branches 13,35,34,99,713 ( 677.499) 13,34,97,00,453 ( 693.104) branch-misses 4,50,17,075 ( 0.34%) 4,47,28,378 ( 0.34%) jobs3 perfstat stalled-cycles-frontend 66,01,57,23,062 ( 57.10%) 65,86,74,97,814 ( 57.30%) instructions 1,28,18,27,80,041 ( 1.11) 1,28,04,92,91,306 ( 1.11) branches 20,06,14,16,000 ( 651.453) 20,02,85,32,864 ( 652.536) branch-misses 7,10,66,773 ( 0.35%) 7,12,75,728 ( 0.36%) jobs4 perfstat stalled-cycles-frontend 91,98,71,83,315 ( 58.09%) 93,70,91,50,920 ( 58.66%) instructions 1,70,82,79,66,403 ( 1.08) 1,71,18,67,74,366 ( 1.07) branches 26,73,53,03,398 ( 621.532) 26,80,89,38,054 ( 618.718) branch-misses 9,82,07,177 ( 0.37%) 9,81,64,098 ( 0.37%) jobs5 perfstat stalled-cycles-frontend 1,47,29,71,29,605 ( 63.59%) 1,47,91,01,92,835 ( 63.86%) instructions 2,18,90,41,63,988 ( 0.95) 2,18,55,73,09,594 ( 0.94) branches 34,64,46,32,880 ( 553.209) 34,55,08,02,781 ( 551.953) branch-misses 14,16,79,279 ( 0.41%) 13,84,85,054 ( 0.40%) jobs6 perfstat stalled-cycles-frontend 2,02,92,92,98,242 ( 66.70%) 2,05,33,49,39,627 ( 67.01%) instructions 2,65,13,90,22,217 ( 0.87) 2,64,84,45,49,149 ( 0.86) branches 42,11,54,07,400 ( 510.085) 42,03,58,57,789 ( 505.746) branch-misses 17,71,33,628 ( 0.42%) 17,74,31,942 ( 0.42%) jobs7 perfstat stalled-cycles-frontend 2,79,22,74,37,283 ( 70.23%) 2,80,02,50,89,154 ( 70.48%) instructions 3,11,90,38,02,741 ( 0.78) 3,09,20,69,87,835 ( 0.78) branches 49,71,39,90,321 ( 460.940) 49,10,44,23,983 ( 455.686) branch-misses 22,43,84,102 ( 0.45%) 21,96,67,440 ( 0.45%) jobs8 perfstat stalled-cycles-frontend 3,59,62,09,66,766 ( 73.38%) 3,58,04,85,16,351 ( 73.37%) instructions 3,43,83,05,02,841 ( 0.70) 3,43,33,76,84,985 ( 0.70) branches 54,02,15,25,784 ( 406.256) 53,91,13,38,774 ( 407.265) branch-misses 25,20,35,507 ( 0.47%) 25,05,71,030 ( 0.46%) jobs9 perfstat stalled-cycles-frontend 4,15,33,64,48,628 ( 73.76%) 4,22,88,52,47,923 ( 74.16%) instructions 3,90,79,09,16,552 ( 0.69) 3,91,12,92,41,516 ( 0.69) branches 61,66,87,76,271 ( 403.896) 61,73,58,17,174 ( 399.363) branch-misses 28,46,21,136 ( 0.46%) 28,45,74,774 ( 0.46%) jobs10 perfstat stalled-cycles-frontend 4,74,43,71,32,846 ( 74.30%) 4,66,34,70,59,452 ( 73.82%) instructions 4,35,23,51,39,076 ( 0.68) 4,38,48,78,54,987 ( 0.69) branches 68,72,17,08,212 ( 396.945) 69,48,52,50,280 ( 405.847) branch-misses 31,73,62,053 ( 0.46%) 32,34,76,102 ( 0.47%) seconds elapsed 11.470858891 10.862984653 seconds elapsed 11.802220972 11.348959061 seconds elapsed 11.847204652 11.850297919 seconds elapsed 12.352068602 12.853222188 seconds elapsed 16.162715423 16.355883496 seconds elapsed 16.605502317 16.855938732 seconds elapsed 18.108333660 18.108347866 seconds elapsed 18.621296174 18.354183020 seconds elapsed 22.366502860 22.357632546 seconds elapsed 24.362417439 24.363003009 Maninder Singh, Vaneet Narang (1): lz4: Implement lz4 with dynamic offset (lz4_dyn). crypto/lz4.c | 64 ++++++++++++++++++++++++++++++++- drivers/block/zram/zcomp.c | 4 ++ fs/pstore/platform.c | 2 +- include/linux/lz4.h | 15 ++++++-- lib/decompress_unlz4.c | 2 +- lib/lz4/lz4_compress.c | 84 +++++++++++++++++++++++++++++++++++-------- lib/lz4/lz4_decompress.c | 56 ++++++++++++++++++++--------- lib/lz4/lz4defs.h | 11 ++++++ 8 files changed, 197 insertions(+), 41 deletions(-)
