Hi all, There are some updated performance statistics with different combinations on my test environment if you are interested. On 1/27/23 6:24 PM, Gao Xiang wrote: > ... > > I've made a version and did some test, it can be fetched from: > git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b > experimental > Setup ====== CPU: x86_64 Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz Disk: 6800 IOPS upper limit OS: Linux v6.2 (with composefs v3 patchset) I build erofs/squashfs images following the scripts attached on [1], with each file in the rootfs tagged with "metacopy" and "redirect" xattr. The source rootfs is from the docker image of tensorflow [2]. The erofs images are built with mkfs.erofs with support for sparse file added [3]. [1] https://lore.kernel.org/linux-fsdevel/5fb32a1297821040edd8c19ce796fc0540101653.camel@xxxxxxxxxx/ [2] https://hub.docker.com/layers/tensorflow/tensorflow/2.10.0/images/sha256-7f9f23ce2473eb52d17fe1b465c79c3a3604047343e23acc036296f512071bc9?context=explore [3] https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental&id=7c49e8b195ad90f6ca9dfccce9f6e3e39a8676f6 Image size =========== 6.4M large.composefs 5.7M large.composefs.w/o.digest (w/o --compute-digest) 6.2M large.erofs 5.2M large.erofs.T0 (with -T0, i.e. w/o nanosecond timestamp) 1.7M large.squashfs 5.8M large.squashfs.uncompressed (with -noI -noD -noF -noX) (large.erofs.T0 is built without nanosecond timestamp, so that we get smaller disk inode size (same with squashfs).) Runtime Perf ============= The "uncached" column is tested with: hyperfine -p "echo 3 > /proc/sys/vm/drop_caches" "ls -lR $MNTPOINT" While the "cached" column is tested with: hyperfine -w 1 "ls -lR $MNTPOINT" erofs and squashfs are mounted with loopback device. | uncached(ms)| cached(ms) ----------------------------------|-------------|----------- composefs (with digest) | 326 | 135 erofs (w/o -T0) | 264 | 172 erofs (w/o -T0) + overlayfs | 651 | 238 squashfs (compressed) | 538 | 211 squashfs (compressed) + overlayfs | 968 | 302 squashfs (uncompressed) | 406 | 172 squashfs (uncompressed)+overlayfs | 833 | 264 Following on are the detailed test statistics: composefs(with digest) - uncached Benchmark 1: ls -lR /mnt/cps Time (mean ± σ): 326.0 ms ± 6.1 ms [User: 64.1 ms, System: 126.0 ms] Range (min … max): 316.3 ms … 334.5 ms 10 runs composefs(with digest) - cached Benchmark 1: ls -lR /mnt/cps Time (mean ± σ): 135.5 ms ± 4.1 ms [User: 59.9 ms, System: 74.8 ms] Range (min … max): 129.5 ms … 144.8 ms 21 runs loopback erofs(w/o -T0) - uncached Benchmark 1: ls -lR /mnt/bootstrap Time (mean ± σ): 264.1 ms ± 2.1 ms [User: 66.7 ms, System: 166.2 ms] Range (min … max): 261.0 ms … 267.5 ms 10 runs loopback erofs(w/o -T0) - cached Benchmark 1: ls -lR /mnt/bootstrap Time (mean ± σ): 172.3 ms ± 3.9 ms [User: 59.3 ms, System: 112.2 ms] Range (min … max): 166.5 ms … 180.8 ms 17 runs overlayfs + loopback erofs(w/o -T0) - uncached Benchmark 1: ls -lR /mnt/ovl/mntdir Time (mean ± σ): 651.8 ms ± 8.8 ms [User: 74.2 ms, System: 391.1 ms] Range (min … max): 632.6 ms … 665.8 ms 10 runs overlayfs + loopback erofs(w/o -T0) - cached Benchmark 1: ls -lR /mnt/ovl/mntdir Time (mean ± σ): 238.1 ms ± 7.7 ms [User: 63.4 ms, System: 173.4 ms] Range (min … max): 226.7 ms … 251.2 ms 12 runs loopback squashfs (compressed) - uncached Benchmark 1: ls -lR /mnt/squashfs-compressed/bootstrap Time (mean ± σ): 538.4 ms ± 2.4 ms [User: 67.8 ms, System: 410.3 ms] Range (min … max): 535.6 ms … 543.6 ms 10 runs loopback squashfs (compressed) - cached Benchmark 1: ls -lR /mnt/squashfs-compressed/bootstrap Time (mean ± σ): 211.3 ms ± 2.9 ms [User: 61.2 ms, System: 141.3 ms] Range (min … max): 206.5 ms … 216.1 ms 13 runs overlayfs + loopback squashfs (compressed) - uncached Benchmark 1: ls -lR /mnt/squashfs-compressed/mntdir Time (mean ± σ): 968.0 ms ± 7.1 ms [User: 78.4 ms, System: 675.7 ms] Range (min … max): 956.4 ms … 977.2 ms 10 runs overlayfs + loopback squashfs (compressed) - cached Benchmark 1: ls -lR /mnt/squashfs-compressed/mntdir Time (mean ± σ): 302.6 ms ± 6.7 ms [User: 67.3 ms, System: 225.6 ms] Range (min … max): 292.4 ms … 312.3 ms 10 runs loopback squashfs (uncompressed) - uncached Benchmark 1: ls -lR /mnt/squashfs-uncompressed/bootstrap Time (mean ± σ): 406.6 ms ± 3.9 ms [User: 69.2 ms, System: 273.3 ms] Range (min … max): 400.3 ms … 414.2 ms 10 runs loopback squashfs (uncompressed) - cached Benchmark 1: ls -lR /mnt/squashfs-uncompressed/bootstrap Time (mean ± σ): 172.8 ms ± 3.2 ms [User: 61.9 ms, System: 101.6 ms] Range (min … max): 168.6 ms … 178.9 ms 16 runs overlayfs + loopback squashfs (uncompressed) - uncached Benchmark 1: ls -lR /mnt/squashfs-uncompressed/mntdir Time (mean ± σ): 833.4 ms ± 8.0 ms [User: 74.1 ms, System: 539.7 ms] Range (min … max): 820.7 ms … 844.3 ms 10 runs overlayfs + loopback squashfs (uncompressed) - cached Benchmark 1: ls -lR /mnt/squashfs-uncompressed/mntdir Time (mean ± σ): 264.4 ms ± 7.2 ms [User: 68.2 ms, System: 186.2 ms] Range (min … max): 256.5 ms … 277.1 ms 10 runs -- Thanks, Jingbo
