Hi folks, A new version erofs-utils 1.5 is available at: git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git tags/v1.5 It mainly includes the following changes: - (fsck.erofs) support filesystem extraction (Igor Ostapenko); - support ztailpacking inline feature for compressed files (Yue Hu); - (dump.erofs) support listing directories; - more liberofs APIs (including iterate APIs) (me, Kelvin Zhang); - use mtime to allow more control over the timestamps (David Anderson); - switch to GPL-2.0+ OR Apache-2.0 dual license for liberofs; - various bugfixes and cleanups; A little bit delay this time for more than half a year. This release mainly includes ztailpacking feature by Yue Hu which can inline the tail pcluster with its inode metadata thus save space and a tail I/O, it's highly recommended to be enabled if possible. Apart from that, fsck.erofs now supports extracting filesystem, thanks to Igor Ostapenko. There are other changes listed above. In the end, I'd like to update the roadmap of EROFS since the last update for the coming year: https://lore.kernel.org/r/20211009061150.GA7479@hsiangkao-HP-ZHAN-66-Pro-G1 Thankfully many of them are finished during the past year. 1. Common stuffs: - Switch to folios and enable large folios if possible in the next cycles; - Get rid of PG_error flag in Linux 5.20 (pending review); - Explore byte-addressed rolling hash compression + deduplication since on-disk format already supports such way but needs runtime tuning; - LZ4 range dictionary support. We don't have enough manpower on this yet, but hopefully it can have some progress in the coming year; - Further code cleanups. 2. Container image use cases: - Recently, we posted a article to introduce erofs over fscache feature working with CNCF Dragonfly Nydus image service and give some performance numbers. https://d7y.io/blog/2022/06/06/evolution-of-nydus/ Our Alibaba kernel team are still working on several stuffs about Nydus image service and fscache, including: - Better flexible cache management, including repacking and blob GC in order to make better use to the local cache database; - Convert and run (e)stargz and others on the fly with fscache feature. In the future, different formats are also able to be merged in one fs tree: https://github.com/dragonflyoss/image-service/pull/486 - Runtime decompression support over fscache; - Blob cache sharing within the same trusted domain; - Page cache sharing between different files with the same chunk; - Enhanced convergent encryption to share chunk data in a trusted domain and runtime verification; - And other fscache/cachefiles common improvements like fallback format, multiple daemons/dirs, FSDAX, etc. - Apart from Nydus, it's planned to introduce a native fscache daemon integrated in erofs-utils to mount EROFS, (e)stargz images from network as well as provide fscache interfaces as liberofs APIs. 3. Embedded devices - Yue Hu is currently working on a fragment-likewise feature, which can merged tails or the whole files into a special inode in order to minimize the space; - Multi-threaded mkfs, fsck.erofs. I know someone is working on this but I'm not sure the current progress. It's a bit delay but needs to be resolved anyway. Thanks, Gao Xiang
