On Thu, Jan 16, 2025 at 8:00 PM Gao Xiang <hsiangkao@xxxxxxxxxxxxxxxxx> wrote: > > (resend this due to lack of subscription. and I just quote some reply to express my basic ideas on this stuff) > > > There's one thing the EROFS developers never talk about, and that's metadata compression. EROFS doesn't do it, but Squashfs does, and it has done since 2002. > > Did you notice that https://erofs.docs.kernel.org/en/latest/faq.html ? > > > The tests are all done in way to flater EROFS and put Squashfs in the worst light. For example see https://erofs.docs.kernel.org/en/latest/comparsion/dedupe.html. They deliberately disable Squashfs metadata compression because EROFS doesn't support it, and use 128K blocks despite Squashfs supporting 1Mbyte blocks (look for the line [1] SquashFS uses –b 131072 by default, -noI will disable its metadata compression.). > > It just shows that SquashFS doesn't support extent-based deduplication (because SquashFS on-disk indexes just record compressed sizes and cannot be randomly indexed), since EROFS doesn't support metadata compression so I benchmarked against SquashFS and explicitly point out the metadata compression is off on the SquashFS side to show the benefits of deduplication. Why it's called unbiased? > > Don't make me wrong, if -b 1048576 (1m chunks) is used, then the chunk deduplication possibility is low. > But not anyone cares image size (especially for smartphone) just by using large extent sizes (like 1MiB) and real-time performance is important for them, decompress 1MiB to get 4KiB random data is unacceptable for them. > > > By doing this they deliberately reduce the compression and speed of Squashfs in their tests. This IMHO is biased and unethical. But that is how it is. > > Also if I were a random filesystem author, I will never argue just due to lack of development resource. > > Whether EROFS will use for Fedora LiveOS finally, honestly I don't care (although I wish it could be used more widely), I will focus on new features that end users really care and it's really an input. I only care EROFS can be totally usable on RHEL/CentOS since we have customers to use these OSes and I will serve for their detailed use cases. > > Please note, EROFS was once proposed to resolve SquashFS unacceptable random runtime performance issues on the high-performance Android scenerios, and almost all smartphone vendors on the market use EROFS now. As for other scenarios, I would just call it as a plus. > > If users think metadata compression is useful to minimize their images, and I will consider to add this in my spare time (Because it's never useful for my paid job too, they even don't care about multithreaded compression). Also I did not ask random vendors to contribute EROFS, just because Bytedance or other vendors need it and they develop new features for their use cases. > I certainly would appreciate features like metadata compression, BCJ filters, and anything to improve image creation time when -Ededupe is used. I also wanted to use zstd compression instead of lzma (since it decompresses way faster), but the compression performance isn't quite there yet. Runtime speed is really valuable for us because typical media that Linux live environments are running from are slower than even most mobile device flash storage modules. At worst, we're talking about actually booting from a DVD (which is still a thing people do in some places), but we also need maximal space savings because of download bandwidth and fitting on small media. -- 真実はいつも一つ！/ Always, there's only one truth! -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
