On Thu, Jan 16, 2025 at 8:28 PM Gao Xiang <hsiangkao@xxxxxxxxxxxxxxxxx> wrote: > > > > On 2025/1/17 09:14, Neal Gompa wrote: > > On Thu, Jan 16, 2025 at 8:00 PM Gao Xiang <hsiangkao@xxxxxxxxxxxxxxxxx> wrote: > >> > >> (resend this due to lack of subscription. and I just quote some reply to express my basic ideas on this stuff) > >> > >>> There's one thing the EROFS developers never talk about, and that's metadata compression. EROFS doesn't do it, but Squashfs does, and it has done since 2002. > >> > >> Did you notice that https://erofs.docs.kernel.org/en/latest/faq.html ? > >> > >>> The tests are all done in way to flater EROFS and put Squashfs in the worst light. For example see https://erofs.docs.kernel.org/en/latest/comparsion/dedupe.html. They deliberately disable Squashfs metadata compression because EROFS doesn't support it, and use 128K blocks despite Squashfs supporting 1Mbyte blocks (look for the line [1] SquashFS uses –b 131072 by default, -noI will disable its metadata compression.). > >> > >> It just shows that SquashFS doesn't support extent-based deduplication (because SquashFS on-disk indexes just record compressed sizes and cannot be randomly indexed), since EROFS doesn't support metadata compression so I benchmarked against SquashFS and explicitly point out the metadata compression is off on the SquashFS side to show the benefits of deduplication. Why it's called unbiased? > >> > >> Don't make me wrong, if -b 1048576 (1m chunks) is used, then the chunk deduplication possibility is low. > >> But not anyone cares image size (especially for smartphone) just by using large extent sizes (like 1MiB) and real-time performance is important for them, decompress 1MiB to get 4KiB random data is unacceptable for them. > >> > >>> By doing this they deliberately reduce the compression and speed of Squashfs in their tests. This IMHO is biased and unethical. But that is how it is. > >> > >> Also if I were a random filesystem author, I will never argue just due to lack of development resource. > >> > >> Whether EROFS will use for Fedora LiveOS finally, honestly I don't care (although I wish it could be used more widely), I will focus on new features that end users really care and it's really an input. I only care EROFS can be totally usable on RHEL/CentOS since we have customers to use these OSes and I will serve for their detailed use cases. > >> > >> Please note, EROFS was once proposed to resolve SquashFS unacceptable random runtime performance issues on the high-performance Android scenerios, and almost all smartphone vendors on the market use EROFS now. As for other scenarios, I would just call it as a plus. > >> > >> If users think metadata compression is useful to minimize their images, and I will consider to add this in my spare time (Because it's never useful for my paid job too, they even don't care about multithreaded compression). Also I did not ask random vendors to contribute EROFS, just because Bytedance or other vendors need it and they develop new features for their use cases. > >> > > > > I certainly would appreciate features like metadata compression, BCJ > > filters, and anything to improve image creation time when -Ededupe is > > used. I also wanted to use zstd compression instead of lzma (since it > > decompresses way faster), but the compression performance isn't quite > > there yet. > > Since it's not my paid job, I will try my best to work them out in > my spare time. But really, for our own use cases we don't care much > about them, it's also called "a labor of love". > I understand this very well. Most of my FOSS work would fall in that category too. > > > > Runtime speed is really valuable for us because typical media that > > Linux live environments are running from are slower than even most > > mobile device flash storage modules. At worst, we're talking about > > actually booting from a DVD (which is still a thing people do in some > > places), but we also need maximal space savings because of download > > bandwidth and fitting on small media. > > Anyway, EROFS was once specifically designed for Android devices > initially, I will try to make it work better for other useful cases, > questions, issues, and features are much helpful for me to improve > the filesystem itself. > Thank you, I appreciate whatever you can do to make things better for our use-cases. :) -- 真実はいつも一つ！/ Always, there's only one truth! -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
