On Thu, Mar 21, 2024 at 8:20 AM Stephen Smoogen <ssmoogen@xxxxxxxxxx> wrote: > > > > On Wed, 20 Mar 2024 at 22:01, Kevin Kofler via devel <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote: >> >> Aoife Moloney wrote: >> > The zstd compression type was chosen to match createrepo_c settings. >> > As an alternative, we might want to choose xz, >> >> Since xz consistently compresses better than zstd, I would strongly suggest >> using xz everywhere to minimize download sizes. However: >> >> > especially after zlib-ng has been made the default in Fedora and brought >> > performance improvements. >> >> zlib-ng is for gz, not xz, and gz is fast, but compresses extremely poorly >> (which is mostly due to the format, so, while some implementations manage to >> do better than others at the expense of more compression time, there is a >> limit to how well they can do and it is nowhere near xz or even zstd) and >> should hence never be used at all. >> > > There are two parts to this which users will see as 'slowness'. Part one is downloading the data from a mirror. Part two is uncompressing the data. In work I have been a part of, we have found that while xz gave us much smaller files, the time to uncompress was so much larger that our download gains were lost. Using zstd gave larger downloads (maybe 10 to 20% bigger) but uncompressed much faster than xz. This is data dependent though so it would be good to see if someone could test to see if xz uncompression of the datafiles will be too slow. > Fedora has been using optimized zstd compression "by default" since Fedora 30 anyway with Zchunk metadata: https://fedoraproject.org/wiki/Changes/Zchunk_Metadata Regular zstd compression is less optimized due to the lack of dictionaries, but it's also effectively the fallback path, though much faster to decompress while providing pretty good compression (which is why we have been gradually switching *everything* to zstd). -- 真実はいつも一つ！/ Always, there's only one truth! -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
