Hello everyone,
Thanks for sharing your ideas and comments about this change.
Thanks to Mohan Boddu, I got RawHide DVD and netinstall images of
RawHide with the optimization features enabled. Those test composes are
available at the following locations:
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20200828.n.0/
(Plain SquashFS)
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20200908.n.1/compose/Server/x86_64/iso/
(Plain SquashFS and different xz compression options)
To select images from the test composes, I applied a patch from
https://github.com/rhinstaller/anaconda/pull/2829, so you can download
and run those images yourself to test the change:
https://drive.google.com/drive/folders/1tGsoqFMg2A6dQZHfuQNb9uDqYu9XEiPI
The result of benchmark will supplement already available data I
previously posted for Fedora Live images at
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
As a reminder, there are 2 levels for this optimization:
1. Making the SquashFS filesystem on the DVD plain (i.e. without
embedded EXT4 inside it) -- has the suffix plain-xz-128k.iso
2. In addition to #1, using a different compression parameters for the
SquashFS filesystem on the installation medium -- has the suffix
plain-xz-1M.iso
I propose to apply both of optimizations, using the highest possible
compression ratio supported by SquashFS. The highest compression ratio
in SquashFS corresponds to xz level 1 (out of 9 available presets).
Making the first change will reduce both x86-64 Boot and the x86-64 DVD
ISO by approximately 24 MiB. With both changes combined, the reduction
of size will be 70 MiB.
Because the SquashFS filesystem on the Live installation medium is of
bigger size, storage savings on the Live images are even higher (I
documented it in https://fedoraproject.org/wiki/Changes/OptimizeSquashFS)
On 05/09/2020 12:43, Zbigniew Jędrzejewski-Szmek wrote:
On Thu, Aug 27, 2020 at 11:13:26AM -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
...
Based on the results above, I'd suggest selecting the following
''optimal configuration'': XZ algorithm, with block size of 1MiB and
without BCJ filter (plain xz -b 1M, without -Xbcj x86).
On the right, you can see the impact of the compression algorithms on
installation time.
As can be seen from the picture on the right hand side, selecting
'plain xz -b 1M configuration' has minimal impact on the installation
time and CPU usage during the installation. The compression will
result in +6.51% or, in real terms, +24.94s additional installation
time, compared to the savings of 142 MiB on the installation media,
== Benefit to Fedora ==
* Reduction of the installation media size and the cost of storing and
distributing Fedora.
* Reduction of the CPU usage at build time. Depending on which
compression parameters chosen.
Hi Bohdan,
I think there's a misalignment of priorities.
My evaluation is the following: users won't care. The image size difference
is not big enough for people to notice. OTOH, slower installation will
impact QA and VM installations. We're doing more and more automated
installations and tests, and this change impacts those tests negatively.
This increase in installation time will be compensated by the change
in the installer:https://github.com/rhinstaller/anaconda/pull/2292
This PR is very interesting. But it seems that the more we optimize
things, the more slow decompression will be noticable. I.e. in some
way, right now, the slow decompression is obscured by the slow IO
speed, multiple levels of block device, or slow processing of the
uncompressed data. Any time the input or output speed is improved,
slow compression will be more of a bottleneck. So the approach of
increasing XZ compression levels has bad perspectives.
Zbyszek
_______________________________________________
devel mailing list --devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email todevel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
--
Bohdan Khomutskyi
RHEL Compose release engineer
EXD Software production
Red Hat
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
