Hello Zbyszek,
> You haven't really answered the "why" part: why is it so important to
save 50MB? And why is the effect on QA less important?
From my perspective, the storage on the installation medium should be
efficiently used. Even though the optimization is just 50MiB, it is an
optimization.
And this optimization is transparent in terms of content.
> And why is the effect on QA less important?
The QA team is only one group of users. We should rather orient at end
users, and the way those users install Fedora.
On 15/09/2020 12:03, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Sep 15, 2020 at 11:18:04AM +0200, Bohdan Khomutskyi wrote:
Hello everyone,
Thanks for sharing your ideas and comments about this change.
Thanks to Mohan Boddu, I got RawHide DVD and netinstall images of
RawHide with the optimization features enabled. Those test composes
are available at the following locations:
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20200828.n.0/
(Plain SquashFS)
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20200908.n.1/compose/Server/x86_64/iso/
(Plain SquashFS and different xz compression options)
To select images from the test composes, I applied a patch from
https://github.com/rhinstaller/anaconda/pull/2829, so you can
download and run those images yourself to test the change:
https://drive.google.com/drive/folders/1tGsoqFMg2A6dQZHfuQNb9uDqYu9XEiPI
The result of benchmark will supplement already available data I
previously posted for Fedora Live images at
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
As a reminder, there are 2 levels for this optimization:
1. Making the SquashFS filesystem on the DVD plain (i.e. without
embedded EXT4 inside it) -- has the suffix plain-xz-128k.iso
This sounds excellent. We should get both better time and space efficiency.
2. In addition to #1, using a different compression parameters for
the SquashFS filesystem on the installation medium -- has the suffix
plain-xz-1M.iso
And this ones trades times for space. Considering that the space
difference is only ~50 MB, I don't think it's worth it, for all the
reasons outlined before.
You haven't really answered the "why" part: why is it so important to
save 50MB? And why is the effect on QA less important?
Zbyszek
I propose to apply both of optimizations, using the highest possible
compression ratio supported by SquashFS. The highest compression
ratio in SquashFS corresponds to xz level 1 (out of 9 available
presets).
Making the first change will reduce both x86-64 Boot and the x86-64
DVD ISO by approximately 24 MiB. With both changes combined, the
reduction of size will be 70 MiB.
Because the SquashFS filesystem on the Live installation medium is
of bigger size, storage savings on the Live images are even higher
(I documented it in
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS)
On 05/09/2020 12:43, Zbigniew Jędrzejewski-Szmek wrote:
On Thu, Aug 27, 2020 at 11:13:26AM -0400, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
...
Based on the results above, I'd suggest selecting the following
''optimal configuration'': XZ algorithm, with block size of 1MiB and
without BCJ filter (plain xz -b 1M, without -Xbcj x86).
On the right, you can see the impact of the compression algorithms on
installation time.
As can be seen from the picture on the right hand side, selecting
'plain xz -b 1M configuration' has minimal impact on the installation
time and CPU usage during the installation. The compression will
result in +6.51% or, in real terms, +24.94s additional installation
time, compared to the savings of 142 MiB on the installation media,
== Benefit to Fedora ==
* Reduction of the installation media size and the cost of storing and
distributing Fedora.
* Reduction of the CPU usage at build time. Depending on which
compression parameters chosen.
Hi Bohdan,
I think there's a misalignment of priorities.
My evaluation is the following: users won't care. The image size difference
is not big enough for people to notice. OTOH, slower installation will
impact QA and VM installations. We're doing more and more automated
installations and tests, and this change impacts those tests negatively.
This increase in installation time will be compensated by the change
in the installer:https://github.com/rhinstaller/anaconda/pull/2292
This PR is very interesting. But it seems that the more we optimize
things, the more slow decompression will be noticable. I.e. in some
way, right now, the slow decompression is obscured by the slow IO
speed, multiple levels of block device, or slow processing of the
uncompressed data. Any time the input or output speed is improved,
slow compression will be more of a bottleneck. So the approach of
increasing XZ compression levels has bad perspectives.
--
Bohdan Khomutskyi
Software Engineer
Red Hat
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
