https://fedoraproject.org/wiki/Changes/OptimizeSquashFS == Summary == Improve compression ratio of SquashFS filesystem on the installation media. == Owner == * Name: [[User:bkhomuts|Bohdan Khomutskyi]] * Email: bkhomuts@xxxxxxxxxx == Detailed Description == As of Fedora 31, the LiveOS/squashfs.img file on the installation image, is compressed with default block size of mksquashfs. The default block size is 128k. Additionaly, lorax sets BCJ filter depending on the architecture. Those parameters can be adjusted which will lead to a better compression ratio and/or reduction of the CPU usage at build time. This is simple to achieve. Recently, Lorax has gotten support for adjusting the compression options for mksquashfs via the configuration file. The file should be altered as following: <pre> [compression] bcj = no args = -b 1M -Xdict-size 1M -no-recovery </pre> Where -b 1M and -Xdict-size 1M are block and dictionary sizes respectively; bcj -- branch-call-jump filter. Based on the results above, I'd suggest selecting the following ''optimal configuration'': XZ algorithm, with block size of 1MiB and without BCJ filter (plain xz -b 1M, without -Xbcj x86). On the right, you can see the impact of the compression algorithms on installation time. As can be seen from the picture on the right hand side, selecting 'plain xz -b 1M configuration' has minimal impact on the installation time and CPU usage during the installation. The compression will result in +6.51% or, in real terms, +24.94s additional installation time, compared to the savings of 142 MiB on the installation media, refer to the documentation section to download the ISO images. This increase in installation time will be compensated by the change in the installer: https://github.com/rhinstaller/anaconda/pull/2292 I noticed, that even with maximum compression, CPU is not fully utilized during installation. == Benefit to Fedora == * Reduction of the installation media size and the cost of storing and distributing Fedora. * Reduction of the CPU usage at build time. Depending on which compression parameters chosen. == Scope == * Proposal owners: The build environment should have support for adjusting the Lorax configuration file and -squashfs-only parameter. Lorax is a program that produces the LiveOS/squashfs.img file on the installation media. One of the ways to enable such customization is to introduce support in Pungi to pass -c option to Lorax. * Other developers: The pungi utility should support passing the custom configuration file location to the Lorax utility. This option should apply during buildInstall phase of pungi. * Release engineering: [https://pagure.io/releng/issue/9127] <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)? Is a mass rebuild required? include a link to the releng issue. The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication --> * Policies and guidelines: Not required. * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == N/A (not a System Wide Change) == How To Test == <pre> mkdir -p /mnt/new /mnt/old sudo mount -o loop,ro FedoraInstallationOld.iso sudo mount -o loop,ro FedoraInstallationNew.iso ls -l /mnt/{new,old}/LiveOS/squashfs.img </pre> And then calculate the size difference. == User Experience == * Decreasing the installation image size will reduce cost of mirroring and storing Fedora installation images. * Decreasing the installation image size will reduce the download time. * Increasing the block size on the current configuration with EXT4 file system, should increase latency while accessing the EXT4 filesystem. The exact impact is to be evaluated. * The impact of latency will be reduced, if the plain SquashFS option is be choosen. == Dependencies == Pungi, a utility that builds the compose, should include new functionality mentioned above. Alternatively, the /etc/lorax/lorax.conf should be altered in the environment where Lorax is running. == Contingency Plan == N/A == Documentation == https://pagure.io/releng/issue/9127.<br/> mksquashfs(1)<br/> lorax(1)<br/> https://docs.pagure.org/pungi<br/> Select ISOs can be downloaded at https://khomutsky.com/fedora-dvd/<br/> Spreadsheet pictured above [[File:Comparison Table SquashFS.ods]] [https://github.com/rhinstaller/anaconda/pull/2292 Multi-core decompression of SquashFS merge request] -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
