On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote: > [ Cc: qemu-block ] > > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben: > > On Tue, Aug 29, 2023 at 05:49:24PM -0000, Daniel Alley wrote: > > > > The background to this is I've spent far too long trying to optimize > > > > the conversion of qcow2 files to raw files. Most existing qcow2 files > > > > that you can find online are zlib compressed, including the qcow2 > > > > images provided by Fedora. Each cluster in the file is separately > > > > compressed as a zlib stream, and qemu uses zlib library functions to > > > > decompress them. When downloading and decompressing these files, I > > > > measured 40%+ of the total CPU time is doing zlib decompression. > > > > > > > > [You don't need to tell me how great Zstd is, qcow2 supports this for > > > > compression also, but it is not widely used by existing content.] > > You make it sound like compressing each cluster individually has a big > impact. If so, does increasing the cluster size make a difference, too? > That could be an change with less compatibility concerns. The issue we're discussing in the original thread is speed of decompression. We noted that using zlib-ng (a not-quite drop-in replacement for zlib) improves decompression speed by 40% or more. Original thread: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/thread/CDNPJ4SOTRQMYVCDI3ZSY4SP4FYESCWD/ zlib-ng proposed change: https://src.fedoraproject.org/rpms/zlib-ng/pull-request/3 Size of the compressed file is also a concern, but wasn't discussed. > > > Independent from the decision to use zlib-ng as a distro-wide zlib > > > replacement, which is a good idea regardless. > > > > > > Are there reasons why Fedora's qcow2 images cannot switch to Zstd > > > compression? Zstd support appears to have been added by QEMU 5.1 in > > > August 2020, and both EL8 and EL9 appear to have newer versions QEMU > > > available (therefore, they ought to be able to support those > > > images). > > > > TBH I think the most probable reason is that people don't know about > > it and it is not obvious that you have to enable it. To generate a > > zlib-compressed qcow2 file, you simply add the -c option, easy. To > > use zstd compression you have to use this mouthful: > > > > qemu-img convert -f raw disk.img -O qcow2 disk.qcow2 -c -o compression_type=zstd > > > > The qemu-img man page doesn't even mention it. > > Good point, that needs to be fixed. > > (Though I don't think that '-o compression_type=zstd' in an unreasonable > mouthful for enabling a non-standard compression algorithm.) > > > I think all recent qcow2-based tools should be fine with zstd, but I > > didn't check them all (RHEL 7 is still quite popular so that platform > > would no longer be compatible). > > So my first thought was that maybe we can just change the default now > that the option has been there for quite a while. But then it occurred > to me that it's not a hard dependency. So at least, the default would > still have to be zlib if zstd isn't even compiled in, which makes it a > bit less nice in theory. In practice, it depends on what build options > distros actually use. > > Unfortunately, we seem to build the RHEL packages with --disable-zstd > (I suppose just because we tend to disable everything nobody explicitly > asked for). Maybe we should check other distros. If zstd is commonly > enabled, we could still make it the default upstream (because honestly, > it probably does makes sense from an upstream perspective). Of course, > for RHEL this would mean that images out there are likely to use zstd > soon, so it might need to enable zstd in newer versions, too. Oh that is bad actually. We really should enable zstd support in RHEL :-/ Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
