Re: zlib-ng as a compat replacement for zlib

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote:
> > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote:
> > > Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben:
> > > > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
> > > > > [ Cc: qemu-block ]
> > > > > 
> > > > > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben:
> > > > > > On Tue, Aug 29, 2023 at 05:49:24PM -0000, Daniel Alley wrote:
> > > > > > > > The background to this is I've spent far too long trying to optimize
> > > > > > > > the conversion of qcow2 files to raw files.  Most existing qcow2 files
> > > > > > > > that you can find online are zlib compressed, including the qcow2
> > > > > > > > images provided by Fedora.  Each cluster in the file is separately
> > > > > > > > compressed as a zlib stream, and qemu uses zlib library functions to
> > > > > > > > decompress them.  When downloading and decompressing these files, I
> > > > > > > > measured 40%+ of the total CPU time is doing zlib decompression.
> > > > > > > > 
> > > > > > > > [You don't need to tell me how great Zstd is, qcow2 supports this for
> > > > > > > > compression also, but it is not widely used by existing content.]
> > > > > 
> > > > > You make it sound like compressing each cluster individually has a big
> > > > > impact. If so, does increasing the cluster size make a difference, too?
> > > > > That could be an change with less compatibility concerns.
> > > > 
> > > > The issue we're discussing in the original thread is speed of
> > > > decompression.  We noted that using zlib-ng (a not-quite drop-in
> > > > replacement for zlib) improves decompression speed by 40% or more.
> > > > 
> > > > Original thread:
> > > > https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/thread/CDNPJ4SOTRQMYVCDI3ZSY4SP4FYESCWD/
> > > > zlib-ng proposed change:
> > > > https://src.fedoraproject.org/rpms/zlib-ng/pull-request/3
> > > > 
> > > > Size of the compressed file is also a concern, but wasn't discussed.
> > > 
> > > I understand the context and didn't really think about file size at all.
> > > 
> > > My question was essentially if decompressing many small blocks (as we do
> > > today) performs significantly different from decompressing fewer larger
> > > blocks (as we would do with a larger cluster size).
> > 
> > I did a quick test just by adjusting the cluster size of a qcow2 file:
> > 
> >   $ virt-builder fedora-36
> >   $ ls -lsh fedora-36.img 
> >   1.2G -rw-r--r--. 1 rjones rjones 6.0G Sep  1 09:53 fedora-36.img
> >   $ cat fedora-36.img fedora-36.img fedora-36.img fedora-36.img  > test.raw
> >   $ ls -lsh test.raw 
> >   4.7G -rw-r--r--. 1 rjones rjones 24G Sep  1 09:53 test.raw
> >   $ qemu-img convert -f raw test.raw -O qcow2 test.qcow2.zlib.4k -c -o compression_type=zlib,cluster_size=4096
> > 
> > (for cluster sizes 4k, 64k, 512k, 2048k, and
> > compression types zlib & zstd)
> > 
> > I tested the speed of decompression using:
> > 
> >   $ hyperfine 'qemu-img convert -W -m 16 -f qcow2 test.qcow2.XXX -O raw test.out'
> >   (qemu 8.0.0-4.fc39.x86_64)
> > 
> >   $ hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XXX --run '\''nbdcopy --request-size "$uri" test.out'\'' '
> >   (nbdkit-1.35.11-2.fc40.x86_64)
> > 
> > Results:
> > 
> >   Cluster  Compression  Compressed size  Prog   Decompression speed
> > 
> >   4k       zlib         3228811264       qemu   5.921 s ±  0.074 s
> >   4k       zstd         3258097664       qemu   5.189 s ±  0.158 s
> > 
> >   4k       zlib/zstd                     nbdkit failed, bug!!
> > 
> >   64k      zlib         3164667904       qemu   3.579 s ±  0.094 s
> >   64k      zstd         3132686336       qemu   1.770 s ±  0.060 s
> > 
> >   64k      zlib         3164667904       nbdkit 1.254 s ±  0.065 s
> >   64k      zstd         3132686336       nbdkit 1.315 s ±  0.037 s
> > 
> >   512k     zlib         3158744576       qemu   4.008 s ±  0.058 s
> >   512k     zstd         3032697344       qemu   1.503 s ±  0.072 s
> > 
> >   512k     zlib         3158744576       nbdkit 1.702 s ±  0.026 s
> >   512k     zstd         3032697344       nbdkit 1.593 s ±  0.039 s
> > 
> >   2048k    zlib         3197569024       qemu   4.327 s ±  0.051 s
> >   2048k    zstd         2995143168       qemu   1.465 s ±  0.085 s
> > 
> >   2048k    zlib         3197569024       nbdkit 3.660 s ±  0.011 s
> >   2048k    zstd         2995143168       nbdkit 3.368 s ±  0.057 s
> > 
> > No great surprises - very small cluster size is inefficient, but
> > scaling up the cluster size gain performance, and zstd performs better
> > than zlib once the cluster size is sufficiently large.
> 
> The default qcow2 cluster size is 64k, which means we've already
> got the vast majority of the perfornmance and file size win. Going
> beyond 64k defaults doesn't seem massively compelling.
> 
> zstd does have a small space win over zlib as expected, but again
> nothing so drastic that it seems compelling to change - that win
> will be line noise in the overall bigger picture of image storage
> and download times.

Yeah, I was a bit surprised by this.  I expected zstd files to be
significantly smaller than zlib even though that's not what zstd is
optimized for.  Not that they'd be about the same.

> The major difference here is that zstd is much faster than zlib
> at decompress. I'd be curious if zlib-ng closes that gap ?

It's quite hard to use zlib-ng in Fedora (currently) since it requires
changes to the source code.  That is what the pull request being
discussed would change, as you could simply install zlib-ng-compat
which would replace libz.so.  But anyway I can't easily get results
for qemu + zlib-ng, but we expect it would be ~ 40% faster at
decompression, and decompression is what is taking most of the time in
the qemu numbers above.

I forgot to say that nbdkit is using zlib-ng, since I made the source
level changes a few weeks back (but most of the nbdkit performance
improvement comes from being able to use lots of threads).

> If it does, then for the sake of image portability it'd be better
> to stick with zlib compression in qcow2 and leverage zlib-ng for
> speed, and ignore zstd.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux