Re: zlib-ng as a compat replacement for zlib

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Aug 5, 2023 at 4:12 PM Richard W.M. Jones <rjones@xxxxxxxxxx> wrote:
>
> The background to this is I've spent far too long trying to optimize
> the conversion of qcow2 files to raw files.  Most existing qcow2 files
> that you can find online are zlib compressed, including the qcow2
> images provided by Fedora.  Each cluster in the file is separately
> compressed as a zlib stream, and qemu uses zlib library functions to
> decompress them.  When downloading and decompressing these files, I
> measured 40%+ of the total CPU time is doing zlib decompression.
>
> [You don't need to tell me how great Zstd is, qcow2 supports this for
> compression also, but it is not widely used by existing content.]
>
> I was looking around for alternative zlib implementations and there
> are several.  This AWS blog is a decent summary:
>
>   https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-comparing-performance-with-other-zlib-forks/
>
> We already package zlib-ng in Fedora:
>
>   https://src.fedoraproject.org/rpms/zlib-ng/blob/rawhide/f/zlib-ng.spec
>   https://github.com/zlib-ng/zlib-ng
>
> In my test zlib-ng is about 40% faster.
>
> Sadly zlib-ng is not compiled with the ZLIB_COMPAT option.  What this
> means in practice is that the zlib functions have different names
> (eg. zng_inflateInit instead of inflateInit).  It is not a drop-in
> replacement for zlib and software would need to be adjusted to use it.
>
> However there is this bug / RFE to package the compat library.
>
>   https://bugzilla.redhat.com/show_bug.cgi?id=2145239
>
> It literally replaces /usr/lib64/libz.so.1, so pretty high stakes.  It
> has the appropriate Provides/Conflicts.
>
> Anyway, 40% (or whatever, but significant) performance improvement
> sounds good for a very widely used operation.
>
> So I'd like to ask Fedora ...
>
> What do we think about the opt in approach of adding a patch similar
> to the one proposed in bug 2145239, where I think you could "simply"
> install zlib-ng to get better performance with existing software?
> ("simply" because it seems high risk of going wrong)
>
> What about replacing zlib with zlib-ng?
>
> Other ideas ...?  Does anyone know what other distros are doing?
>

It's certainly an idea. In CentOS Hyperscale, we've discussed
proposing this for Fedora because of the better maintainability and
performance.

openSUSE ships zlib-ng-compat as a provider of zlib, though it is not
yet the default. They are working on build failures caused by
switching to zlib-ng as the provider of zlib.

I think it'd be worth us moving to zlib-ng as the sole zlib provider.


-- 
真実はいつも一つ!/ Always, there's only one truth!
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux