On 07/08/2023 15:29, Tulio Magno Quites Machado Filho wrote:
"Richard W.M. Jones" <rjones@xxxxxxxxxx> writes:
The background to this is I've spent far too long trying to optimize
the conversion of qcow2 files to raw files. Most existing qcow2 files
that you can find online are zlib compressed, including the qcow2
images provided by Fedora. Each cluster in the file is separately
compressed as a zlib stream, and qemu uses zlib library functions to
decompress them. When downloading and decompressing these files, I
measured 40%+ of the total CPU time is doing zlib decompression.
This number may go even higher on s390x [1] because zlib-ng supports
hardware acceleration.
qatzip [2] and libnxz [3] should provide performance on the same order of
magnitude for Intel and Power9 processors, with the negative side of not
using a single library.
We already package zlib-ng in Fedora:
https://src.fedoraproject.org/rpms/zlib-ng/blob/rawhide/f/zlib-ng.spec
https://github.com/zlib-ng/zlib-ng
In my test zlib-ng is about 40% faster.
Sadly zlib-ng is not compiled with the ZLIB_COMPAT option. What this
means in practice is that the zlib functions have different names
(eg. zng_inflateInit instead of inflateInit). It is not a drop-in
replacement for zlib and software would need to be adjusted to use it.
However there is this bug / RFE to package the compat library.
https://bugzilla.redhat.com/show_bug.cgi?id=2145239
It literally replaces /usr/lib64/libz.so.1, so pretty high stakes. It
has the appropriate Provides/Conflicts.
Anyway, 40% (or whatever, but significant) performance improvement
sounds good for a very widely used operation.
So I'd like to ask Fedora ...
What do we think about the opt in approach of adding a patch similar
to the one proposed in bug 2145239, where I think you could "simply"
install zlib-ng to get better performance with existing software?
("simply" because it seems high risk of going wrong)
Wearing my zlib-ng fedora maintainer hat:
I like the idea of this patch, although I think it needs a few changes.
I'd like to merge it and add another package with the same source code,
but different value for compat, i.e. package zlib-ng would still
distribute the zlib-ng API while the new package would distribute the
zlib-compatible API.
That way, if there are any projects using the zlib-ng API, they could
still be supported.
[1] https://github.com/rust-lang/libz-sys/pull/72#issuecomment-904020105
[2] https://src.fedoraproject.org/rpms/qatzip
[3] https://src.fedoraproject.org/rpms/libnxz
We were discussing the use of zlib-ng in Hyperscale and ISA SIG with Florian Weimer, Honza Horak, Matej Muzila and Neal Gompa for a while.
I've started to prepare a change proposal for Fedora 40, so any additional ideas are more than welcome.
--
Ali Erdinc Koroglu
Intel, SSE | Linux OS Systems Engineering
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
