Re: zlib-ng as a compat replacement for zlib

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



In the end most of the patches were dropped the uplift wasn't worth the
effort of maintaining them downstream, when the effort can be better
spent getting a 10X uplift using a more modern compression
implementation (ex zstd/lz4/lzo/etc) that isn't written with so many
byte oriented assumptions.

Yes, generally having downstream patches is not the goal we want to chase in Fedora/RHEL.
It takes too much effort and knowledge which could be used much more efficiently.

And also one of the key values of Red Hat is "upstream first", so we always propose the patches to upstream making it a win-win for both sides.
Unfortunately, zlib's upstream isn't too welcoming for complex patches such as these, and most of them are stuck in open PR.

That is a big plus for zlib-ng as its upstream is open for such PRs.
On the other hand, changes like these could lead to broken compatibility which is a big concern for our products in the case of widely used packages such as zlib.
 


On Tue, Aug 22, 2023 at 10:26 PM Jeremy Linton <jeremy.linton@xxxxxxx> wrote:
Hi,

On 8/6/23 08:33, John Reiser wrote:
> On 8/6/23 02:00, Peter Robinson wrote:
>> We tried to pull some of the optimisations in some time ago to the
>> Fedora package and they caused some issues with compatibility.
>
> Please provide *any* documentation!  Such as: the dates the work was
> performed,
> the participants, the nature of the issues, the "other side" of the problem
> cases (the other packages, the use cases, etc.)

Waves, some of this was my fault.

example bug: https://bugzilla.redhat.com/show_bug.cgi?id=1582555
look at the zlib rpm history you will see things like:

https://src.fedoraproject.org/rpms/zlib/c/71a74f9c8684dd99c0e0657f53affb90a3ca3219?branch=rawhide

there were a few others that were ignored, or we reverted part of the
original set of 5 or 6 patches. The original patches were aarch64 + NEON
optimizations, but there were a number of issues around unittests in
various packages that zipped something then validated the results
against a known crc/hash/etc which then failed because the hash changed,
the size changed, padding issues, the optimized code touched valid parts
of the buffer and tripped buffer poisoning logic, etc.

Turns out zlib is old school byte oriented and any slight behavioral
change can result in compatibility issues. The first obvious
optimization is to increase the fetched word/matching sizes, which
maintain binary compatibility with the zlib format/decompressor but
results in buffer len/compressed size deltas.

Of course some of these were potentially the fault of the patches, but
you have to decide between perf or compatibility when writing these, and
if the goal is faster, then the compatibility gets sacrificed.

Bugzilla is taking its time retrieving some of the BZs that were closed
without fixes. So you will have to search for them yourself.

In the end most of the patches were dropped the uplift wasn't worth the
effort of maintaining them downstream, when the effort can be better
spent getting a 10X uplift using a more modern compression
implementation (ex zstd/lz4/lzo/etc) that isn't written with so many
byte oriented assumptions.




> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue


--
S pozdravom/ Best regards

Lukáš Javorský

Software Engineer, Core service - Databases

Red Hat

Purkyňova 115 (TPB-C)

612 00 Brno - Královo Pole

ljavorsk@xxxxxxxxxx

_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux