Re: A proof-of-concept for delta'ing repodata

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, 2018-02-13 at 10:52 +0100, Igor Gnatenko wrote:
> What about zstd? Also in latest version of lz4 there is support for
> dictionaries too.

So I've investigated zstd, and, here are my results:

Latest F27
primary.gz - 3.1MB

zlib zchunk (including custom dict)
primary.zck - 4.2MB ~35% increase

zstd zchunk (including dict generated from last three Fedora GA
primaries)
primary.zck - 3.7MB ~20% increase

Using zstd for filelists.xml has roughly the same increase as with
zlib, which is expected as the chunks are larger and thus get better
compression even without a dict.

I did also look briefly at lz4, but it seems that it's major advantage
is speed, and I'm not sure that metadata decompression speed is our
main bottleneck in dnf.

With these numbers, I think it makes sense to move forward with zstd
instead of zlib.

Jonathan

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx

[Index of Archives]     [Fedora Development]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]

  Powered by Linux