Re: [Rpm-ecosystem] Zchunk update

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, 2018-04-23 at 00:27 -0400, Neal Gompa wrote:
> On Tue, Apr 17, 2018 at 3:05 PM, Jonathan Dieter <jdieter@xxxxxxxxx> wrote:
> > I'm assuming that you're referring here to getting zchunk packaged into
> > Fedora.  I'd really like to finalize the file format (we're close, but
> > I still need a good way of storing signatures in it) and the download
> > API before releasing it into Fedora proper.
> > 
> 
> I'm looking forward to this!

I've updated the file format to allow for multiple signatures, updated
the zchunk code to recognize the existence of a signature (while still
not checking it), and have released as zchunk-0.3.0 in COPR.  I've also
added in 32-bits of flags that we can use to extend the format in a
backwards-compatible way.

The current zchunk format description is at:
https://github.com/jdieter/zchunk/blob/master/zchunk_format.txt

> I would recommend using the dicts mentioned above as they give me over
> > 40% space savings for both other.xml.zck and primary.xml.zck.  Do
> > please let me know if you run into any problems.
> > 
> 
> Are those dictionaries Fedora specific? If so, how can other
> distributions generate similar ones? If not, still, how were they
> made? :)

They were generated from Fedora metadata, but they should help with any
distribution's repodata.  I generated them by splitting a few day's
worth of metadata along package boundaries, stripping out any
checksums, and then running zstd --train * on the directory containing
the split metadata.  The script I used is available at
https://www.jdieter.net/downloads/zchunk-dicts/split.py, and I hope to
write up proper instructions at some point.

Jonathan
_______________________________________________
infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx




[Index of Archives]     [Fedora Development]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]

  Powered by Linux