On Mon, 2018-04-23 at 00:27 -0400, Neal Gompa wrote: > On Tue, Apr 17, 2018 at 3:05 PM, Jonathan Dieter <jdieter@xxxxxxxxx> wrote: > > I'm assuming that you're referring here to getting zchunk packaged into > > Fedora. I'd really like to finalize the file format (we're close, but > > I still need a good way of storing signatures in it) and the download > > API before releasing it into Fedora proper. > > > > I'm looking forward to this! I've updated the file format to allow for multiple signatures, updated the zchunk code to recognize the existence of a signature (while still not checking it), and have released as zchunk-0.3.0 in COPR. I've also added in 32-bits of flags that we can use to extend the format in a backwards-compatible way. The current zchunk format description is at: https://github.com/jdieter/zchunk/blob/master/zchunk_format.txt > I would recommend using the dicts mentioned above as they give me over > > 40% space savings for both other.xml.zck and primary.xml.zck. Do > > please let me know if you run into any problems. > > > > Are those dictionaries Fedora specific? If so, how can other > distributions generate similar ones? If not, still, how were they > made? :) They were generated from Fedora metadata, but they should help with any distribution's repodata. I generated them by splitting a few day's worth of metadata along package boundaries, stripping out any checksums, and then running zstd --train * on the directory containing the split metadata. The script I used is available at https://www.jdieter.net/downloads/zchunk-dicts/split.py, and I hope to write up proper instructions at some point. Jonathan _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
