On Wed, Apr 08, 2020 at 08:27:08PM +0000, Jeremy Cline wrote: > Hi folks, > > The Fedora kernel is moving to maintaining the package in a source > (sometimes people refer to it as an "exploded") tree. Basically just a > fork of upstream. This makes a lot of packager tasks easier, but has > introduced a minor issue with respect to the lookaside cache. > > Right now, it's configured to create a tarball from the git tree and > upload it to the lookaside cache for each build. We build the rawhide > kernel every weekday (give or take) and the xz compressed source > tarball is ~110MB. This works out to about 28GB per year for Rawhide > alone (if this is a drop in the bucket and no one cares please let me > know and we'll just do this). The old approach uploaded a release > tarball and then incremental tarballs on top of that. Well, the netapp will dedupe some of those blocks, but it would be nice to save some room I would think. > If, however, Fedora allowed packagers to optionally generate tarballs > from a git repository we could just push the linux git repository. The > entire repository with history going back 15 years is under 4GB total, > which is pretty good when compared to ~419GB which is the space > required for the equivalent time using the lookaside cache. > > What would need to change: > > * Fedora offers a git repository to push source trees to. > > * A new file in the dist-git repository could be added if the packager > wishes called "source-repos". In it, it contains a git url and commit > identifier. For example, an entry might look like: > " > https://src.fedoraproject.org/sources/kernel.git v5.6" > where v5.6 is a tag in the repository. We can restrict it so the git > repository must be hosted by Fedora so we keep all the sources > forever. > > * fedpkg and fedpkg-minimal would need to be updated to pull the > source tree if the "source-repos" file is found and run > "git archive". Fortunately this work is actually already done since > Red Hat's version of fedpkg already supports this. > > I'm happy do to all the work for fedpkg/fedpkg-minimal to make this > possible because the other option is to add a bunch of hacks to the > kernel tooling to spit out a bunch of incremental tarballs to reduce > what we have to upload. > > I assume this is something that will need to go through the packaging > SIG, but from an infra side of things are there any thoughts/concerns? So, this was actually discussed already at some length: https://pagure.io/releng/issue/7498 Most of my concerns were answered, but then nim suggested this could already be done with the macros we have today and no one said "no, that won't work for us" so I stopped doing anything until someone answered and then it dropped off my radar. ;) Anyhow, I'd say look at that and see if there's any way to do what you want with existing macros, if not, we can revisit. I guess FPC and FESCo would want to sign off on it, and it would probibly need to get some devel list discussion :) kevin
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
