So, I had an idea of one method of attack for composes and lifecycles that I thought I would toss out there to see what others thought of it. Instead of redesigning and reimplementing our current build/compose/test pipeline we improve it by making the current process cache as much of everything as it possibly can and listening and operating on any changes in inputs. So, instead of gathering everything all the time from all the inputs every time we only do 'full' composes every so often (weekly?) and then less and less often as our caching is fixed. At each place we currently gather or query an input, we instead query it and check it against what we already have. If they are the same, go on, if they are different, cache it and change only those things that are affected by that input. Some examples: a new xfce4-session package is built. The composer sees this and goes through all things affected by 'new package': * Does it pass CI? * remove old package from package caches/repos and update them. * is it in a deliverable? yes, a Xfce live and a xfce arm image * make those (just those) and test * If everything passes, release those deliverables (a updated repo where you just have the one package updated and new repodata, some images and checksum files) A new kernel is built: * Does it pass CI? * remove old package from package caches/repos and update them. * is it in a deliverable? yes, almost all of them. * build and test each deliverable in order of importance. * If they all pass, release those deliverables (new repos, new images, new checksums, etc). This of course really only works for rawhide, and in practice (at least at first) we would swamp it pretty easily (ie, kernel and coreutils and dnf update nearly at the same time could result in tons of activity), but it could be nice down the road as it would mean we are always ready for a release anytime we choose to do so. There will of course be cases of caches not updating, or updating, but not building all the affected outputs. It would also require a lot of mapping of input to outputs, but almost any approach we have will need that. Thoughts? or better yet: Other concrete ideas how to approach this re-tooling? kevin
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
