On 07/11/2018 06:37 PM, Andrew Lutomirski wrote: > From my perspective as an occasional Fedora packager, I'm regularly > surprised by just how long it takes for Koji builders to install > dependencies. I've never tried to dig in too far, but it looks like the > builders download package metadata, download packages, and then install > things. Surely this could be massively optimized by having the metadata > pre-downloaded (at least when side tags aren't involved) and by having the > packages already present over NFS or similar. Koji gets repodata and packages from HTTP servers, through caching proxies located in the same datacenters as builders. Most often used packages are cached in memory, so download speeds are not a problem. At least for non-s390x builders. Accessing packages directly from NFS would be slower. The slowest parts of setting up chroot is writing packages to disk, synchronously. This part can be speeded up a lot by enabling nosync in site-defaults.cfg mock config on Koji builders, setting cache=unsafe on kvm buildvms, or both. These settings are safe because builders upload all results to hubs upon task completion. With these settings chroot setup can take about 30 seconds. Once this is optimized, another slow part is loading repodata into memory - uncompressing it, parsing and creating internal libsolv data structures. This could be speeded up by including solv/solvx files in repodata, but I think that would require some code changes. -- Mikolaj Izdebski Senior Software Engineer, Red Hat IRC: mizdebsk _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/message/3AG3QBIBQZ6S6P3BPDDSQQJ4D7C4Y74M/
