Hello all, In my "spare time," I wrote a simple yum-plugin that uses the pycurl library to download the RPM packages as a predownload hook. The goal is to better use the client connection bandwidth by downloading multiple packages at a time. I looked through the history of this and the dev list, but only saw an attempt by someone hacking something into yum itself. This was shot down, with the suggestion to write it as a plugin and make use of pycurl to handle the downloads. I saw no follow-ups, so here I am. From a high-level, this plugin uses pycurl and downloads all the packages max_connections (16) at a time before the yum client processes downloads. Once all packages are downloaded, control is returned to yum, which sees that it has all the packages and begins the transactions as normal. Right now the plugin is still in its infancy, so I am looking for quick feedback. You can download the plugin from: http://dev.beyond-syntax.com/yum/multithread.py (for an example config file just replace .py with .conf). There are a few things I would like to fix up, but haven't figured out how: - Better output would be nice, but I'm not sure how to get there. - It should integrate with fastestmirror (since it orders the repos), but it would be nice to integrate with presto too, not sure how. - Reordering downloads, starting biggest ones first and letting smaller ones fill the download during that time. - Distribute the downloads over more than just one mirror (if not already). In my completely unscientific testing, I took a VM snapshot and ran yum without presto and specified '--downloadonly'. The update was 94 packages, totaling 102 Megabytes. (Both runs use the same mirror.) - Multithread plugin downloaded all packages in 34 seconds. - Normal yum downloaded all packages in 53 seconds. Obviously, this is strongly dependent on the connection and server capabilities, but I would assume any Fedora/RPM mirror will have bandwidth >> client bandwidth. By creating more connections, multithread consumes more bandwidth to speed up the overall throughput. Questions, comments, observations? Thanks, -- Michael J. Schultz mjschultz@xxxxxxxxx
Attachment:
pgpqLpVXYH6SQ.pgp
Description: PGP signature
_______________________________________________ Yum mailing list Yum@xxxxxxxxxxxxxxxxx http://lists.baseurl.org/mailman/listinfo/yum
