On Sat, Jun 09, 2007 at 04:22:24AM -0400, Anthony Bryan wrote: > Hello, this is a repost of a message to fedora-devel, as this seems to > be the place for it. > > >From: Jesse Keating <jkeating@xxxxxxxxxx> > > > >On Friday 08 June 2007 14:24:47 Anthony Bryan wrote: > >> I was hoping Fedora could investigate using Metalinks for their ISO > >> downloads. Metalink is an XML format for listing all the ways you can > >> get a file or collection of files (mirrors + their location, rsync, > >> p2p) along with checksums to automatically repair a file in case of > >> error, signatures, language, OS/arch, and other metadata. It's mainly > >> used for large files like ISOs, where errors can be very frustrating. > >> > >> It's supported by about 20 programs on unix, mac, and win, including > >> aria2 (already in the Fedora repos). It's used by openSUSE, > >> OpenOffice.org, cURL, and many other distributions. > >> > >> Here's a screenshot of a Metalink download in the DownThemAll Firefox > >> extension (nightly build). What you don't see are all the mirrors and > >> checksums. > >> http://code.downthemall.net/maierman/metaselect4.png > >> > >> http://en.wikipedia.org/wiki/Metalink > > > >This is something interesting, and I wonder if we could make use of > >MirrorManager ( https://hosted.fedoraproject.org/projects/mirrormanager ) > >to > >have dynamic .metalink files created with updated mirror readiness info. > >Certainly something that looks worth looking into. > > That would be quite nice, no one else has dynamic .metalinks on a > large scale. When I got the F7 ISO, I noticed it would fit in well w/ > the download pages which tell which mirrors have which releases. I > think it would make things less frustrating for end users trying to > get things, and hopefully create less strain on mirrors. Certain > metalink clients will download from domestic mirrors first, if country > info is in there, which should hopefully be more efficient for > everyone. > > What can we do to make this happen? Is this the type of thing that's > easier for the maintainer of MirrorManager to add, or should we supply > a patch? I saw some articles on metalink and spent time time looking at it a few weeks ago. Good stuff. Patches are certainly welcome. What I picture is a new application 'generate_metalinks' along the lines of the generate_mirrorlists application within mirrormanager that connects to the database, finds what it wants, and generates static .metalink text files. You'd probably only want to metalink the .ISOs, of which there are plenty. No sense making .metalink files for every file in the (presently) 660GB file system. Then we also need to generate static web pages that include HTML links to all the .metalink files. And link to those from somewhere in the mirrors.fedoraproject.org/ namespace, maybe mirrors.fp.o/metalink/index.html plus all the .metalink files in a dir there. Then we add it into the update-static-content script that runs at the top of the hour, so they're dynamically generated. > Here's the current tools people have done, if that helps - > http://www.metalinker.org/implementation.html#generate > > Metalink Editor - in Python, GUI > cURL - they use a short Perl script that makes them based on location. > Simba/RoPkg::Metalink - Perl > Bouncer - there's a patch for it. > Metalink tools - CLI, C++ As mm is written in python + TurboGears, it would be great if the metalink creator tool was also in python. :-) Source to mm is available here: https://hosted.fedoraproject.org/projects/mirrormanager Thanks, Matt -- Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
