On Wed, Mar 17, 2010 at 02:46:35PM -0400, James Antill wrote: > Adrian Reber <adrian@xxxxxxxx> writes: > > > For RPM Fusion's mirrorlist I am also running Matt Domsch's > > MirrorManager and we have two servers serving the mirrorlist. It is one > > DNS name which resolves to two IPs. Yum automatically uses the correct > > mirrorlist server if one of the servers is down (or if nothing answers > > on port 80). > > As far as I can see there is no code to do this anywhere, for either > mirrorlist or metalink yum just has a single URL and requests that > from urlgrabber (and AFAIK there's nothing in urlgrabber to try all > IPs for a given host, but maybe there is). > > > I do not know if this is on purpose but if something is > > answering on the port the mirrorlist is supposed to run with a http > > status code 500 or 40? would it not also make sense to try another of > > the available mirrorlist servers? This is especially a problem for RPM > > Fusion because the mirrorlist infrastructure is not in as good hands as > > the Fedora infrastructure. > > Now, in theory, it'd be possible to change _getMetalink() to have a > list of URLs and try each in turn until we get "something that > parses". But it's not super easy, and it'd need a lot of testing > (... patches accepted :). I am not sure I formulated my problem correctly, so I have written a patch that does what I want. Maybe this helps to describe my problem better. To test it I have created a DNS entry (mirrors.lisas.de) which points to four IPs: $ host mirrors.lisas.de mirrors.lisas.de has address 129.143.116.10 mirrors.lisas.de has address 134.108.44.54 mirrors.lisas.de has address 88.198.59.188 mirrors.lisas.de has address 129.143.116.9 Two of those entries are working mirrorlists, one has a webserver which does know nothing about mirrorlist and returns a 404 (129.143.116.10) and one does not even have anything running on port 80 (129.143.116.9). I am using following repo file: [lisas] name=test mirrorlist=http://mirrors.lisas.de/mirrorlist?repo=free-fedora-$releasever&arch=$basearch enabled=1 metadata_expire=0 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmfusion-free-fedora-$releasever-$basearch With the attached patch it goes to the next mirrorlist server if it fails for any reason. It still has some debug code but I would like to have some feedback if that is something that would be acceptable? Adrian
--- yumRepo.py 2009-10-16 22:44:39.000000000 +0200
+++ /tmp/yumRepo.py 2010-03-18 13:32:20.156362252 +0100
@@ -49,6 +49,8 @@
import errno
import tempfile
+import socket
+
# If you want yum to _always_ check the MD .sqlite files then set this to
# False (this doesn't affect .xml files or .sqilte files derived from them).
# With this as True yum will only check when a new repomd.xml or
@@ -1626,6 +1628,9 @@
return (returnlist, content)
+ def _unique(self, list):
+ return {}.fromkeys(list).keys()
+
def _getMirrorList(self):
"""retrieve an up2date-style mirrorlist file from our mirrorlist url,
also save the file to the local repo dir and use that if cache expiry
@@ -1644,14 +1649,36 @@
url = 'file://' + self.mirrorlist_file # just to keep self._readMirrorList(fo,url) happy
else:
url = self.mirrorlist
- scheme = urlparse.urlparse(url)[0]
+ parsed_url = urlparse.urlparse(url)
+ scheme = parsed_url.scheme
+ hostname = parsed_url.hostname
+ hosts = []
+ for i in socket.getaddrinfo(hostname, None):
+ hosts.append(i[4][0])
+ hosts = sorted(self._unique(hosts))
+ print hosts
if scheme == '':
url = 'file://' + url
- try:
- fo = urlgrabber.grabber.urlopen(url, proxies=self.proxy_dict)
- except urlgrabber.grabber.URLGrabError, e:
- print "Could not retrieve mirrorlist %s error was\n%s: %s" % (url, e.args[0], misc.to_unicode(e.args[1]))
- fo = None
+ if (scheme == 'http') or (scheme == 'https'):
+ for host in hosts:
+ print "----------------> " + host
+ try:
+ _url = url
+ _url = _url.replace(hostname, host)
+ print _url
+ fo = urlgrabber.grabber.urlopen(_url, proxies=self.proxy_dict, http_headers = (('Host', hostname),))
+ break
+ except urlgrabber.grabber.URLGrabError, e:
+ if host != hosts[len(hosts) - 1]:
+ continue
+ print "Could not retrieve mirrorlist %s error was\n%s: %s" % (url, e.args[0], misc.to_unicode(e.args[1]))
+ fo = None
+ else:
+ try:
+ fo = urlgrabber.grabber.urlopen(url, proxies=self.proxy_dict)
+ except urlgrabber.grabber.URLGrabError, e:
+ print "Could not retrieve mirrorlist %s error was\n%s: %s" % (url, e.args[0], misc.to_unicode(e.args[1]))
+ fo = None
(returnlist, content) = self._readMirrorList(fo, url)
Attachment:
pgpnBKqbthz0O.pgp
Description: PGP signature
_______________________________________________ Yum mailing list Yum@xxxxxxxxxxxxxxxxx http://lists.baseurl.org/mailman/listinfo/yum
