On Thu, Jan 28, 2016 at 08:41:19AM -0500, Matthew Miller wrote: > I'm getting 404 errors from several places to which mirrormanager wants > to send me to get > https://download.fedoraproject.org/pub/alt/atomic/stable/Cloud-Images/x86_64/Images/Fedora-Cloud-Atomic-23-20160127.x86_64.qcow2 > (Specifically, mirrors.rit.edu and mirrors.kernel.org.) Presumbably, > these just haven't synced yet. But doesn't mirrormanager get feedback > about when mirrors are fresh? I can try to give a few reasons (excuses) why you were redirected to a wrong file. * The file does not exist anymore on the master mirror. Not being 100% sure how the file names are constructed but it seems to include a date and the file on the master mirror (and a few other mirrors includes a '.2'). So it seems like the file was recreated. Using the new file name I am redirected to a mirror which has the file. * /pub/alt (also known as the MM category 'Fedora Other') on the master mirror is only scanned once per day (12:15 UTC). As the scanning takes a very long time and as it creates a high I/O load the less important MM categories like 'Fedora Other' and 'Fedora Archive' are only scanned once a day. * mirrors.rit.edu and mirrors.kernel.org are both running report_mirror to update the status of their mirror after the sync. We trust mirrors running report_mirror. We only get the information that they claim the transmitted directories are up to date. The actual content is not checked when using report_mirror. In the current case both mirrors have been crawled by since the last report_mirror, but at the time of the crawl the file 'Fedora-Cloud-Atomic-23-20160127.2.x86_64.qcow2' (with the '.2') was not yet part of the database as it has a time of 'Jan 27 17:29' which is after the previous master mirror scan. By now the new file should be part of the database and soon the crawler should detect it also on the mirrors. So there are many reasons why it did fail but the main reason probably is, that due to large amount of files it is difficult to have real-time view of what is on the mirrors and the master mirror. Finding the right balance between massive amounts of I/O on the master and mirrors and freshness of the MM database is the hard part. Adrian
Attachment:
pgpx2Djodvl86.pgp
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
