On Fri, Jun 05, 2015 at 05:09:06PM +0200, Pierre-Yves Chibon wrote: > Good morning everyone, > > Here is another release of the day, MirrorManager2: 0.2.0 > > And here is the changelog: > * Fri Jun 05 2015 Pierre-Yves Chibon <pingou@xxxxxxxxxxxx> - 0.2.0-1 > - Update to 0.2.0 > - Include the background header file in MM2 itself (Adrian Reber) > - Support always update hosts which are unreachable in the crawler (Adrian > Reber) > - Adjust the spec file to the systemd packaging guidelines for Fedora > - Multiple improvements to the crawler, including a start of a canary mode > (Adrian Reber) > - Offer possibility to sort by product, bringing back MM1 behavior (Adrian > Reber) > - Couple of UI fixes about who is allowed to access what > - Fix peer ASNs (in the same spirit, who can access) > - Create noauthed master for mirror publiclist so that it can be cached in > memcachd (Patrick Uiterwijk) I think this should read varnish and not memcachd. I think it still uses the 'default' cache age of 120 seconds. The publiclist used to be created once or twice a day so that we could easily keep the cache much longer than the current 120 seconds. I think 12 hours would be no problem, as this list is 'only' used as an overview and independent of the results of the actual yum/dny mirrorlist/metalink. > - Fix the report_mirror to correctly catch the xmlrpclib.ProtocolError > - Add a new utility script to upgrade repo from -alpha or -beta to release > - Adjust the logrotate configuration to fix the permission denied error > - Create 2 API endpoints, one for zodbot's .mirroradmin and one for nagios > > Note: this is currently only in *staging*, we did not feel like pushing this to > prod on a Friday afternoon :) > > We'll likely push on Monday though, assuming nothing burns down over the > week-end. I tested the new canary and repodata crawl mode and I was able to crawl all mirrors in a much short time: $ time mm2_crawler --repodata -c ~/mirrormanager2.cfg --threads 75 --timeout 30 real 36m38.639s user 27m20.689s sys 3m13.829s $ time mm2_crawler --canary -c ~/mirrormanager2.cfg --threads 75 --timeout 30 real 6m33.624s user 0m16.061s sys 0m2.279s Canary mode only checks if at least one base URL of all categories on the mirror works. If no URL works the whole mirror is marked as not being update. This also means that the mirror will not be re-enabled until a full crawl or successful report_mirror run. This behaviour that canary mode disables the mirror until a full crawl or report_mirror run could be changed but would require database changes and therefore we decided to support this form of canary mode as a first step. Repodata mode crawls all repodata directories and only marks those directories as being up to date or not. A mirror will not be completely disabled in repodata mode in contrast to canary mode. As canary mode seems to finish in under 10 minutes we can now think about using full crawls and canary or repodata mode in combination to get better overview about the availability and the status of the mirrors. Adrian
Attachment:
pgpaMp8GeP4Vu.pgp
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
