On Wed, 11 Feb 2015 14:10:01 +0100 Pierre-Yves Chibon <pingou@xxxxxxxxxxxx> wrote: > Hi everyone, > > We currently have MirrorManager2 running on staging. It's apparently > not 100% set-up since we get emails once in a while that one of the > crons failed (iirc among other we need to finish configuring fedmsg). Yeah, there's two crons that are failing due to lack of fedmsg. :( > Other that this, MirrorManager2 is currently in a decent shape I > think. However, we really need to make sure nothing broke in the > re-write and we want to make sure we won't break it in the future. > To try to ensure that last part, I have try to write some tests for > the UI but also for the backend part (all the different scripts). > The pull-request is opened for review: > https://github.com/fedora-infra/mirrormanager2/pull/14 Cool. > I have also been trying to capitalize on the knowledge we acquired > during the FAD by starting to write down how mirrormanager works in > the documentation: > https://github.com/fedora-infra/mirrormanager2/compare/tests...doc > (pull-request to be opened once the tests branch is merged) I would > appreciate if those that were at the FAD could go through this > branch/changes and adjust as needed. I have been thinking about > asking Matt to do the review so that we can adjust and improve the > documentation. That is an excellent idea. ;) The docs all look good/accurate to me. ;) > Before we move MirrorManager2 to prod, here is what I think would be > nice to do/have: > > - Pickle validation > - Figure a way to validate a pickle after its creation and before > moving it to the mirrorlist boxes If it would help that we have tracebacks from the mm2 mirrorlist for when a bad pkl is loaded (the mm1 ones just fail, but the mm2 one does provide a traceback): Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: Traceback (most recent call last): Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: File "/usr/share/mirrormanager2/mirrorlist_server.py", line 877, in handle Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: r = do_mirrorlist(d) Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: File "/usr/share/mirrormanager2/mirrorlist_server.py", line 718, in do_mirrorlist Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: allhosts, cache, file, pathIsDirectory=pathIsDirectory) Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: File "/usr/share/mirrormanager2/mirrorlist_server.py", line 423, in append_path Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: s = hcurl_cache[hcurl_id] Feb 9 10:11:42 mirrorlist-host1plus python2[25890]: KeyError: 7716 > - Find out if we can improve our tests some more > (to improve our confidence that we're ready) > - Engage the mirror mailing list and try to get them to react on the > coming changes Good idea. > The pickle validation might also be an interesting idea to check if > there is a difference between the pickle generated by prod and the > one generated in stg. > > > Finally, at DevConf we have been speaking quite a bit with Dennis > around updates and MirrorManager and here is some of the ideas we > spoke about: > - Be able to run the UMDL script on only a part of the tree > (ie: be able to say, we updated f21-updates and we only update this > part) > - Crawl the mirrors for only a part of the tree > (This goes together with updating only part of the tree via UMDL) > - Consider if we should/could drop the content of the > host_category_dir table before running the crawler > - Mirror versioning: > - run UMDL, detect changes, increase master mirror's version by 1 > - run the crawler, check for the changes, align that mirror's > version with the master mirror one > - be able to see the difference between two versions > - be able to crawl a mirror only for the difference between the > version it is at and the version the master mirror is at > note: we might still want to run a full crawl once in a while > (daily? bi-daily?) > > This list of ideas is more a long term todo list, not something we > would want to have working for pushing MirrorManager2 to prod. > > > Thoughts? Agreements? Disagreements? Sounds good. There's some minor changes to make in mirrorlist rpms, but once thats done we can look at replacing the other mirrorlists anytime. For the other parts, I think we will need to just create them all in production and run them in parallel to the mm1 setup for a short time until we are ready to cut over. kevin
Attachment:
pgp906ulwwk8h.pgp
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
