On 11 February 2015 at 06:10, Pierre-Yves Chibon <pingou@xxxxxxxxxxxx> wrote:
Hi everyone,
We currently have MirrorManager2 running on staging. It's apparently not 100%
set-up since we get emails once in a while that one of the crons failed
(iirc among other we need to finish configuring fedmsg).
Other that this, MirrorManager2 is currently in a decent shape I think.
However, we really need to make sure nothing broke in the re-write and we
want to make sure we won't break it in the future.
To try to ensure that last part, I have try to write some tests for the UI but
also for the backend part (all the different scripts).
The pull-request is opened for review:
https://github.com/fedora-infra/mirrormanager2/pull/14
I have also been trying to capitalize on the knowledge we acquired during the
FAD by starting to write down how mirrormanager works in the documentation:
https://github.com/fedora-infra/mirrormanager2/compare/tests...doc
(pull-request to be opened once the tests branch is merged)
I would appreciate if those that were at the FAD could go through this
branch/changes and adjust as needed.
I have been thinking about asking Matt to do the review so that we can adjust
and improve the documentation.
Before we move MirrorManager2 to prod, here is what I think would be nice to
do/have:
- Pickle validation
- Figure a way to validate a pickle after its creation and before moving it to
the mirrorlist boxes
- Find out if we can improve our tests some more
(to improve our confidence that we're ready)
- Engage the mirror mailing list and try to get them to react on the coming
changes
The pickle validation might also be an interesting idea to check if there is a
difference between the pickle generated by prod and the one generated in stg.
Finally, at DevConf we have been speaking quite a bit with Dennis around
updates and MirrorManager and here is some of the ideas we spoke about:
- Be able to run the UMDL script on only a part of the tree
(ie: be able to say, we updated f21-updates and we only update this part)
- Crawl the mirrors for only a part of the tree
(This goes together with updating only part of the tree via UMDL)
For a dumb optimization (which may be there laready) I would only crawl the trees which we know change "hourly" (updates/X/, development/X/) and only scan releases etc daily or weekly because it should not change that much.
- Consider if we should/could drop the content of the host_category_dir table
before running the crawler
- Mirror versioning:
- run UMDL, detect changes, increase master mirror's version by 1
- run the crawler, check for the changes, align that mirror's version with the
master mirror one
- be able to see the difference between two versions
- be able to crawl a mirror only for the difference between the version it is
at and the version the master mirror is at
note: we might still want to run a full crawl once in a while (daily?
bi-daily?)
Those last couple of steps would be very useful in cutting down load from mirrors running rsync against enchillada but only needing a couple of packages in updates since the last time they did it.
This list of ideas is more a long term todo list, not something we would want to
have working for pushing MirrorManager2 to prod.
Thoughts? Agreements? Disagreements?
Thanks,
Pierre
_______________________________________________
infrastructure mailing list
infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Stephen J Smoogen.
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
