On 02/25/2015 04:46 PM, Kevin Fenzi wrote:
On Wed, 25 Feb 2015 16:29:14 +0100
Ralf Corsepius <rc040203@xxxxxxxxxx> wrote:
If so, then you have cached metadata thats older than a few days.
No.
If not, then it's indeed those mirrors that are out of sync.
These mirrors are out of sync. Whether you like it or not,
mirrormanager is dysfunctional.
That is not very helpfull.
The user-side of this problem is quite simple:
The metalinks/mirrorlists as being used in yum or mock configs are
pointing to mirrors, which are broken or out of sync. I.e. the
metalinks/mirrorlist are incorrect.
The problem is that managing mirrors is not a simple problem,
I don't know how fedora mirror works, esp. whether they are polling or
whether they served with pushes.
In another (much smaller) project, when pushing updates, we had removed
all mirrors from our mirrorlists and had let the server poll
"repodata/repomd.xml" from the mirrors to re-adde a mirror to the
mirrorlists if this file matched. Of course, this is a primitive
heuristic, but it had worked quite well for us.
It is a provable matter of fact that it points users (yum, mock, ...)
to broken and out of sync mirrors.
There will be such times, sure.
Here (Germany), in recent times, they happen almost daily. My guess is
the "fedora flavors", the launch of f22 and the mass-rebuild on rawhide
are showing their nasty side.
I don't see any way of eliminating
that. We can reduce it as much as we can with the resources we have.
IMO, the issues yum/mock/dnf have, partially need to be worked-around by
heuristics in yum/mock/dnf.
I sometimes observe yum and mock seemingly to lock up to dead mirrors
(downloaded metadata is older than previous one) and not to try a more
recent mirror.
I also occasionally see consecutive yum/mock runs to re-iterate and fail
over apparently the same mirrorlists (I feel, it retries by priority and
therefore always stumbles of the same broken/dead mirrors).
Having a past of scientific work on Genetic Algorithms, my first try
would be to introduce some "randomness" into (mirror) selection - It
would help to breakout of deterministic causes :-)
There are some things we can do:
* If there's a mirror that is reporting that it's up to date, but is
not, we can remove it from the list and ask mirror admins to check
it.
* In mirrorlists and metalinks we provide a list of mirrors. If some
are out of sync, yum/dnf should move on to the next and retry.
My feel is, this currently doesn't work. The worst cases seem to be yum
alternating between several high-priorized broken/dead mirrors.
It
sounds like you might see some cases where your metadata is updated
locally and all mirrors fail? Please report such bugs.
https://fedorahosted.org/mirrormanager/
Yes, this had happened for a longer period (>24 hours) some time last
week (IIRC, Thursday) and for a shorter period (2-4 hours) yesterday.
My guess is, all EU f22/rawhide mirrors were out of sync during these times.
* We can try and urge more mirrors to sync based on fedmsg (using
last-sync) instead of just randomly N times a day. They would then
reduce load on master mirrors and get content faster:
https://fedoraproject.org/wiki/Infrastructure/Mirroring#Mirror_Frequency
* We can finish deploying our mirrormanager2 re-write. It doesn't add
many/any new features, but it moves mirrormanager to a codebase we
can work on more easily and bugfix/add features to.
https://github.com/fedora-infra/mirrormanager2
Sorry, lack of knowledge on these details :(
Constructive bugs, ideas or patches welcome.
In this case, I am mostly a "p***ed user", who is struggling ;)
Ralf
--
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
