Matt Domsch wrote:
First, the problems with the current mirroring is really twofold:
( ... snip ... )
2) the global mirrors aren't being notified when content has changed on the master, such that they should start a new rsync run. mirrormanager takes a per-host email address, which the master sign-and-push scripts will eventually send an email to when the content changes. As it stands, syncing every 6 hours when nothing has changed doesn't make any sense. Now, mirrormanager has 2 methods by which it can know a give mirror host is up-to-date. First is a new report_mirrors script that uploads directory data back to the database from the mirror itself. Not everyone will want to run that, and there's always the 'trust but verify' model, so I've also got a (fast?) crawler that crawls each host using HTTP HEADs and keepalives or FTP DIR calls looking for content it should be carrying compared to the master list, and tracking presence and up-to-date-ness on a per-directory level. Those that aren't up2date get dropped from the appropriate per-directory lists (e.g. the repodata dirs) in real time.
Sounds good, but it still seems like we need special tools to determine on the fly, with standard tools, if a mirror is in a sync'ed state. If I need to create a local mirror of fedora X with updates, with standard tools, I have to figure out when to mirror by trial and error and have no way to check before the sync, if the mirror is in the middle of syncing itself.
The mirrormanager you're working on will be useful for monitoring and flagging stale mirrors. It could work well for yum too, but for the basic mirroring, we still lack a simple mechanism to allow us to track the state of upstream mirrors. Mirroring a few gigs of inconsistent data makes no sense at all and this can be implemented very very easily. It's all a matter of documenting "HOWTO setup a fedora mirror".
That's the idea. A lot of the code is implemented, there's more to go. If you're good with python, turbogears, and the like, I'm sure I could put you to work on it. Drop me a note.
Never worked with turbogears, sorry ;-) /Thomas -- fedora-devel-list mailing list fedora-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-devel-list
