On Fri, Jun 26, 2015 at 11:04:35AM -0600, Stephen John Smoogen wrote: > On 26 June 2015 at 10:11, Pierre-Yves Chibon <pingou@xxxxxxxxxxxx> wrote: > > So the UMDL script currently supports three ways of crawling the tree: > > * file > > * rsync > > * directory > > > > We, in Fedora, are only using the last one. I believe the `rsync` mode was added > > to support Ubuntu and the file mode is basically a simplified version of the > > directory mode, but that we do not use at at the moment. > > > > I would like to propose that we drop support for rsync. I feel that it may be > > simpler and easier to create an UMDL and a crawler for each distro that would > > like to use MirrorManager than maintaining a one-script-fits-all UMDL that is > > in fact tested for only one of the scenario. > > That being said, if we ever have interest from Ubuntu, CentOS or any other > > communities, we should definitively look into making the UMDL and crawler as > > re-usable as possible for them, but keeping the distro-specific bits separated. > > > > I thought rsync was added for speed versus Ubuntu and that we actually > use it a lot on the crawlers... because that is the reason we needed > to move the memory usage up.. it uses a lot of memory but takes a lot > shorter period to 'crawl' or am I conflating two different things? That's the rsync of the crawler scanning the mirrors, that's much faster compared to http/ftp. umdl analyzing the master mirror can either use the local filesystem or rsync or a file as input. Adrian
Attachment:
pgpJUmLIADdMQ.pgp
Description: PGP signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
