[Centos] Re: How to make a mirror of CentOS sites?

[subsolar at subsolar.org (Paul)]

On Tue, 2005-03-22 at 16:23 +0100, Tru Huynh wrote:
> On Tue, Mar 22, 2005 at 10:42:00AM -0500, israel.garcia@xxxxxxxxxxxx wrote:
> 
> > Hi Ralph, thanks for your soon answer, but my access to internet is trough http
> > proxies, rsyn is not allowed. I think wget  can do this but I dont know how?
> > 
> use lftp(1)
> 
> lftp your_ftp_target <<EOF
> cd ...../centos
> mirror
> EOF
> 


The script I use is as follows:
#!/bin/sh
#
# mirror-centos.sh - Mirror CentOS Linux Updates
#

# Variables
MIRROR="mirror.cs.wisc.edu"
WBDIR="pub/mirrors/linux/caosity.org/centos"
VERSIONS="4"
ARCHS="SRPMS i386 x86_64"
WBUPDATES="/home/ftp/pub/centos"
PROXY="http://firewall.company.com:8080/";

# Limit Transfer Rate to 10K bytes per second
MAXRATE="10k"

# Do the mirror
for d in $VERSIONS; do
  for y in $ARCHS; do
    cd $WBUPDATES
    ## env http_proxy=$PROXY \
    wget --non-verbose --no-host-directories --cut-dirs=8 --mirror \
     --passive-ftp --limit-rate=$MAXRATE \
     --directory-prefix=$WBUPDATES/$d/updates/$y \
     ftp://$MIRROR/$WBDIR/$d/updates/$y 
     ##--output-file=/tmp/mirror-centos.log 

    # the RPMS should be checked for valid sigs, not ready for that yet
    ##rpm --checksig $WBUPDATES/$d/updates/$y/*.rpm | grep 'NOT OK'
  done
done


# Done
exit