Re: CentOS are Debian / Ubuntu mirror

Valeri Galtsev <galtsev@xxxxxxxxxxxxxxxxx> · Tue, 20 Apr 2021 16:10:58 -0500

On 4/20/21 4:03 PM, Valeri Galtsev wrote:

On 4/20/21 3:53 PM, Rudi Ahlers wrote:
ok, so do you simply rsync the repositories from the other distributions
you need?

I forgot to mention: before mirroring using higher tier repository: 
contact their admin, ask if they do not mind that you will mirror using 
them as a higher tier mirror, and tell them all details: public/private 
mirror, your bandwidth, how often you plan to run sync - and ask them 
what frequency of sync is OK with them.

After you roll out and know all works, if your plan is public mirror, 
next step will be to register your mirror.

All in all, every distribution has HOWTO instruction to become their 
mirror, follow these.

And as always, use web search to find these instructions: duckduckgo is 
your friend.

Valeri

Short answer: yes.

Long answer:
Every repository has its own suggestion how to mirror. I do not have 
Debian mirror, my friend sysadmin in the next building maintains one, no 
need for two in the same institution. Here in my cron job script for 
mirroring CentOS (with some local details obfuscated):

#==============
#!/usr/local/bin/bash
# We use locks...

touch=/usr/bin/touch
echo=/bin/echo
ps=/bin/ps
cat=/bin/cat
grep=/usr/bin/grep
wc=/usr/bin/wc
rm=/bin/rm
sendmail=/usr/sbin/sendmail
date=/bin/date
rsync=/usr/local/bin/rsync
# DEFAULTS
ADDR=recipient@server
FROM=sender@hostingmirror
lockfile=/.../lock/centosmirror.lock
mirror=/.../mirror/centos
logfile=/.../mirrorlog/centos.log

# check if log file exists, and create if not

if [ ! -f $logfile ]; then
$touch $logfile
fi

# check if lock exists, then bale out sending warning,
# otherwise rsync

if [ -f $lockfile ]; then

# check if the process resembling lock still exists,
# if not, just delete lock

if [ "$($ps -p `$cat $lockfile` | $wc -l)" -gt 1 ]; then

# Send the message
(
$echo "From: $FROM"
$echo "To: $ADDR"
$echo "Subject: CentOS mirror failed"
$echo ""
$echo "CentOS mirror process bailed out as there is another process"
$echo "with the same name and the following PID:"
$echo ""
$echo "`$cat $lockfile`"
echo ""
echo "You may want to check logs:"
echo ""
echo "/.../mirrorlog/centos.log"
) | $sendmail -F $ADDR -t

else

# no process, just delete the lock, and mirror

$rm -f $lockfile

$echo $$ > $lockfile
$echo "`$date`  started mirroring" >> $logfile
$rsync -aqH --exclude .~tmp~/ --exclude .~tmp~ --delete 
us-msync.centos.org::CentOS /data/mirror/centos
$echo "`$date`  finished mirroring" >> $logfile
$rm $lockfile

fi

else

$echo $$ > $lockfile
$echo "`$date`  started mirroring" >> $logfile
$rsync -aqH --exclude .~tmp~/ --exclude .~tmp~ --delete 
us-msync.centos.org::CentOS /data/mirror/centos
$echo "`$date`  finished mirroring" >> $logfile
$rm $lockfile

fi
#================================

On Tue, Apr 20, 2021 at 3:14 PM Valeri Galtsev 
<galtsev@xxxxxxxxxxxxxxxxx>
wrote:

On Apr 20, 2021, at 12:34 AM, Rudi Ahlers <rudiahlers@xxxxxxxxx> wrote:

Pulp is a bit overkill for my liking.

Coming back to the original question: mirror hosting variety of
distributions, does not have to have any packaging or other tools 
used by
those distributions. Public mirror box I support runs FreeBSD, and hosts
mirrors of CentOS, Ubuntu, CPAN, EPEL, …

I hope, this helps.

Valeri

On Mon, Apr 19, 2021 at 2:44 PM Arnaud Gelly <arnaud.gelly@xxxxxxxxx>
wrote:

Hello,

Is Pulp not supposed to do what you want : RPM, DEB and more in the 
same
software ?

I'm not using Pulp, just reading their website.

Regards,
--

On Mon, 19 Apr 2021 at 14:26, Rudi Ahlers <rudiahlers@xxxxxxxxx> 
wrote:

Thank you. This at least point me in the right direction. I don't 
quite
want to setup 2 servers, or 2 VM's for this.

On Mon, Apr 19, 2021 at 2:03 PM Robert Heller <heller@xxxxxxxxxxxx>
wrote:

At Mon, 19 Apr 2021 08:55:54 +0200 CentOS mailing list <
centos@xxxxxxxxxx>
wrote:

Hi,

Does anyone have some instructions on setting up a CentOS server as
mirror
for Debian and Ubuntu distributions? I already setup a YUM 
mirror and
this
works fairly well, but cannot seem to figure out how to mirror 
Debian
and
Ubuntu repositories.

You need to grab a copy of debmirror, which is just a Perl 
script, so
it
should work under CentOS. I don't know where to get a version as a
tarball
--
I had an available VM running Ubuntu and just installed it there and
NFS
mounted the mirror disk from the CentOS server. There is a config 
file
for
debmirror to control where you mirror from and just what you mirror
(versions,
arches, etc.). There is a man page and and example config file.  You
then
run
debmirror from crontab (eg every day).

Unfortunately, the Debian flavor repositories are not structured to
just
be
rsync'ed like the CentOS repositories.

--
Robert Heller             -- Cell: 413-658-7953 GV: 978-633-5364
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/ ; -- Linux Administration Services
heller@xxxxxxxxxxxx       -- Webhosting Services

_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos

--
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos

_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos

--
Kind Regards
Rudi Ahlers
Website: http://www.rudiahlers.co.za
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos

_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos

--
++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos