Re: No more deltarpms by default

Reindl Harald <h.reindl@xxxxxxxxxxxxx> · Mon, 20 Oct 2014 10:22:36 +0200

Am 20.10.2014 um 04:02 schrieb Nico Kadel-Garcia:
On Sun, Oct 19, 2014 at 5:26 AM, Reindl Harald <h.reindl@xxxxxxxxxxxxx> wrote:

Am 19.10.2014 um 06:37 schrieb Nico Kadel-Garcia:

On Sat, Oct 18, 2014 at 8:28 PM, Reindl Harald <h.reindl@xxxxxxxxxxxxx>
wrote:

#!/usr/bin/bash
basearch=`uname -i`
releasever=`rpm -q --qf "%{version}\n" fedora-release`
for g in `ls -1b /var/cache/yum`
do
   if [ -d /var/cache/yum/$g/packages ]
   then
    echo "/var/cache/yum/$g/packages/ > /repo/cache/fc$releasever/"
    sudo mv --verbose /var/cache/yum/$g/packages/*.rpm
/repo/cache/fc$releasever/ 2> /dev/null
   fi
done
/buildserver/repo-create.sh

And that only works on one machine, and is sensitive to exclusions
written into yum.conf or /etc/yum.repos.d/*, etc., etc., etc. You also
forgot to use 'gpgcheck' to verify the contents of the already
downloaded RPM, since 'repsync' does not track successful versus
partial downloads without it.

Anyway. Now multiply your 'reposync' based tool by 200 thin
provisioned virtual machines, and you're chewing up quite a lot of
disk space doing this

you don't get it

* that single machine has all used packages installed
* that single machine builds a repo below /repo/cache/fc$releasever/
* that repo don't mirror blindly anything, just used RPM's
* that RPMs are the result *after* apply deltarpm out of yum cache

I get it

no

It starts with the the fact that your script has not
verified *any* of your RPM's, and reposync does not ordinarily verify
file contents unless you activate GPG verification. So any partial
transfer or interrupted transfer will effectively corrupted your local
repo, especially if anything else is dependent on that package. Do
take a look at the github scripts I pointed out, they do a more
thorough job.

what are you talking about reposync all the time?

frankly i have posted the whole source above
"/buildserver/repo-create.sh" can be read as "createrepo && chown && 
chmod && rsync-to-failover maschine on the cluster"

*/var/cache/yum/* is the source *after* ran updates on that machine 
ordinary with yum - guess what - after that suceeded the RPM's GPG 
verified and *that* is what i try to explain the whole time - yum don't 
need to know about deltarpm and so other scripts using the yum-cache 
don't too

And running 'reposync" still downloads the upstream repodata. You've
really saved very little bandwidth as long as you keep doing this, and
you *need* to keep downloading the bulky repodata to ensure getting
the latest releases of packages. Yum repodata itself has no 'deltarps'
for its own information, it's compressed binary database blobs.

Last, doing this on an individual machine does not scale You're
chewing up disk space on each machine, and bandwitdh, with all the
required 'repodata' downloads

WTF - no single other machine has *any* repo in the WAN configured

Attachment:
signature.asc

Description: OpenPGP digital signature
-- 
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxxx
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct