Hi Tony, My add rpm script has been significantly refactored to clean up some rpm selection problems in the case when there are several releases each of two or more different versions (i.e. poo-1.2-3, poo-1.2-4, poo1.3-5, poo-1.6-1, and poo-1.6-3) I tested it against invented rpms designed to be tricky, and made sure it worked. There is better message verbosity control now, and it will also run genhdlist automatically (provided anaconda and anaconda-runtime are installed, and yes, it checks for that too). If I can figure out how to decide if foobar-2.3-5 is older or younger than foobar-2.3-5.7.3 (i.e., work around RedHat's weird fetish for sticking distribution version numbers into utility release strings) then I'll fix that case (where the progam punts for the moment). Note that that family of problems applies also to things like 2.4.19-rc1 and 2.4.19-1. I should probably use hardlinking, falling back on cp if that tanks. cheers, .paul. #!/bin/bash # # addrpm Adds RPM(s) to a Redhat Tree # Copyright (C) 2002 Paul Knowles # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # # I don't believe in battling reality, so the tree structure # I use is like the tree pictured below since that is what the # RedHat ftp mirror sites use. In fact, that is very convenient # since the updating commands are just # $> cd $UPDATES/redhat-updates # $> ncftpget -FRT ftp://really.fast.server/mirror/linux/distributions/redhat-updates/7.3 # and all is magically updated. In fact, cron does this for me at 3 am. # # The Redhat 7.3 tree is set out like: #$BASE/redhat... #redhat #`-- 7.3 # `-- en # `-- os # `-- i386 # |-- RedHat # | |-- RPMS # | `-- base # |-- SRPMS # |-- dosutils # | |-- autoboot # | |-- fips15c # | | |-- restorrb # | | `-- source # | |-- fips20 # | | |-- restorrb # | | `-- source # | |-- fipsdocs # | `-- rawritewin # |-- images # | |-- de # | |-- es # | |-- fr # | |-- it # | `-- pxeboot # |-- doc # `-- megroup <--- Medium Energy Group specific RPMs # |-- i386 # `-- SRPMS # # and the updates tree looks like # #$BASE/redhat-updates... #redhat-updates #|-- 7.3 #| `-- en #| |-- megroup #| | |-- SRPMS #| | `-- i386 #| `-- os #| |-- SRPMS #| |-- athlon #| |-- i386 #| |-- i486 #| |-- i586 #| |-- i686 #| `-- noarch #|-- attic #| |-- SRPMS #| |-- athlon #| |-- i386 #| |-- i486 #| |-- i586 #| |-- i686 #| `-- noarch #`-- tosort # # This shell script has been very successful in sorting through all the # updates in 7.3/en/os/*/*.rpm, figuring out the latest in the bunch # and then replacing the RedHat/RPMS/*.rpm packages. Unlike RPM, # addrpm IGNORES the EPOCH flag (and does so in a consistent manner). # i.e., it does the ``right thing'' and will replace hdparm-4.1-2 # with hdparm-4.6-1 even though 4.1-2 has epoch 1, # and 4.6-1 has epoch (none). # As far as I can tell, RPM does version checking via # %{epoch}:%{version}:%{release} # which is utterly, totally, and completely braindamaged. # [using %{version}:%{release}:%{epoch} would be sane(er)]. # People who set "Epoch:" in their rpm spec files deserve # plagues of ravenous shrews infesting their nether regions. # # It will tank if the updates directory gets too many different # and ``uncomparable'' versions of %{name}. This is on my list # of things to fix. It is related to the problem of assholes releasing # foobar-2.3-1 and then foobar-2.3-1.2 instead of foobar-2.3-2. # Release numbers are just that, and shouldn't be used for anything else. # However, if you have 1.3-4, 1.3-5, 1.3-6, 1.4-0, and 1.4-10 all in # the same place, it will figure out that 1.4-10 is the best to take. # # command line options: # -d,-v : increase the verbosity level by one # default verbosity is 1, 2 will make you go to sleep # 3 and 4 are only useful for spotting errors in the script itself # -q : set verbosity to zero: i.e., say nothing, # always assume no for any questions asked (except when -a is used) # -n : do not run genhdlist after a successful update # -t : run in test mode: file move commands are printed instead of executed # : and genhdlist will never be run # -a : aggressive mode; assume "y" as response to all questions # # see the lines: # export putdir=/home/updates/redhat/7.3/en/os/i386/RedHat/RPMS/ # export extradir=/home/updates/redhat/7.3/en/os/i386/megroup/ # export getdir=/home/updates/redhat-updates/7.3/en/os/$u # export atticdir=/home/updates/redhat-updates/attic/$u # and the big for loop over SRPMS i386 etc # if your directory structure is different. # # cheers, # P. Knowles, August 5, 2002 # <Paul.Knowles@xxxxxxxx> ###################################################### # comparison constants # please don't change... export LT=0 export EQ=1 export GT=2 export PUNT=3 # print out messages based on debugging level # invoked via: mess level "message" mess() { if [ $1 -le $VERBOSITY ]; then echo $2 fi } # the comparison function # called with two strings of form a.b.c.d.e.f # first is the champion, second is the challenger # return $GT if $2 is ``above'' $1, i.e., 1.2.3 is above 1.2.2, and 2.19.22 is above 3.0.0 # return $PUNT if $1 and $2 have differing composition (punt value!) # return $EQ or $LT if $1 <= $2 # only generates messages at VERBOSITY level 4 valcomp() { mess 4 "valcomp: Comparing champion $1 and challenger $2" if [ $1 == $2 ]; then mess 4 "valcomp: Champion $1 and Challenger $2 are equal" return $EQ fi # bugger, we have to work declare -a champ declare -a chall champ=(`echo $1 | sed s/\\\./\ /g`) chall=(`echo $2 | sed s/\\\./\ /g`) mess 4 "valcomp: broke champ into ${champ[@]}" mess 4 "valcomp: broke chall into ${chall[@]}" if [ ${#champ[@]} -ne ${#chall[@]} ]; then # The strings don't have the same composition: there is no sane way out; punt # Ya hafta love that losetup-2.11n-12 is younger than losetup-2.11n-12.7.3 # Sometimes I just don't understand RedHat packaging {ill}logic # Why not just losetup-2.11n-13? What is acomplished by changing the # makeup of the release string? mess 4 "valcomp: Punting due to length comparison" return $PUNT fi local -i i=0 local -i j=${#champ[*]} mess 4 "valcomp: Comparing $j elements in succession" for (( i=0 ; i < j ; i++ )) do # either we have just number to compare # or we must use strings dostringcomp=0 mess 4 "valcomp: Element $i: challenger=${chall[$i]}, champion=${champ[$i]}" if [ ${chall[$i]} != `echo ${chall[$i]} | sed -e s/[[:alpha:]]//g` ]; then dostringcomp=$(( $dostringcomp + 1 )) fi if [ ${champ[$i]} != `echo ${champ[$i]} | sed -e s/[[:alpha:]]//g` ]; then dostringcomp=$(( $dostringcomp + 1 )) fi if [ "$dostringcomp" == "1" ]; then mess 4 "valcomp: Punting due to alphanumeric versus numeric" return $PUNT fi # # directly comparing ${champ[$i]} and ${chall[$i]} fails # some weird shell syntax problem I dont' yet understand? # use this intermediate step form poo=${chall[$i]} bar=${champ[$i]} if [ $dostringcomp -eq 2 ]; then # compare via strings if [ "$poo" \> "$bar" ]; then mess 4 "valcomp: strings found that ${chall[$i]} is greater than ${champ[$i]}" return $GT fi fi if [ $poo -gt $bar ]; then mess 4 "valcomp: numerically found ${chall[$i]} is greater than ${champ[$i]}" return $GT elif [ $(( ${chall[$i]} == ${champ[$i]} )) == "1" ]; then mess 4 "valcomp: at i=$i, ${chall[$i]} is the same as ${champ[$i]}" continue fi # we arrive here when ${chall[$i]} is less than ${champ[$i]} mess 4 "valcomp: ${chall[$i]} is less than as ${champ[$i]}" return $LT done # we should never get here: the initial comparison should catch this case! # at end, no seen difference mess 4 "valcomp: oops, Officially Impossible weird shit. Call a thaumaturge." return $PUNT } # # # doreplace() is called with the full pathways of the new file, # the old file, the attic directory, and the test flag # gives messages at level 4 and level 1 doreplace() { new=$1 old=$2 local test=$3 if [ $TEST == "n" ]; then if ! mv $old $atticdir/ ; then mess 1 "doreplace: mv $old $atticdir/ failed." exit 22 fi if ! cp $new `dirname $old`/ ; then mess 1 "doreplace: cp $new `dirname $old`/ failed." exit 22 fi else mess 4 "doreplace: testmode: mv $old $atticdir/" mess 4 "doreplace: testmode: cp $new `dirname $old`/" fi } # # called when the program gets too confused # ask for operator intervention directly # asks for help at level 1, does nothing at level 0 punt() { mess 1 "Oops: comparing " mess 1 " $2" mess 1 " and" mess 1 " $4" mess 1 "is too confusing for me: Should I replace $4 with $2" mess 1 "(default is no)?" ans="" if [ $AGGRESS = "y" ]; then ans="y" else if [ $VERBOSITY == "0" ]; then ans="n" else read -r -p y/N -n1 ans echo fi fi if [ $ans == "y" ]; then mess 1 "punt: O.k., upgrading to $1/$2" doreplace $1/$2 $3/$4 $TEST else mess 1 "punt: O.k., skipping $2" fi } #################################################################### # Done with function definitions PROG=`basename $0` # # The BASE directory is machine dependent # I wrote this to run on any machine under my control # but only make changes on those where it is necessary... IAM=`hostname` unset BASE case "$IAM" in "pine.acq.megrp") BASE=/nfs/data/updates ;; "birch.acq.megrp" | "mercurey.acq.megrp" ) BASE=/home/updates ;; *) mess 1 "$PROG: $IAM is not an update server." exit 20 ;; esac # parse command line arguments # here are the defaults AGGRESS="n" VERBOSITY=1 REGEN_HDLIST="y" TEST="n" for op in $@ do case "$op" in "-a") AGGRESS="y" mess 3 "$PROG: AGGRESS now $AGGRESS" ;; "-d" | "-v" ) VERBOSITY=$((VERBOSITY + 1)) mess 3 "$PROG: VERBOSITY now at $VERBOSITY" ;; "-n") REGEN_HDLIST="n" mess 3 "$PROG: REGEN_HDLIST now $REGEN_HDLIST" ;; "-q") VERBOSITY=0 mess 3 "$PROG: VERBOSITY now at $VERBOSITY" ;; "-t") TEST="y" mess 3 "$PROG: Test mode is $TEST" ;; *) mess 3 "$PROG: unknown option $op" ;; esac done export TEST VERBOSITY REGEN_HDLIST AGGRESS mess 1 "$PROG: Working on tree in $BASE" mess 2 "$PROG: Test mode is $TEST" mess 2 "$PROG: VERBOSITY set to $VERBOSITY" mess 2 "$PROG: AGGRESS set to $AGGRESS" mess 2 "$PROG: REGEN_HDLIST set to $REGEN_HDLIST" # # start work on the actual updates # for u in noarch SRPMS i386 i486 i586 i686 athlon do # where the updates come from export getdir=$BASE/redhat-updates/7.3/en/os/$u # where the main RedHat/RPMS directory is export putdir=$BASE/redhat/7.3/en/os/i386/RedHat/RPMS/ # your own supplementary rpm dir on the RH disk export extradir=$BASE/redhat/7.3/en/os/i386/megroup/ # where to put the old rpms export atticdir=$BASE/redhat-updates/attic/$u # export getdir=/usr/src/redhat/RPMS/$u # export putdir=/tmp/ # export extradir=/tmp/extra # export atticdir=/tmp/poo/$u mess 2 "getdir=$getdir" mess 2 "putdir=$putdir" mess 2 "extradir=$extradir" mess 2 "atticdir=$atticdir" tag=$u if [ $u == "SRPMS" ]; then tag=src putdir=$BASE/redhat/7.3/en/os/i386/SRPMS fi # # these are the possible replacement package names # there may be more than one replacement available # for each name: we need to find the latest version # # note that rpm doesn't separate stdout and stderr... list f can # contain crap. sorry. If the script fails with # messages about files with names like "read manifest failed: Success" # then this is what has happened. Blame RedHat RPM, not me. # for f in `rpm -qp --qf '%{name} \n' $getdir/*.rpm | sort | uniq` do mess 1 "Working on name $u/$f" # Get the current version and release # This will tank if there are non-rpm files in f # as well as when you have two versions of %{name} in putdir # (which is a big error anyway) vcurr=`rpm -qp --qf '%{version}' $putdir/$f-[0123456789]*$tag.rpm` rcurr=`rpm -qp --qf '%{release}' $putdir/$f-[0123456789]*$tag.rpm` currtest=`ls -alf $putdir/$f-$vcurr-$rcurr.$tag.rpm 2> /dev/null` mess 3 "Current version should be $f-$vcurr-$rcurr.$tag.rpm" if [ -z "$currtest" ]; then mess 3 "File $f-$vcurr-$rcurr.$tag.rpm doesn't exist in $putdir" else mess 3 "Found current: $currtest" fi # the candidate replacements declare -a vlist declare -a rlist # # rpm doesn't separate error messages and regular output # so scuttling errors to /dev/null wont work here # If one of the .rpm files isn't a real rpm # the script will get very confused # since weird things will show up in the version and release lists vlist=(`rpm -qp --qf '%{version} ' $getdir/$f-[0123456789]*$tag.rpm`) rlist=(`rpm -qp --qf '%{release} ' $getdir/$f-[0123456789]*$tag.rpm`) mess 3 "Candidate versions: ${vlist[*]}" mess 3 "Candidate releases: ${rlist[*]}" tst=$((${#vlist[*]}+${#rlist[*]})) vi=0 ri=0 if [ $tst -gt 2 ]; then # loop over the versions first # need to find the biggest version and release of the bunch # set i to the index # # use vi as sentinal, eventually to hold the best version index # use sa as number of files with the same version declare -i vi=-1 declare -i i=1 declare -i j=${#vlist[*]} declare -i sa=1 declare -i verdex=0 mess 2 "Sorting out the $j possible replacements via version numbers" mess 3 "Using version ${vlist[$verdex]} as best first guess" for (( i=1 ; $(( i < j)) ; i++ )) do mess 3 "Testing i=$i, ${vlist[$i]}" valcomp ${vlist[$verdex]} ${vlist[$i]} ret=$? case "$ret" in "$GT") mess 3 "Received that ${vlist[$i]} is bigger than ${vlist[$verdex]}" verdex=$i reldex=$i vi=$i sa=1 ;; "$EQ") sa=$(( $sa + 1 )) mess 3 "Received that ${vlist[$i]} is the same as ${vlist[$verdex]}" mess 3 "There are now $sa files of this version: ${vlist[$i]} " ;; "$PUNT") mess 1 "ERROR: $u:$f is too hard a case for me" break 2 # give up and get another file ;; *) mess 4 "Received return value = $ret causing no action" ;; esac done mess 2 "Best candidate version number is ${vlist[$verdex]}, with $sa files of this version" intver=${vlist[$verdex]} # the above can end with vi =-1, sa>1 : we have several files all with the same version # the above can end with vi!=-1, sa=1 : we have several files but one with a winning version # the above can end with vi!=-1, sa>1 : we have several files and with multiple versions # and of the highest version there are several files too # in the first and third case we need to check release numbers if [ $sa != "1" ]; then # we have a case of identical versions # rebuild the lists list for the best version number found above # we can do this because $intver is holding the best version string now vlist=(`rpm -qp --qf '%{version} ' $getdir/$f-$intver*$tag.rpm`) rlist=(`rpm -qp --qf '%{release} ' $getdir/$f-$intver*$tag.rpm`) declare -i i=1 declare -i j=${#rlist[*]} declare -i reldex=0 mess 2 "Checking the $sa copies of version $intver" for poo in $getdir/$f-$intver*$tag.rpm; do mess 3 " file: $poo" done mess 3 "Using release ${rlist[$reldex]} as best first guess" for (( i=1 ; $(( i < j)) ; i++ )) do mess 3 "Testing release ${rlist[$i]}" valcomp ${rlist[$reldex]} ${rlist[$i]} ret=$? case "$ret" in "$GT") mess 3 "Found ${rlist[$i]} bigger than ${rlist[$reldex]}" reldex=$i verdex=$i ;; "$PUNT") mess 1 "ERROR: This is too hard a case for me" break 2 # give up and get another file ;; *) mess 3 "Found ${rlist[$i]} less than or equal to ${rlist[$reldex]} " ;; esac done fi vi=$verdex ri=$reldex fi vcandidate=${vlist[$vi]} rcandidate=${rlist[$ri]} mess 2 "All candidates sorted: $vcandidate-$rcandidate is best" # # see if candidate should replace curr # three cases # - curr doesn't exists: ask user if we add the rpm # (and where: $putdir or $extradir) # - curr needs replacing: # move curr to atticdir then # cp candidate into place # - curr doesn't need replacing: do nothing # # Dealing with the case that curr is missing # is a bit of a kludge since it doesn't # handle existing rpm's in $extradir very gracefully. # You probably want to hand tune extradir in any case # if [ ! -f $putdir/$f-$vcurr-$rcurr.$tag.rpm ]; then if [ -f $extradir/$f-$vcandidate-$rcandidate.$tag.rpm ]; then mess 1 "$f-$vcandidate-$rcandidate.$tag.rpm exists already in $extradir" continue fi moved="" mess 1 "Oops: no version of $f exists in $tag flavour in $putdir" mess 1 "Shall I add the update it to $putdir (default is no)?" if [ $AGGRESS = "y" ]; then ans="y" else if [ $VERBOSITY == "0" ]; then ans="n" else read -r -p y/N -n1 ans echo fi fi if [ $ans == "y" ]; then ans="" moved=1 mess 1 "O.k., Adding $f-$vcandidate-$rcandidate.$tag.rpm" if [ $TEST == "n" ]; then if ! cp $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $putdir/ ; then mess 0 "Copy failed: $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $putdir/" exit 22 fi else mess 1 "cp $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $putdir/" fi # # finished with name $f, get another name now mess 4 "About to continue from putdir choice" continue else mess 1 "Shall I add it to $extradir (default is no)?" # you will never get here in aggressive mode if [ $VERBOSITY == "0" ]; then ans="n" else read -r -p y/N -n1 ans echo fi if [ $ans == y ]; then ans="" moved=1 mess 1 "O.k., Adding $f-$vcandidate-$rcandidate.$tag.rpm" if [ $TEST == "n" ]; then if ! cp $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $extradir/ ; then mess 0 "Copy failed: $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $extradir/" exit 22 else echo cp $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $extradir/ fi fi fi if [ -z $moved ]; then mess 1 "O.k., skipping $f-$vcandidate-$rcandidate.$tag.rpm" fi # finished with name $f, get another name now mess 4 "About to continue from extradir choice" continue fi # # because of the continue above, we know curr and candidate both exist else mess 3 "Present included version is $vcurr-$rcurr" valcomp $vcurr $vcandidate ret=$? case "$ret" in "$GT") mess 2 "$f-$vcandidate-$rcandidate.$tag.rpm replaces $f-$vcurr-$rcurr.$tag.rpm" doreplace $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $putdir/$f-$vcurr-$rcurr.$tag.rpm $TEST continue ;; "$PUNT") punt $getdir $f-$vcandidate-$rcandidate.$tag.rpm $putdir $f-$vcurr-$rcurr.$tag.rpm continue ;; "$LT") # less than, ignore mess 1 "This probably shouldn't have happened" mess 1 " current version is $u/$f-$vcurr-$rcurr.$tag.rpm" mess 1 " which is better than any update version available" mess 1 " (best is: $f-$vcandidate-$rcandidate.$tag.rpm)" mess 1 " Check your repository for corruption please..." continue ;; "$EQ") # if we got here, $ret=$EQ, so versions are the same # compare releases valcomp $rcurr $rcandidate rret=$? case "$rret" in "$GT") mess 1 "$f-$vcandidate-$rcandidate.$tag.rpm replaces $f-$vcurr-$rcurr.$tag.rpm" doreplace $getdir/$f-$vcandidate-$rcandidate.$tag.rpm $putdir/$f-$vcurr-$rcurr.$tag.rpm $TEST continue ;; "$PUNT") punt $getdir $f-$vcandidate-$rcandidate.$tag.rpm $putdir $f-$vcurr-$rcurr.$tag.rpm continue ;; "$EQ") # equal, nothing to do mess 2 "Is already included: $u/$f-$vcurr-$rcurr.$tag.rpm " continue ;; "$LT") # less than, ignore mess 1 "This probably shouldn't have happened" mess 1 " current version is $u/$f-$vcurr-$rcurr.$tag.rpm" mess 1 " which is better than any update version available" mess 1 " (best is: $f-$vcandidate-$rcandidate.$tag.rpm)" mess 1 " Check your repository for corruption please..." continue ;; esac ;; esac fi done # for f in namelist done # for u in SRPMS i386 noarch # # rebuild the hdlist file # if [ $REGEN_HDLIST == "y" -a $TEST == "n" ]; then if ! rpm -q anaconda 2>&1 > /dev/null; then mess 1 "anaconda not installed, unable to rebuild hdlist" exit 21 fi if ! rpm -q anaconda-runtime 2>&1 > /dev/null; then mess 1 "anaconda-runtime not installed, unable to rebuild hdlist" exit 21 fi mess 1 "Rebuilding hdlist file" export RHROOT=$BASE/redhat/7.3/en/os/i386/ export PYTHONPATH=/usr/lib/anaconda export PATH="$PATH:/usr/lib/anaconda-runtime" genhdlist $RHROOT exval=$? if [ ! $exval == "0" ]; then mess 1 "genhdisk failed with exit value $exval" exit $exval fi fi exit 0
