On 8 Aug 2003, Peter Peltonen wrote: > --<snip>-- > i386/XFree86-Mach64-3.3.6-29.i386.rpm is uptodate > i386/XFree86-Mono-3.3.6-29.i386.rpm is uptodate > i386/XFree86-75dpi-fonts-3.3.6-29.i386.rpm > i386/XFree86-8514-3.3.6-29.i386.rpm > i386/XFree86-S3V-3.3.6-29.i386.rpm is uptodate > i386/XFree86-AGX-3.3.6-29.i386.rpm > i386/XFree86-FBDev-3.3.6-29.i386.rpm > i386/XFree86-Mach8-3.3.6-29.i386.rpm > i386/XFree86-VGA16-3.3.6-29.i386.rpm is uptodate > --</snip>-- > > How come i386/XFree86-AGX-3.3.6-29.i386.rpm is not uptodate? This seems > to happen always with the same files. I'm clueless, but here are some debugging questions in case you haven't already looked or tried them. Is there anything odd about the stat data of these files? Do the timestamps, ownerships, and perms all coincide with those on the server (as far as you can tell) and are they the same before and after? What happens if you e.g. mv i386/XFree86-AGX-3.3.6-29.i386.rpm /tmp (wait overnight for the cron script to run) ls -al i386/XFree86-AGX-3.3.6-29.i386.rpm /tmp/XFree86-AGX-3.3.6-29.i386.rpm diff i386/XFree86-AGX-3.3.6-29.i386.rpm /tmp/XFree86-AGX-3.3.6-29.i386.rpm looking for timestamp changes and/or diffs. If the first night reveals nothing I'd add -c to your transfer arguments for that night as well -- a checksum comparison should give you a separate/direct observation of what is "different", maybe. I suggest this because one possible explanation is server side stuff beyond your control -- something that touches these files on the server and continually alters e.g. their timestamp or makes some sort of change that rsync interprets as the files not being identical, so it updates them. It is quite fussy -- I don't know for sure what it's criterion for update is, but it seems to involve most of the stat data except e.g. access times. This isn't impossibly implausible. Perhaps the server side people are using a nightly cron script to synchronize their repository with "something", maybe a mounted CD, maybe somebody else's site, and are using plain old rcp or scp or cp without a -p so that the timestamp changes every night for certain files. > If I run the cronjob manually the behaviour does not occur. Note that a server site bug would probably exhibit this behavior. Their update (that re-timestamps these files) happens at 2 am. Your cron script runs at 3 am, so it updates the files as their timestamp has changed and it doesn't use a full checksum to determine "identity". The next day, perplexed, you run the update by hand but of course nothing has changed since their 2 am update and your 3 am update, so the files aren't updated. This too can be tested, both by the ritual above and by simply skipping the nightly cron altogether and running the update by hand the next day. If the hypothesis is approximately correct, the FIRST time you run the update script after their nightly cron you'll get an unnecessary update, but of course the second and so forth you won't. > Maybe someone here can figure out what is wrong in my scripts and > explain me what is happening? Well, the above is ONE possible situation. I'm not sure that we've reached the Holmesian situation where all that remains, however implausible, must be the truth but I have a fair bit of faith in rsync doing the right thing and suspect that it must be something like the above. It is also extremely odd that it works "by hand" and fails "by cron", where they only possible difference I can think of between them might be inherited environment (but shouldn't be as you use su -). The last suggestion is to try playing with the su - ayo part of your script. Either let the nightly cron run as root (including the rsync) or set up a cron for the user ayo to run the mirror. The reason I suggest this is that you're defining a bunch of variables in the update script and then su'ing to a new login environment where those variables may not be inherited. Also, if I'm not mistaken the su -c line is a bit ambiguous -- shouldn't the "'s around SRPMS be escaped, as they are inside the "" pair for the -c line? You might check to be sure this string is expanding the way you think it is. rgb > > My /etc/cron.daily/mirroring script: > --<snip>-- > #!/bin/bash > MIRRORDIR=/scratch/yum/scripts > rm -f $MIRRORDIR/updates.log > echo "Beginning mirrorin at `date`. " > $MIRRORDIR/updates.log 2>&1 > $MIRRORDIR/get-updrh62.sh >> $MIRRORDIR/updates.log 2>&1 > $MIRRORDIR/get-updrh72.sh >> $MIRRORDIR/updates.log 2>&1 > $MIRRORDIR/get-updrh73.sh >> $MIRRORDIR/updates.log 2>&1 > touch /scratch/yum/updates-fetched.txt > --</snip>-- > > > And for example the get-updrh73.sh looks like this: > > --<snip>-- > #!/bin/bash > MIRRORDIR=/scratch/mirror > VERBOSE=1 > MIRROR=ftp.funet.fi > REMOTE="$MIRROR::ftp/pub/mirrors/ftp.redhat.com/pub/redhat/linux/updates/7.3/en/os/" > LOCAL="$MIRRORDIR/redhat/linux/updates/7.3/en/os/" > if [ ! -z $VERBOSE ]; then > VRB="-vv --stats" > fi > echo "Fetching RH73 updates." > su - ayo -c "rsync $VRB -a --partial --exclude "SRPMS" --delete \ > $REMOTE $LOCAL" > echo " " > --</snip>-- > > Regards, > -- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:rgb@xxxxxxxxxxxx
