Jeff Janes <jeff.janes@xxxxxxxxx> wrote:
> Kevin Grittner <Kevin.Grittner@xxxxxxxxxxxx> wrote:
>> We pipe the file into pg_clearxlogtail | gzip and pipe it out to
>> the archive directory (with a ".gz" suffix), rather than using cp
>> and processing it later. Well, actually, we pipe it to a
>> directory on the same mount point as the archive directory and mv
>> it into place, as part of our scheme to avoid problems with
>> partial files.
>
> Do you have an example of that which you could share?
>
> I've run into two problems I'm trying to overcome. One is that
> pg_clearxlogtail fails on file formats it doesn't recognize but is
> asked to archive anyway, such as
> '000000010000000200000065.00000020.backup', for example. Perhaps
> it could just issue a warning and then pass the unrecognized file
> through unchanged, instead of bailing out with a fatal error.
>
> The other is that a pipeline in bash reports success even if an
> interior member of it failed. I think I know how to fix that
> under an actual shell script (as opposed to a pipeline stuffed
> into "archive_command"), but would like to see how other people
> have dealt with it.
I make no claims that this couldn't be improved upon, but it has
been humming along without any problems without modification for
about five years now on 80-some machines. Our archive_command is
just a call to a bash script that looks like this:
walPathname=$1
walFilename=$2
tempfile=/var/pgsql/data/wal-files-temp/$walFilename.gz
targetfile=/var/pgsql/data/wal-files/$walFilename.gz
errfile=/home/ccsa/archive.err
emailfile=/home/ccsa/$walFilename.rpt
cat <<END > runtime
walPathname=$walPathname
walFilename=$walFilename
tempfile=$tempfile
targetfile=$targetfile
errfile=$errfile
emailfile=$emailfile
END
if [ -f $targetfile ] ; then
echo "Cannot copy $walPathName to $targetfile \
because it already exists" 2> $errfile
archivereturnval=3
else
if echo $walFilename | grep -Eq ^[0-9A-F]{24}$ - ; then
pg_clearxlogtail < $walPathname 2> $errfile \
| gzip > $tempfile 2>> $errfile \
&& mv $tempfile $targetfile 2>> $errfile
archivereturnval=$?
else
gzip < $walPathname > $tempfile 2> $errfile \
&& mv $tempfile $targetfile 2>> $errfile
archivereturnval=$?
fi
fi
if [ "$archivereturnval" -eq "0" ] ; then
# Successful archive.
# Delete any email file from a possible prior failure.
rm -f $emailfile $errfile
exit 0
fi
# Failed to archive.
if [ ! -f $emailfile ] ; then
# No existing email files -- make one and send it.
echo "date: $(date +%F_%H%M)" > $emailfile
echo "to: dba.group@xxxxxxxxxxxx" >> $emailfile
echo "subject: Problem with archiving $walFilename ">> $emailfile
echo " " >> $emailfile
echo "$archivereturnval return code from archive" >> $emailfile
echo " " >> $emailfile
cat $errfile >> $emailfile
echo " " >> $emailfile
cat runtime >> $emailfile
/usr/sbin/sendmail -t < $emailfile
fi
cat $errfile > /dev/stderr
exit $archivereturnval
--
Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
