Re: Reliable WAL file shipping over unreliable network

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




Just use "-ac”;  you want -c option to ensure no data corruption during the transfer.  Do not delete the file; let Postgres manage that.

Here is a snippet from I script I use for archiving.  You also want to make your script returns failure or success correctly.  

# SSH Command and options
SSH_CMD="ssh -o ServerAliveInterval=20 $ARCH_SERVER"
STS=3

OUTPUT=$(rsync -ac --rsync-path="mkdir -p $ARCH_DIR && rsync" $XLOGFILE $ARCH_SERVER:$ARCH_DIR/$WALFILE)
if [ $? == 0 ]; then 
   STS=0
fi

exit $STS

On Feb 28, 2018, at 11:38 AM, scott ribe <scott_ribe@xxxxxxxxxxxxxxxx> wrote:

On Feb 28, 2018, at 8:46 AM, Nagy László Zsolt <gandalf@xxxxxxxxxxxx> wrote:

Another problem with rsync might be that it does not know if a WAL file
on the source side is growing (e.g. being written) and it might start
copy that before it is fully flushed to disk. I see this as a big
problem, but I don't have experience.

You use rsync in the archive command, not by itself. Postgres does not try to copy the file until it is ready.

E.g. move the file into the archive dir only after it has been fully
copied to the destination volume. But then we still have the problem of
partially written files on the slave side.

No, you do not have that problem at all.

Given an unreliable network, you probably do want to use --partial


--
Scott Ribe
https://www.linkedin.com/in/scottribe/
(303) 722-0567




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux