Re: Reliable WAL file shipping over unreliable network

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On Mar 1, 2018, at 12:21 AM, scott ribe <scott_ribe@xxxxxxxxxxxxxxxx> wrote:

The false report of success is not good, but it's not the root problem.

A false success if a problem; especially in this use case as the source WAL file will be deleted by Postgres before it was truly successful.  While monitoring is nice to avoid the issue it is not a fix for the issue.

I personally cannot recommend the use of rsync in this application for two reasons.  

1. It adds no value; it’s a more complex cp command (no bandwidth saved, etc as archive processes a single file at a time).
2. It lies on success/failure — Period.


I have use “cat” longer than I have used rsync to archive WALs.  I can say that I’ve lost zero WAL files using cat; I can not say the same for rsync.

The following code is more reliable than rsync and works with across multiple platforms and filesystems without fail.  

STS=3

OUTPUT=$(cat $XLOGFILE | $SSH_CMD "(mkdir -p $ARCH_DIR && cat > $ARCH_DIR/$WALFILE.swap) && mv $ARCH_DIR/$WALFILE.swap $ARCH_DIR/$WALFILE")
if [ $? == 0 ]; then 
   STS=0
fi

exit $STS



[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux