Re: Reliable WAL file shipping over unreliable network

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On Mar 5, 2018, at 10:02 AM, Stephen Frost <sfrost@xxxxxxxxxxx> wrote:

It doesn’t- but please don't encourage partial solutions which have ver clear issues.

Then problem is there are no good base utilities that is useful with archive_command; unless you’re writing directly to an NFS mount or a tape library.  Even Barman recommends rsync with the archive_command; if you are unable to use pg_receivexlog solution.  There are countless Postgres documentation out there that recommends use of rsync with the archive_command.  

Here a solution that will fsync() file on the other end.

SSH_CMD="ssh -o ServerAliveInterval=20 $ARCH_SERVER"
STS=3

OUTPUT=$(cat $XLOGFILE | $SSH_CMD "(mkdir -p $ARCH_DIR && ~/bin/fwrite $ARCH_DIR/$WALFILE)")
if [ $? == 0 ]; then 
   STS=0
fi

exit $STS


fwrite code:

#include <sys/stat.h>
#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>


#define BUFSIZE 131072

int
main(int argc, char *argv[])
{
  int fd, r, w;
  char *buf;
  char *name;
  
  if (argc != 2) {
    fprintf(stderr, "usage: fwrite [file]\n");
    exit(1);
  }

  if ((buf = malloc(BUFSIZE)) == NULL)
    err(1, "malloc");

  ++argv;
  if ((name = (char *) malloc(strlen(*argv) + 10)) == NULL)
    err(1, "malloc");

  strcat(strcpy(name, *argv), ".XXXXXX");

  if ((fd = mkstemp(name)) < 0)
    err(1, "mkstemp");

  while ((r = read(STDIN_FILENO, buf, BUFSIZE)) > 0)
    if ((w = write(fd, buf, r)) == -1)
      err(1, "write");

  if (r < 0)
   err(1, "read");

  if (fsync(fd) != 0)
   err(1, "fsync");

  if (close(fd) != 0)
   err(1, "close");

  if (rename(name, *argv) != 0)
   err(1, "rename");
 
  free(name);
  exit(0);
}



[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux