Errors from archive restore from S3 and wrong size archive file

[Jorge Torralba <jorge.torralba@xxxxxxxxx>]

I have been troubleshooting this for the past few days with little headway.

I have a test environment where my archive directory is an S3 bucket

I use rclone to transfer the files

My recovery.conf file looks like this

restore_command = '/var/lib/postgresql/walsfroms3.sh %f %p'

recovery_target_time = '2018-08-02 16:05:04.859461+00'

which is a call to

this script

export S3_DEST="bla/bla/bla/wal_files"

rclone -v --log-file=/var/lib/postgresql/rclone.log copy remote:${S3_DEST}/$1 $2 

On the attempt to restore. I keep getting the following error

2018-08-02 17:14:42.157 UTC [] [193825]: [2-1] user=,db=,host= LOG:  starting point-in-time recovery to 2018-08-02 16:05:04.859461+00

2018-08-02 17:14:49.330 UTC [] [193825]: [3-1] user=,db=,host= FATAL:  archive file "000000010000000000000002" has wrong size: 38 instead of 16777216

HOWEVER

My rclone logfile and RECOVERYXLOG has the correct file in there.

~ $ cat rclone.log

2018/08/02 17:14:42 NOTICE: Config file "/var/lib/postgresql/.config/rclone/rclone.conf" not found - using defaults

2018/08/02 17:14:42 INFO  : Local file system at /var/lib/postgresql/datas3/pg_xlog/RECOVERYXLOG: Modify window is 1ns

2018/08/02 17:14:42 INFO  : Local file system at /var/lib/postgresql/datas3/pg_xlog/RECOVERYXLOG: Waiting for checks to finish

2018/08/02 17:14:42 INFO  : Local file system at /var/lib/postgresql/datas3/pg_xlog/RECOVERYXLOG: Waiting for transfers to finish

2018/08/02 17:14:49 INFO  : 000000010000000000000002: Copied (new)

2018/08/02 17:14:49 INFO  : 

Transferred:    16 MBytes (2.232 MBytes/s)

Errors:                 0

Checks:                 0

Transferred:            1

Elapsed time:        7.1s

~/datas3/pg_xlog/RECOVERYXLOG $ ls -l

total 16384

-rw-------    1 postgres postgres  16777216 Aug  2 16:03 000000010000000000000002

It's almost as if the recovery is reading the S3 folder name instead of the file name. Any help would be appreciated.

--

Thanks,

Jorge Torralba
----------------------------

Note: This communication may contain privileged or other confidential information. If you are not the intended recipient, please do not print, copy, retransmit, disseminate or otherwise use the information. Please indicate to the sender that you have received this email in error and delete the copy you received. Thank You.