| Hi all, Today, I tried to recover a postgresql cluster (pitr). I think it went fine (although I'm not sure), but I got some warnings afterwards... 2010-07-15 23:01:48 CEST [22907]: [2-1] user=,db= LOG: starting archive recovery 2010-07-15 23:01:48 CEST [22907]: [3-1] user=,db= LOG: restore_command = '/opt/postgres/data/dov/data/pitr_ssh.sh %p %f' 2010-07-15 23:01:48 CEST [22907]: [4-1] user=,db= LOG: recovery_target_time = '2010-07-15 14:00:00+02' /usr/bin/nsr/recover -d /zones/postgres-on-charlotte/fs/dov/data/pg_xlog -a /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001.history 6469:recover: /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001.history not in index 67847:recover: Failed to recover files: Nothing to recover /usr/bin/nsr/recover -d /zones/postgres-on-charlotte/fs/dov/data/pg_xlog -a /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000C.00000020.backup Recovering 1 file from /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/ into /zones/postgres-on-charlotte/fs/dov/data/pg_xlog Requesting 1 file(s), this may take a while... ./00000001000000000000000C.00000020.backup Received 1 file(s) from NSR server `backup-master1' Recover completion time: Thu Jul 15 23:01:59 2010 2010-07-15 23:01:59 CEST [22907]: [5-1] user=,db= LOG: restored log file "00000001000000000000000C.00000020.backup" from archive /usr/bin/nsr/recover -d /zones/postgres-on-charlotte/fs/dov/data/pg_xlog -a /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000C Recovering 1 file from /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/ into /zones/postgres-on-charlotte/fs/dov/data/pg_xlog Requesting 1 file(s), this may take a while... Total estimated disk space needed for recover is 5 KB. ./00000001000000000000000C Received 1 file(s) from NSR server `backup-master1' Recover completion time: Thu Jul 15 23:02:02 2010 2010-07-15 23:02:02 CEST [22907]: [6-1] user=,db= LOG: restored log file "00000001000000000000000C" from archive 2010-07-15 23:02:02 CEST [22907]: [7-1] user=,db= LOG: automatic recovery in progress 2010-07-15 23:02:02 CEST [22907]: [8-1] user=,db= LOG: redo starts at 0/C000068, consistency will be reached at 0/C000088 2010-07-15 23:02:02 CEST [22907]: [9-1] user=,db= LOG: consistent recovery state reached /usr/bin/nsr/recover -d /zones/postgres-on-charlotte/fs/dov/data/pg_xlog -a /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000D Recovering 1 file from /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/ into /zones/postgres-on-charlotte/fs/dov/data/pg_xlog Requesting 1 file(s), this may take a while... Total estimated disk space needed for recover is 7169 KB. ./00000001000000000000000D Received 1 file(s) from NSR server `backup-master1' Recover completion time: Thu Jul 15 23:02:10 2010 2010-07-15 23:02:10 CEST [22907]: [10-1] user=,db= LOG: restored log file "00000001000000000000000D" from archive 2010-07-15 23:02:12 CEST [22907]: [11-1] user=,db= LOG: recovery stopping before commit of transaction 1003, time 2010-07-15 14:46:23.65437+02 2010-07-15 23:02:12 CEST [22907]: [12-1] user=,db= LOG: redo done at 0/D0010F0 /usr/bin/nsr/recover -d /zones/postgres-on-charlotte/fs/dov/data/pg_xlog -a /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000002.history 6469:recover: /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000002.history not in index 67847:recover: Failed to recover files: Nothing to recover 2010-07-15 23:02:13 CEST [22907]: [13-1] user=,db= LOG: selected new timeline ID: 2 /usr/bin/nsr/recover -d /zones/postgres-on-charlotte/fs/dov/data/pg_xlog -a /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001.history 6469:recover: /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001.history not in index 67847:recover: Failed to recover files: Nothing to recover 2010-07-15 23:02:15 CEST [22907]: [14-1] user=,db= LOG: archive recovery complete 2010-07-15 23:02:15 CEST [25436]: [1-1] user=,db= LOG: checkpoint starting: end-of-recovery immediate wait 2010-07-15 23:02:15 CEST [25436]: [2-1] user=,db= LOG: checkpoint complete: wrote 6 buffers (0.1%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=0.000 s, sync=0.009 s, total=0.180 s 2010-07-15 23:02:15 CEST [27805]: [1-1] user=,db= LOG: autovacuum launcher started 2010-07-15 23:02:15 CEST [22886]: [5-1] user=,db= LOG: database system is ready to accept connections I think everything went fine till here. If not, please tell me what I did wrong. From here on, the log file is filled with messages like this: 3817:save: Using backup-master1 as server /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000E: No such file or directory 2010-07-15 23:02:16: BACKUP WAL: ERROR (1) - /usr/bin/nsr/save -b PGLOGS /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000E 2010-07-15 23:02:16 CEST [27810]: [1-1] user=,db= LOG: archive command failed with exit code 1 2010-07-15 23:02:16 CEST [27810]: [2-1] user=,db= DETAIL: The failed archive command was: /opt/postgres/dba/scripts/backup_WAL-135.sh pg_xlog/00000001000000000000000E 3817:save: Using backup-master1 as server /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000E: No such file or directory 2010-07-15 23:02:18: BACKUP WAL: ERROR (1) - /usr/bin/nsr/save -b PGLOGS /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000E 2010-07-15 23:02:18 CEST [27810]: [3-1] user=,db= LOG: archive command failed with exit code 1 2010-07-15 23:02:18 CEST [27810]: [4-1] user=,db= DETAIL: The failed archive command was: /opt/postgres/dba/scripts/backup_WAL-135.sh pg_xlog/00000001000000000000000E 3817:save: Using backup-master1 as server /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000E: No such file or directory 2010-07-15 23:02:20: BACKUP WAL: ERROR (1) - /usr/bin/nsr/save -b PGLOGS /zones/postgres-on-charlotte/root/opt/postgres/data/dov/data/pg_xlog/00000001000000000000000E 2010-07-15 23:02:20 CEST [27810]: [5-1] user=,db= LOG: archive command failed with exit code 1 2010-07-15 23:02:20 CEST [27810]: [6-1] user=,db= DETAIL: The failed archive command was: /opt/postgres/dba/scripts/backup_WAL-135.sh pg_xlog/00000001000000000000000E 2010-07-15 23:02:20 CEST [27810]: [7-1] user=,db= WARNING: transaction log file "00000001000000000000000E" could not be archived: too many failures 3817:save: Using backup-master1 as server Saving WAL file 00000001000000000000000E gives an error. The problem is that the WAL file does not exist in pg_xlog. Does someone know why? The only WAL file I could find in pg_xlog is 00000002000000000000000D (timeline id is 2). The next thing I tried was looking for 00000001000000000000000E on our backup server. It seems that it didn't exist (or has never been backupped). Finally I created an empty 00000001000000000000000E in pg_xlog and postgresql saved the fake WAL file, so I get rid off the warnings. Did I do something wrong during the backup? Or with the recover? Kind regards, Guy |
