On Sat, 2008-01-19 at 11:28 -0600, Erik Jones wrote: > All of the warnings are below. For tables that had multiple warnings > they seem to be for consecutive pages. All of these tables were > seeing some pretty decent write traffic during the base backup which > took place Tuesday night. The good news, for us, is that none of the > queue tables matter -- that was transient data. For the other two > history tables, since the PITR recovery ended at 4:58 am Wednesday > morning when it ran up against the dropped WAL file they've already > seen a good amount of traffic. > However, I've been able to verify > missing data by checking the newest timestamp field is older than > those in other tables that had no errors, finding differences of as > much as 11 hours. So we definitely have missing data. I think the multi-phase rsync is definitely suspect and should be avoided until we get to the bottom of this. Can you examine the server logs for that 11 hour period and see if there are any other messages that might be relevant on both primary and standby servers? Can you save the WAL files covering that period also. We may want to inspect them later to confirm whether the data was actually in them or not. Can you save the very first WAL files after the recovery started? We can use those to examine the first data blocks touched, which would help confirm when the gun was fired. Has rsync produced any messages of note? Might your script be ignoring errors? What versions of rsync are you using? Is the History table Insert only? We might use that fact to examine the LSNs of the equivalent blocks on the Primary. If the LSNs are prior to the start of the recovery, as noted in the backup label file of the original base backup, then we can confirm rsync or postgres as the cause. Do you still have the base backup label file or base backup? -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---------------------------(end of broadcast)--------------------------- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
