I'm running Postgresql 9.3. I have a streaming replication server. Someone was running a long COPY query (8 hours) on the standby which halted replication. The replication stopped at 3:30 am. I canceled the long-running query at 9:30 am and replication data started catching up.
The data up until 10 am got restored fine (took until 10:30 am to restore that much). Then I started getting errors like "FATAL: could not receive data from WAL stream: ERROR: requested WAL segment 00000001000003C300000086 has already been removed".
I'm confused about how pg could restore data from 3:30 am to 10 am, then start complaining about missing WAL files.
What's the best way to avoid this problem? Increase wal_keep_segments?
Joe