Hi everyone,
(sorry if this is a dup - I'm not sure if my previous message get through)
I need an advice in database recovery. Postgresql 9.2.9.
Last night we moved one of our databases to another server following this procedure:
https://wiki.postgresql.org/wiki/Streaming_ReplicationReplication, slave becomes master, pgbouncer restarts with new config, everyone happy.
Until
soon we noticed strange errors. Turns out data was broken somewhere -
we had unique indexes with duplicate records, we had missing chunks in
toast tables, and the like.
Scale of the problems wasn't clear
and I fixed some of the most obvious problems - recreated indices,
cleared duplicates. Around 0:30 I started database-wide VACUUM, it
started printing messages about fixing things. Everything seemed to work
normally.
Around 3:30 database stopped responding to queries;
it was noticed around 9:30 and I attempted to stop VACUUM (ctrl+c in
psql command line). 30 minutes and nothing. I decided to restart
postmaster. After another 30 minutes we switched back to old server. Already more than 6
hours passed and new server postmaster is still shutting down. And I
would like to get it back to restore yesterday data.
At
this point I could do with new server anything that will get database up
and running sooner. kill -9, start, wait for recovery? Server reboot?
Any help will be greatly appreciated.
Best regards,
Dmitriy Shalashov
