Yesterday, one of the (replicated) remote databases I work with somehow got corrupted, so I attempted to drop a new copy off of the master (on a different box) and rebuild the database. Creation, language install, schema reload, all appeared to go well. On the actual data reload, I set the system aside and went on to something else, as the db takes a while to load. I came back to discover that the connection between my system and the one where the db was being rebuilt had been severed. Opening a new remote connection, I went in and attempted to rebuild the database, only to discover that neither postgres nor the postmaster was running. And when I attempted to restart the postmaster process, I received the following error:
postgres@district20:/usr/local/pgsql/bin> ./postmaster -D /usr/local/pgsql/data/
LOG: database system was interrupted while in recovery at 2007-07-31 08:17:22 CDT
HINT: This probably means that some data is corrupted and you will have to use the last backup for recovery.
LOG: checkpoint record is at 3C/D7008078
LOG: redo record is at 3C/D7008078; undo record is at 0/0; shutdown FALSE
LOG: next transaction ID: 59170527; next OID: 532878
LOG: next MultiXactId: 1; next MultiXactOffset: 0
LOG: database system was not properly shut down; automatic recovery in progress
LOG: redo starts at 3C/D70080BC
PANIC: block 39 unfound
WARNING: autovacuum not started because of misconfiguration
HINT: Enable options "stats_start_collector" and "stats_row_level".
LOG: startup process (PID 6403) was terminated by signal 6
LOG: aborting startup due to startup process failure
LOG: database system was interrupted while in recovery at 2007-07-31 08:17:22 CDT
HINT: This probably means that some data is corrupted and you will have to use the last backup for recovery.
LOG: checkpoint record is at 3C/D7008078
LOG: redo record is at 3C/D7008078; undo record is at 0/0; shutdown FALSE
LOG: next transaction ID: 59170527; next OID: 532878
LOG: next MultiXactId: 1; next MultiXactOffset: 0
LOG: database system was not properly shut down; automatic recovery in progress
LOG: redo starts at 3C/D70080BC
PANIC: block 39 unfound
WARNING: autovacuum not started because of misconfiguration
HINT: Enable options "stats_start_collector" and "stats_row_level".
LOG: startup process (PID 6403) was terminated by signal 6
LOG: aborting startup due to startup process failure
A google search on the Panic clause lead me to an old thread in the [Hackers] list,
which looks like it was a similar problem, but I can't figure out from that thread how the problem was solved. Would someone please help me figure out what I need to do to correct this and get my database running again?
Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us.
