Hi all I've been following a strange issue on dba.stackexchange.com and I could use opinions from those more clueful than I on the log files posted there. http://dba.stackexchange.com/questions/20959/recover-postgresql-database-from-wal-errors-on-startup/20961#comment34356_20961 The short version is that the person did a bulk-load of some PostGIS data using the osm2pgsql data-loader tool ( http://wiki.openstreetmap.org/wiki/Osm2pgsql) to populate a Pg 9.1 database. This appeared to succeed, but when the server was restarted it failed to come up, complaining that "WAL contains references to invalid pages" and "page 1493172 of relation base/16385/477861 was uninitialized" (for many different pages). The logs of the shutdown suggest that a backend probably crashed, but that shouldn't cause the WAL and heap corruption observed by the OP. It's interesting to observe the presence of both a fast and an immediate shutdown request in the log. A trimmed log follows, original linked to in the article above: LOG: received fast shutdown request LOG: aborting any active transactions LOG: received immediate shutdown request WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. LOG: could not send data to client: Broken pipe WARNING: terminating connection because of crash of another server process DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. HINT: In a moment you should be able to reconnect to the database and repeat your command. LOG: could not send data to client: Broken pipe [Note: The posted log of the shutdown ends here. The poster asserts that this is the complete log and that the following statement is the next one in their log. Very strange.] LOG: database system was interrupted; last known up at 2012-07-13 00:15:20 UTC LOG: database system was not properly shut down; automatic recovery in progress LOG: redo starts at 4D/A25FE548 LOG: unexpected pageaddr 4D/6F5C6000 in log file 77, segment 196, offset 6053888 LOG: redo done at 4D/C45C5278 WARNING: page 1493172 of relation base/16385/477861 was uninitialized WARNING: page 2247965 of relation base/16385/477861 was uninitialized WARNING: page 1493172 of relation base/16385/477861 was uninitialized PANIC: WAL contains references to invalid pages LOG: startup process (PID 21574) was terminated by signal 6: Aborted LOG: aborting startup due to startup process failure -- Craig Ringer |