Had a DB that had bloated due to lots of updates and insuffcient auto vacuum settings. Attempted the following in the following order:
VACUUM VERBOSE full tablename;
CLUSTER tablename;
Did the full first to ensure that the cluster had enough space to run.
The db had already been previously clustered to an index. At some point during the VACUUM I started seeing the following errors in the logs:
CONTEXT: writing block 12125253 of relation base/2651908/652397108
WARNING: could not write block 12125253 of base/2651908/652397108
DETAIL: Multiple failures --- write error might be permanent.
Also seeing the same error for block 12125253, 2978.
And the cluster failed as well. I have attempted a check point using pg_start_backup() and that failed as well.
The system is still running and I have yet to shut it down. I am hoping to find out what possibilities I have if any to recover from this. Also how do I avoid this in the future?
-Ken
