Re: backup postgresql with snapshot AWS

[Karl Denninger <karl@xxxxxxxxxxxxx>]

On 1/19/2018 10:23, Achilleas Mantzios wrote:

On 19/01/2018 17:40, Rui DeSousa wrote:

If the snapshot is not atomic and/or using multiple filesystems; that is fine as long as it’s issued and completed within the pg_start_backup() and pg_stop_backup() calls.

And as long as the needed wals belonging to that interval between start and stop are to be found in the archive when the problem arises. So the whole system must be well documented and periodically tested.

The latter is the problem (e.g. when it says "complete" it really is done and it really is safe to issue the pg_stop_backup() and grab the WAL) and further you are relying on an assumption of atomicity within the I/O subsystem when it is requested by the application within your VM (in which it's reasonable to believe that when fsync() returns it really HAS committed the data from the perspective of said VM) yet you're doing some other operation *behind* the VM on which your application is running.

IMHO the more you intertwine things you cannot be certain of because they operate outside of your particular VM's space with a process that relies on them the more you are risking finding out that your assumptions are wrong the hard way.  You *shouldn't* get bit by this in a well-designed cloud or cluster system but that's not an acceptable word to describe the expected results when it's important.

--
Karl Denninger
karl@xxxxxxxxxxxxx
The Market Ticker
[S/MIME encrypted email preferred]

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature