Search Postgresql Archives

Re: Methods to quickly spin up copies of an existing databases

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Pre-copying is not really an option since we could potentially need 1-X instances so it needs to be scalable.  XFS also allows for "cp --reflink" which I could do on a PGDATA directory and then change the port number.  That's probably the method I'll try first.

We do use barman, but again a barman recover operation is pretty much just an rsync. 

I will first try XFS, then ZFS, and if those don't work, I'll look into a SAN option.



On Fri, Mar 1, 2019 at 4:29 PM Bruce Klein <brucek@xxxxxxxxx> wrote:
Apologies for the low tech suggestion, but if this really is a clone of a previously existing template, could the clone operation just be done ahead of time? I.e., have the build server keep X copies ready for use and generate additional copies as those are consumed, so that the cloning is no longer on the critical path?

On Fri, Mar 1, 2019 at 11:09 AM Jerry Sievers <gsievers19@xxxxxxxxxxx> wrote:
Kenneth Marshall <ktm@xxxxxxxx> writes:

> On Fri, Mar 01, 2019 at 11:57:30AM -0800, Kevin Wilkinson wrote:
>
>> if you are able/willing to use ZFS (rather than ext4, xfs, ...) to
>> store your database, then it might work for you. ZFS is
>> copy-on-write so it can very quickly clone a database.
>>
>> kevin
>
> Hi Arjun
>
> Redhat 7 does have LVM snapshots that does something similar. Kevin is
> correct, COW is the secret.

Going a bit further...

Any sort of storage backend that can support *atomic* snapshots across
*all* volumes (in case multiple tablespaces ar involved), can be used to
permit $instantaneous cloning where instantaneous relates to the actual
snapshot time and crash recovery.

Inability to make *atomic* snaps but perhaps seperate snaps very
quickly, combined with PITR can result in clones of high-churn systems
sized in TBs (as in our use case) to be provisioned in about 1 minute.

Nothing but the most trivial system can be cloned rapidly and perhaps
any number of times in succession without employment of
thin-provisioning, copy-on-write (as mentioned already), etc.

                   Virtual copy is more and more compelling as physical
                   size, or more precisely, *physical* copy time grow.

HTH



>
> Regards,
> Ken
>
>

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@xxxxxxxxxxx


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Postgresql Jobs]     [Postgresql Admin]     [Postgresql Performance]     [Linux Clusters]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Postgresql & PHP]     [Yosemite]

  Powered by Linux