On Mon, Jun 3, 2024 at 11:22 PM Scott Ribe <scott_ribe@xxxxxxxxxxxxxxxx> wrote:
> On Jun 3, 2024, at 9:06 PM, Ron Johnson <ronljohnsonjr@xxxxxxxxx> wrote:
>
> Takes 2 hr 45 min for us to do a weekly full backup of the 6.1TB instance using integrated gzip compression (it's an old RHEL6 system, so nothing better) and 24 threads. Size 3.3T after compression, which is surprising based on how much already-compressed bytea data there is.
FYI, Naveed, there's a tradeoff available in current versions of pgbackrest. It can be configured to use lz4 compression. For us, that means close to 50% larger backups than gzip, but many times faster. 3.2TB in 30 minutes, using 8 processes, 725GB backup size--no already compressed bytea, but a good bit of incompressible UUID columns.
Good enough that I stopped doing daily incrementals, and just take full backups daily.
How big were the incrementals? (I'm thinking more weeks of backup set retention when using full+incr.)
