On Wed, Oct 16, 2019 at 03:35:24PM -0700, Kevin Fenzi wrote: > Greetings folks. > > As you may know, currently database backups on db-koji01 are causing > very heavy load, disrupting our users builds ( > https://pagure.io/fedora-infrastructure/issue/8292 ) > so, they are currently disabled. > > However, not having current backups is not a good thing, IMHO. > > So, I am considering the idea of adding a db-koji02 vm (also rhel7 using > the same postgres version that db-koji01 is) and enabling streaming > replication from db-koji01 -> db-koji02 and then once thats working, run > the database backups on db-koji02. > > It turns out this doesn't require that many changes on db-koji01: > * adding a replication user > * Setting 3 new lines of postgresql config and restarting: > wal_level = 'hot_standby' > max_wal_senders = 10 > wal_keep_segments = 100 > (May need to adjust senders and keep segments) > > All the other changes are on db-koji02: > * create/setup the vm > * run pg_basebackup to pull all the current data from 01 > * setup postgresql.conf and recovery.conf files > * start server and confirm it keeps up with 01 > * run pg_dump and confirm it keeps up with 01 > > This is, of course a really big change in a freeze to a criticial > service, so I'd like to get thoughts from others about it. > > Should we wait until after freeze and do without backups until then > (note, that we have never had to restore this db from backups in the > past, although we have dumped/restored it to move to newer postgres > versions)? > > Is there something else we can do thats easier to mitigate the issues > right now? > > Thoughts? ideas? Rotten fruit? +1 for me as well, the changes to the prod server are fairly limited, db-koji02 doesn't exist atm so isn't concerned with freeze. My only question/worry is the original load that 02 will impose on 01. Though I hope it will be at worst equivalent to a db_dump. Pierre
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
