We're continuing to have timeout issues in Taskotron staging and my
investigation so far seems to indicate the cause to be slow queries in
the database.
This change does require a database restart but due to the way that
Taskotron works, I can do this so that there are no lost jobs and no
significant downtime. I'd stop incoming jobs until all queues are empty
and record the jobs which would have been scheduled on a machine
outside of infra. Once those queues are empty, I'd shut down all the
db-using processes, apply the patch, restart the db, start everything
back up and enqueue the jobs which would have been scheduled.
This template change to the postgresql-server module will only affect
db-qa01.qa but will make it look like other postgres servers have a
pending change due to the way I've changed the postgresql.conf template.
I'm not crazy about doing this during freeze but I'm worried that the
timeout problem will start affecting more than stg before long and
want to get this figured out before that happens.
+1s?
Tim
diff --git a/roles/postgresql_server/templates/postgresql.conf b/roles/postgresql_server/templates/postgresql.conf
index 603f9ea..c9756b8 100644
--- a/roles/postgresql_server/templates/postgresql.conf
+++ b/roles/postgresql_server/templates/postgresql.conf
@@ -319,9 +319,15 @@ log_rotation_size = 0 # Automatic rotation of logfiles will
# fatal
# panic (effectively off)
+{% if ansible_hostname.startswith("db-qa01") %}
+log_min_duration_statement = 500 # -1 is disabled, 0 logs all statements
+ # and their durations, > 0 logs only
+ # statements running at least this time.
+{% else %}
#log_min_duration_statement = -1 # -1 is disabled, 0 logs all statements
# and their durations, > 0 logs only
# statements running at least this time.
+{% endif %}
#silent_mode = off # DO NOT USE without syslog or
# logging_collector
Attachment:
pgpGQOyMQY7XI.pgp
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/infrastructure
