On Wed, Jan 24, 2018 at 3:54 AM, pavan95 <pavan.postgresdba@xxxxxxxxx> wrote:
Hi Claudio,
We didn't configure any replication to our production server. Which strace
are you talking about?
You can attach it to a process (assuming you have the necessary permissions) and it will report all the syscalls the process does. That does slow down the process though.
Then lsof ( https://linux.die.net/man/8/lsof ) can be used to map file descriptor numbers to file paths. You have to do it as soon as you read the output, because files get closed and file descriptors reused. So it's better to have a script that directly reads from /proc/pid/fd or fdinfo, but that takes some programming.
It is nontrivial, but sometimes it's the only tool in your belt. You may want to try something else first though.
We did a keen observation that only at the time 9'th minute of the hour and
39'th minute of the hour the so called archive logs are generated even when
nobody is connecting from application(off the business hours).
Well, if you don't know what happens at those times (and only at those times), it's not that useful.
Since you don't know what is causing this for certain, first thing you have to do is ascertain that. Try increasing logging as much as you can, especially around those times, and see what turns on then and not at other times. You can monitor autovacuum processes as well in pg_stat_activity, so make sure you check that as well, as autovacuum will only log once it's done.
You do know autovacuum is running at those times, you have to check whether it isn't when WAL isn't being generated, and whether autovacuum is vacuuming the same tables over and over or what. Your earlier mails show autoanalyze runs, not vacuum. Those shouldn't cause so much WAL, but if it's running very often and you have lots of stats, then maybe.
You can also try pg_stat_statements: https://www.postgresql.org/docs/9.1/static/pgstatstatements.html
Again, concentrate on the differential - what happens at those times, that doesn't at other times.
Another idea would be to check for freeze runs in autovacuum. Ie, what's described here: https://wiki.postgresql.org/wiki/VacuumHeadaches#FREEZE
There's a nice blog post with some queries to help you with that here: http://www.databasesoup.com/2012/09/freezing-your-tuples-off-part-1.html
(and it's continuation here: http://www.databasesoup.com/2012/10/freezing-your-tuples-off-part-2.html ). I'm not saying you should tune those parameters, what you were showing was autoanalyze activity, not vacuum freeze, but you should check whether you need to anyway.
