On Mon, Jan 11, 2021 at 9:06 AM Craig Jackson <craig.jackson@xxxxxxxxxxxx> wrote:
How far apart are the min/max connection settings on your application connection pool? We had a similar issue with connection storms in the past on Oracle. One thing we did to minimize the storms was make sure there was not a wide gap between the min/max, say no more than a 5-10 connection difference between min/max.
App instances by default are configured for 10 connections in their Hikari connection pool, with no different setting for min so it's always 10. Some have begun setting a minimumIdle to 5. I'm pushing them now to lower their minimumIdle to something like 2, and if they are scaled out to multiple app instances then they should think about lowering their max from 10 to 5 perhaps.
From a recent spike from this morning (names have been changed, but other data is real):
From a recent spike from this morning (names have been changed, but other data is real):
From this morning's spike. At 06:03:15, service foo had 2 active sessions and 127 idle sessions. At 06:03:30 (the next tick in grafana), it had 5 active sessions but 364 idle sessions. The foo_db DB overall had 9 active and 337 idle at 06:03:15, and then 5 active and 788 idle overall in the next tick. So a flood of new connections were created within that 15 second interval (probably all within the same second) and more or less abandoned.
Don.
Don Seiler
www.seiler.us
www.seiler.us
