On 6 Feb 2021, at 23:06, Atul Kumar wrote:
Thanks for providing the details, I need more clarification on this as
how
should I analyze that what should be ideal no. of connections should
we set
to avoid IO overhead based on the available hardware resources.
How to do this analysis properly is outside my expertise and likely off
topic for this list. But you can get an overall idea doing what I
suggested: measure the performance (i.e., total time to completion) when
using different numbers of threads). In an ideal world the time will get
less as you invoke more threads. But there will be a point where there
is a bottle neck (or some resource hits its limit) and the time will be
more or less the same no matter how many threads.
How to do this calculation ?
That’s likely too hard. Measurement will tell you what you need to
know. Once you know how many threads it takes to saturate you will know
you need to use a lesser number. This may, or may not, get you to
revisit the max 700 connections setting but once you know there is a
performance limit, and you decide it’s not good enough, then you can
ask the hardware people for help.
Note: even during 300 threads, my RAM utilisation is totally normal.
That’s not unexpected. The CPU activity should be using the data
held in RAM not spending all those cycles allocating RAM.
Gavan Schneider
——
Gavan Schneider, Sodwalls, NSW, Australia
Explanations exist; they have existed for all time; there is always a
well-known solution to every human problem — neat, plausible, and
wrong.
— H. L. Mencken, 1920
