Florian, thanks for your response.
(I am not subscribed to the list so cannot reply properly, sorry!)
Would be interesting to see
conntrack -L --p tcp dport 22 --state ESTABLISHED | wc -l
and see what conntrack state actually is.
I checked - it displays the same number of connections as netstat. Also,
I assume you meant "--dport 7777" because without dashes or with port 22
it was displaying something totally different.
I also tries changing my iptables rule and instead of
-m conntrack --ctstate NEW
tried
--syn
this is just a flag-based comparison as I understand it so it does not
involve conntrack. And the connlimit still allows more connections that
it should.
Another interesting observation - I played with a delay that client
threads sleep for between consecutive connection requests. And results
are quite interesting. Below is how many connections client could
establish when different delays caused different alignment of connection
attempts from different threads. (The notation "0.03, 0.05, 0.07, 0.11"
means that the first thread sleeps 0.03s between each of its
connections, second - 0.05s, third - 0.07s and fourth - 0.11s
0.03, 0.05, 0.07, 0.11 => 2037 # each thread uses a different delay
0.07, 0.05, 0.07, 0.11 => 2219 # two threads aligned with delay of 0.07
0.07, 0.07, 0.07, 0.11 => 3116 # three threads aligned with delay of 0.07
0.07, 0.07, 0.07, 0.07 => 3398 # all four threads aligned
In each of these tests, server's --connlimit-above was set to 2000
connections.
So looks like it is indeed some concurrency issue - the limit gets
breached when multiple connections are established "in parallel" and
close in time from each other.
Another super-interesting thing is that when I start m4.xlarge instead
of m5.xlarge AWS instance from *exactly the same AMI*, the problem goes
away! And regardless of number of threads on the client, server limit
number of connections correctly. I am totally puzzled on how machine
type can matter given it is the same kernel etc. I think I will raise a
case with Amazon.
But still, really puzzling!
Cheers
--
hivehome.com <http://www.hivehome.com>
Hive | London | Cambridge |
Houston | Toronto****
The information contained in or attached to this
email is confidential and intended only for the use of the individual(s) to
which it is addressed. It may contain information which is confidential
and/or covered by legal professional or other privilege. The views
expressed in this email are not necessarily the views of Centrica plc, and
the company, its directors, officers or employees make no representation or
accept any liability for their accuracy or completeness unless expressly
stated to the contrary. ****
Centrica Hive Limited (company no: 5782908),
registered in England and Wales with its registered office at Millstream,
Maidenhead Road, Windsor, Berkshire SL4 5GD.
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
