Eliezer,
It now seems from working on the squid issue that the problem is not in squid. All of your suggestions were helpful when I started working through the issues.
We now think that something in the Code used to build the ASG caused whatever timeout we were seeing.
I am now trying to get to the cache manager as you suggested. I suspect I am having problems because of the Security groups. But I just wanted to tell you your information was very valuable and got
me thinking more about squid.
Thanks for the information and the time you took to look at the issue.
On 6/4/18, 7:40 PM, "Eliezer Croitoru" <eliezer@xxxxxxxxxxxx> wrote:
So just to make sure I understand.
Is squid acting only as an ACL proxy server from inside AWS internal network toward the outside world?
Increasing the timeout to 5 minutes will maybe increase the usage of FD but if this squid has only one worker(basic simple setup with ACL's)
then you can change a timeout and increase the FD the system can handle...
A single working instance of Squid can handle up to a certain amount of traffic and if the instance has let say 2 GB you can safely upper the limit to 64k FD.
My Atom based PC here can handle 64k FD just fine while the actual hardware technically limits it to something like 32k.
On my Xeon based Server I am building and packaging squid with 16k basic limit and it works for most of the business setups out there(not including ISP's).
If all these servers that are using the Squid service are on the same network segment then it would be very weird to change any timeout.
If these servers are not on the same network segment what you need is to turn on keep alive probe let say to 15 seconds per probe.
It will "increase" from n packets to n+(4*connection minutes duration) but as long it is a single worker basic proxy it's nothing.
Try to look at the cache manager interface output for the "info" page and see what is the average connections per second on the Squid service.
(let me know if you need help to get the info cache manager page)
With these numbers you would be able to understand what might causing service disruption.
Eliezer
* by any chance AWS Linux AMD 2018.03.0 has systemd in it or I am imagining that it still uses sysVinit?
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer@xxxxxxxxxxxx
-----Original Message-----
From: Cheadle, Edward <Edward.Cheadle@xxxxxxxxxxxxxxxx>
Sent: Monday, June 4, 2018 23:07
To: Eliezer Croitoru <eliezer@xxxxxxxxxxxx>
Cc: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Connection Timeouts
Eliezer, you are absolutely right. I got in a hurry and forgot the basics such as version numbers and all the other details.
The version currently on our squid server is: squid-3.5.27-1.el6.x86_64.rpm
We are running AWS Linux: Amazon Linux AMI 2018.03.0
We are a health care company. We are using squid proxy to control what the servers in an account can connect to on the internet. AWS looked at an issue we had with code deploy and they said connections were timing out because the default connection timeout is 1 min, and suggested we change the timeout to 5 min. It issue has to do with Codedeploy. Since AWS services are on the internet, I was thinking if we could set an overall timeout, and then one for services that are known to take more time, I thought it would be a way keep the length of the timeout down for most things and free up resources for the majority of tasks.
My concern, as stated below is that connections will take a while to timeout and it will put more pressure on the number of file descriptors we use. We ran into an issue with the number of file descriptors used, but figured it out and we are fine, but increasing the timeout to 5 min set off a warning flag in my mind, not having a lot of experience with squid. I am not even sure it is an issue, but I thought I try to make sure before we ran into production issues.
The reason for including the link, is that it was the first one I found and in the description they mentioned the ability to set timeouts on a site/domain-specific basis, but in the info that followed and in subsequent searches, I did not see how it was done, so the failure to find information on the subject led me to join the list.
In looking at the docs, there are a number of other timeouts, so I obviously have some homework to do.
Thanks for the quick response.
On 6/4/18, 12:31 PM, "Eliezer Croitoru" <eliezer@xxxxxxxxxxxx> wrote:
Hey Edward,
First congrats!.
I hope we can help you to figure out the relevant details.
I am not sure why you have spoken to AWS teams about Squid-Cache, may I ask what OS are you using in AWS?
Also what version of Squid are you using?
The timeout settings are "critical" indeed but depends on what you are using and doing with Squid-Cache.
Despite to the fact that https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.visolve.com%2Fsquid%2Fsquid30%2Ftimeout&data=02%7C01%7CEdward.Cheadle%40cambiahealth.com%7C8be888b30a484f0d8b4f08d5ca49570f%7Ce964274919d44f7fb4df802b2b75a809%7C0%7C0%7C636637338708424102&sdata=SpOxewYBxY1Y7qeK7fk5cEF0pWN2l%2B4UOM6IclHVrbw%3D&reserved=0 Is in a way still a lead it's not "up-to-date"
Please note that without understanding what issues have you been facing and the purpose of the Squid-Cache instance(s?) there is no way to even guess what might fit your needs.
Eliezer
----
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fngtech.co.il%2Flmgtfy%2F&data=02%7C01%7CEdward.Cheadle%40cambiahealth.com%7C8be888b30a484f0d8b4f08d5ca49570f%7Ce964274919d44f7fb4df802b2b75a809%7C0%7C0%7C636637338708424102&sdata=Mpu0Ottn255qQxnsXGT%2F%2ByR432Yz9%2FckeKTuVpZ6aUM%3D&reserved=0
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer@xxxxxxxxxxxx
From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of Cheadle, Edward
Sent: Monday, June 4, 2018 21:06
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Connection Timeouts
We had a person leave and I got selected to update and maintain our squid proxy. We are talking to AWS and they told us that we needed to change the connection_timeout value from the default to 5 min.
We have people stress testing out installation and I was concerned that if connection timeouts are too long we may see congestion.
Should I be worried that connection timeouts will use up file descriptors at a higher rate?
And what might be the options?
Doing and internet search I found a web page at https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.visolve.com%2Fsquid%2Fsquid30%2Ftimeout.php&data=02%7C01%7CEdward.Cheadle%40cambiahealth.com%7C8be888b30a484f0d8b4f08d5ca49570f%7Ce964274919d44f7fb4df802b2b75a809%7C0%7C0%7C636637338708424102&sdata=FSq%2FnnFycwsbQaw8xRMzHkBWFY4Iw5F8KeJtdd1hRyc%3D&reserved=0 and in the TIMEOUT description I read
“TIMEOUT
Timeout parameters in Squid can be based on overall connection timeouts, peer-specific timeouts, site/domain-specific timeouts, request-specific timeouts etc. Proper setting of timeout values is critical to optimal Squid performance. Relevant parameters for timeout settings are listed”
Is it possible to narrow the connection timeout to a specific site? I looked at the website information, squid documentation and did an internet search.
I did not see anything that narrowed the timeout to a specific timeout.
I am trying to set connection timeouts to AWS sites, but keep connection timeouts to the default, because it is working well.
IMPORTANT NOTICE: This communication, including any attachment, contains information that may be confidential or privileged, and is intended solely for the entity or individual to whom it is addressed. If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message is strictly prohibited. Nothing in this email, including any attachment, is intended to be a legally binding signature.
Ensure a sustainable future - only print when necessary.
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
