Hey Vieri,
FD and other limits are a tricky thing in gentoo and couple other systems.
In ubuntu for example to apply a ulimit for squid what they did is add a ulimit command at the beginning of the init script.
Ie 'ulimit -hn 65535;ulimit -sn 65535'
In other init or startup systems like systemd which are not started from a restricted shell like bash you can define these settings.
P.S. You should not restart squid because of a FD limit from a crontab job....
If a specific user abuses the service you should try create some firewall rules to limit clients requests bursts.
All The Best,
Elieser
----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd@xxxxxxxxx
-----Original Message-----
From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of Vieri
Sent: Saturday, October 31, 2020 1:03 PM
To: Squid Users <squid-users@xxxxxxxxxxxxxxxxxxxxx>
Subject: squid restart
Hi,
Around every hour or so, the Squid proxy client experience comes to a crawl.
It takes a very long time to load a simple web page.
This is a snapshot taken when this happens:
# squidclient mgr:info
HTTP/1.1 200 OK
Server: squid
Mime-Version: 1.0
Date: Sat, 31 Oct 2020 10:43:21 GMT
Content-Type: text/plain;charset=utf-8
Expires: Sat, 31 Oct 2020 10:43:21 GMT
Last-Modified: Sat, 31 Oct 2020 10:43:21 GMT
X-Cache: MISS from inf-fw1
X-Cache-Lookup: MISS from inf-fw1:3128
Connection: close
Squid Object Cache: Version 5.0.4-20201013-r6b13b73d3
Build Info:
Service Name: squid
Start Time: Sat, 31 Oct 2020 04:50:45 GMT
Current Time: Sat, 31 Oct 2020 10:43:21 GMT
Connection information for squid:
Number of clients accessing cache: 320
Number of HTTP requests received: 519828
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Number of HTCP messages received: 0
Number of HTCP messages sent: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 1474.3
Average ICP messages per minute since start: 0.0
Select loop called: 9044075 times, 2.339 ms avg
Cache information for squid:
Hits as % of all requests: 5min: 2.1%, 60min: 2.5%
Hits as % of bytes sent: 5min: -68.9%, 60min: -402.5%
Memory hits as % of hit requests: 5min: 78.3%, 60min: 62.3%
Disk hits as % of hit requests: 5min: 0.0%, 60min: 1.7%
Storage Swap size: 29040 KB
Storage Swap capacity: 88.6% used, 11.4% free
Storage Mem size: 29212 KB
Storage Mem capacity: 89.1% used, 10.9% free
Mean Object Size: 17.31 KB
Requests given to unlinkd: 11815
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.04519 0.04776
Cache Misses: 0.06286 0.06286
Cache Hits: 0.00000 0.00000
Near Hits: 0.04277 0.02317
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.00000 0.00000
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 21155.513 seconds
CPU Time: 1334.166 seconds
CPU Usage: 6.31%
CPU Usage, 5 minute avg: 8.60%
CPU Usage, 60 minute avg: 9.88%
Maximum Resident Size: 4287872 KB
Page faults with physical i/o: 0
Memory accounted for:
Total accounted: 744703 KB
memPoolAlloc calls: 136343652
memPoolFree calls: 140190831
File descriptor usage for squid:
Maximum number of file descriptors: 4096
Largest file desc currently in use: 4009
Number of file desc currently in use: 3997
Files queued for open: 0
Available number of file descriptors: 99
Reserved number of file descriptors: 100
Store Disk files open: 0
Internal Data Structures:
1852 StoreEntries
1849 StoreEntries with MemObjects
1754 Hot Object Cache Items
1678 on-disk objects
If I issue the '-k reconfigure' command then the user experience is "great again".
A data snapshot taken right after the latter command shows this:
# squidclient mgr:info
HTTP/1.1 200 OK
Server: squid
Mime-Version: 1.0
Date: Sat, 31 Oct 2020 10:48:40 GMT
Content-Type: text/plain;charset=utf-8
Expires: Sat, 31 Oct 2020 10:48:40 GMT
Last-Modified: Sat, 31 Oct 2020 10:48:40 GMT
X-Cache: MISS from inf-fw1
X-Cache-Lookup: MISS from inf-fw1:3128
Connection: close
Squid Object Cache: Version 5.0.4-20201013-r6b13b73d3
Build Info:
Service Name: squid
Start Time: Sat, 31 Oct 2020 10:46:51 GMT
Current Time: Sat, 31 Oct 2020 10:48:40 GMT
Connection information for squid:
Number of clients accessing cache: 179
Number of HTTP requests received: 4663
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Number of HTCP messages received: 0
Number of HTCP messages sent: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 2575.3
Average ICP messages per minute since start: 0.0
Select loop called: 51220 times, 2.121 ms avg
Cache information for squid:
Hits as % of all requests: 5min: 0.3%, 60min: 0.3%
Hits as % of bytes sent: 5min: 3.6%, 60min: 3.6%
Memory hits as % of hit requests: 5min: 60.0%, 60min: 60.0%
Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.0%
Storage Swap size: 10292 KB
Storage Swap capacity: 31.4% used, 68.6% free
Storage Mem size: 10456 KB
Storage Mem capacity: 31.9% used, 68.1% free
Mean Object Size: 15.07 KB
Requests given to unlinkd: 4657
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.05046 0.05046
Cache Misses: 0.06286 0.06286
Cache Hits: 0.00000 0.00000
Near Hits: 0.15048 0.15048
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.00000 0.00000
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 108.639 seconds
CPU Time: 10.588 seconds
CPU Usage: 9.75%
CPU Usage, 5 minute avg: 12.90%
CPU Usage, 60 minute avg: 12.90%
Maximum Resident Size: 462736 KB
Page faults with physical i/o: 0
Memory accounted for:
Total accounted: 37879 KB
memPoolAlloc calls: 1256976
memPoolFree calls: 1307898
File descriptor usage for squid:
Maximum number of file descriptors: 4096
Largest file desc currently in use: 567
Number of file desc currently in use: 559
Files queued for open: 0
Available number of file descriptors: 3537
Reserved number of file descriptors: 100
Store Disk files open: 0
Internal Data Structures:
997 StoreEntries
997 StoreEntries with MemObjects
683 Hot Object Cache Items
683 on-disk objects
This did not happen with Squid 4, or maybe it wasn't as obvious.
I guess the reason could be for this:
Maximum number of file descriptors: 4096
Largest file desc currently in use: 4009
Number of file desc currently in use: 3997
However, I set the following directive in squid.conf:
max_filedescriptors 65536
It doesn't seem to be honored here unless I stop and restart the squid service again (/etc/init.d/squid restart from command line):
File descriptor usage for squid:
Maximum number of file descriptors: 65535
It seems that if I run the same command (/etc/init.d/squid restart) from crontab, that ulimit is not honored. I guess that's the root cause of my issue because I am asking cron to restart Squid once daily. I'll try not to, but I was hoping to see if there was a reliable way to fully restart the Squid process.
Vieri
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
