Squid Proxy timing out 500/503 errors

["M, Anitha (CSS)" <anitha.m@xxxxxxx>]

Hi team,

We are using squid service deployed as a KVM VM on SLES 15 Sp5 os image.

We are using squid. Rpm: squid-5.7-150400.3.20.1.x86_64

 

We are seeing too many 503 errors with this version of squid.

This is the squid configuration file. Pls review it and let us know if issues.

 

We are performing squid scale testing, where every secs there will be 200+requests reaching the squid and squid is spitting out 500/503 errors.

 

Squid.conf:

gl-pcesreblr-squidproxy03:/var/log/squid # cat /etc/squid/squid.conf
# Recommended minimum configuration:
acl localnet src 172.28.1.0/24
acl localnet src 172.28.4.0/24
acl localnet src 172.28.0.0/24
acl localnet src 172.28.0.12/32
connect_timeout 120 seconds
connect_retries 10
#debug_options ALL,5
#connect_retries_delay 5 seconds
acl localnet src 0.0.0.1-0.255.255.255  # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8             # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10          # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16         # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.28.11.0/24
#acl localnet src 172.16.0.0/12         # RFC 1918 local private network (LAN)
#acl localnet src 192.168.0.0/16                # RFC 1918 local private network (LAN)
#acl localnet src fc00::/7              # RFC 4193 local private network range
#acl localnet src fe80::/10             # RFC 4291 link-local (directly plugged) machines

 

acl blocksites url_regex "/etc/squid/blocksites"
http_access deny blocksites

 

debug_options ALL,7

 

acl SSL_ports port 443
acl SSL_ports port 8071
acl SSL_ports port 11052
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl Safe_ports port 53          # pdns
acl Safe_ports port 5300        # pdns
acl Safe_ports port 123     #NTP
acl Safe_ports port 8071
acl Safe_ports port 11052       # pdns web server
acl Safe_ports port 514         # rsyslog
acl CONNECT method CONNECT
acl SSL_ports port 8053
acl Safe_ports port 8053
acl SSL_ports port 3002
acl Safe_ports port 3002
acl SSL_ports port 3006
acl Safe_ports port 3006
acl SSL_ports port 8203
acl Safe_ports port 8203
acl SSL_ports port 8204
acl Safe_ports port 8204
acl SSL_ports port 8071
acl Safe_ports port 8071
acl Safe_ports port 8200
acl SSL_ports port 8099
acl Safe_ports port 8099
tcp_outgoing_address 20.20.30.5

 

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

 

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

 

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

 

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

 

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

 

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

 

# And finally deny all other access to this proxy
#http_access deny all
#http_access allow all

 

cache_peer proxy-in.its.hpecorp.net parent 443 0 no-query no-delay default
#cache_peer 16.242.46.11 parent 8080 0 no-query default
#cache_peer 10.132.100.29 parent 3128 0 no-query default

 

acl parent_proxy src all
http_access allow parent_proxy
never_direct allow parent_proxy

 

# Squid normally listens to port 3128
http_port 3128

 

# Leave coredumps in the first cache dir
coredump_dir /var/cache/squid

 

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

 

dns_nameservers 172.28.0.121 16.110.135.52

 

max_filedescriptors 3200
cache_dir ufs /var/cache/squid 8192 16 256
cache_mem 2096 MB
cache_swap_high 95
cache_swap_low 90
ftp_passive on
maximum_object_size 4096 MB
memory_replacement_policy lru
minimum_object_size 0 KB

 

# Recommended minimum configuration:
acl localnet src 172.28.4.0/24
acl localnet src 172.28.0.0/24
acl localnet src 172.28.1.0/24 # OOBM Network outbound access
#acl HOGAN dst hogan.nimblestorage.com
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 “this” network (LAN)
acl blocksites url_regex “/etc/squid/blocksites”
http_access deny blocksites
acl SSL_ports port 443
acl SSL_ports port 8071
acl SSL_ports port 11052
acl SSL_ports port 8200
acl SSL_ports port 8282
acl Safe_ports port 8282
#acl HOGAN_port port 2222 # hogan.nimblestorage.com:2222 SSH support tunnel
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
acl localnet src 172.16.117.0/24
http_access allow localnet
http_access allow localhost
#http_access allow HOGAN HOGAN_port
acl localnet src 20.20.30.0/21
acl parent_proxy_exclude dst 20.20.30.0/21
acl parent_proxy_exclude_ST0100 dst 20.20.30.222/22
always_direct allow parent_proxy_exclude_ST0100
acl servicenet dst 172.28.4.0/24
always_direct allow parent_proxy_exclude
always_direct allow servicenet

 

 

 

Logs:

 

2024/03/05 22:42:57.000 kid1| 1,5| CodeContext.cc(60) Entering: master1604

2024/03/05 22:42:57.000 kid1| 5,3| Read.cc(148) HandleRead: FD 756, size 65535, retval 206, errno 0

2024/03/05 22:42:57.000 kid1| 5,3| IoCallback.cc(112) finish: called for conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1 (0, 0)

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCall.cc(97) ScheduleCall: IoCallback.cc(131) will call TunnelBlindCopyReadHandler(conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1, data="" size=206, buf=0x557f30d277a0) [call131521]

2024/03/05 22:42:57.000 kid1| 1,7| CodeContext.cc(70) Leaving: master1604

2024/03/05 22:42:57.000 kid1| 1,5| CodeContext.cc(60) Entering: master1604

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCallQueue.cc(59) fireNext: entering TunnelBlindCopyReadHandler(conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1, data="" size=206, buf=0x557f30d277a0)

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCall.cc(42) make: make call TunnelBlindCopyReadHandler [call131521]

2024/03/05 22:42:57.000 kid1| 26,3| tunnel.cc(526) ReadServer: conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1

2024/03/05 22:42:57.000 kid1| 26,3| tunnel.cc(534) readServer: conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1, read 206 bytes, err=0

2024/03/05 22:42:57.000 kid1| 26,3| tunnel.cc(486) bytesIn: len=0 + count=206

2024/03/05 22:42:57.000 kid1| 26,3| tunnel.cc(603) keepGoingAfterRead: from={conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1}, to={conn3867 local=20.20.30.2:3128 remote=20.20.31.153:49724 FD 753 flags=1}

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCall.cc(30) AsyncCall: The AsyncCall tunnelTimeout constructed, this=0x557f30a20410 [call132703]

2024/03/05 22:42:57.000 kid1| 5,3| comm.cc(571) commSetConnTimeout: conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1 timeout 900

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCall.cc(30) AsyncCall: The AsyncCall tunnelTimeout constructed, this=0x557f2ebd3660 [call132704]

2024/03/05 22:42:57.000 kid1| 5,3| comm.cc(571) commSetConnTimeout: conn3867 local=20.20.30.2:3128 remote=20.20.31.153:49724 FD 753 flags=1 timeout 900

2024/03/05 22:42:57.000 kid1| 26,3| tunnel.cc(646) copy: Schedule Write

2024/03/05 22:42:57.000 kid1| 5,5| AsyncCall.cc(30) AsyncCall: The AsyncCall TunnelBlindCopyWriteHandler constructed, this=0x557f2ef09120 [call132705]

2024/03/05 22:42:57.000 kid1| 5,5| Write.cc(37) Write: conn3867 local=20.20.30.2:3128 remote=20.20.31.153:49724 FD 753 flags=1: sz 206: asynCall 0x557f2ef09120*2

2024/03/05 22:42:57.000 kid1| 5,5| ModEpoll.cc(118) SetSelect: FD 753, type=2, handler=1, client_data=0x7f766627b060, timeout=0

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCallQueue.cc(61) fireNext: leaving TunnelBlindCopyReadHandler(conn3875 local=20.20.30.5:50937 remote=10.120.125.146:443 FIRSTUP_PARENT FD 756 flags=1, data="" size=206, buf=0x557f30d277a0)

2024/03/05 22:42:57.000 kid1| 1,7| CodeContext.cc(70) Leaving: master1604

2024/03/05 22:42:57.000 kid1| 1,5| CodeContext.cc(60) Entering: master87

2024/03/05 22:42:57.000 kid1| 5,5| comm.cc(1633) checkTimeouts: checkTimeouts: FD 45 Expired

2024/03/05 22:42:57.000 kid1| 5,5| comm.cc(1636) checkTimeouts: checkTimeouts: FD 45: Call timeout handler

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCall.cc(97) ScheduleCall: comm.cc(1639) will call Comm::ConnOpener::timeout(conn85 local=20.20.30.5 remote=10.120.125.146:443 FIRSTUP_PARENT flags=1, data="" [call419]

2024/03/05 22:42:57.000 kid1| 1,7| CodeContext.cc(70) Leaving: master87

2024/03/05 22:42:57.000 kid1| 1,5| CodeContext.cc(60) Entering: master138

2024/03/05 22:42:57.000 kid1| 5,5| comm.cc(1633) checkTimeouts: checkTimeouts: FD 91 Expired

2024/03/05 22:42:57.000 kid1| 5,5| comm.cc(1636) checkTimeouts: checkTimeouts: FD 91: Call timeout handler

2024/03/05 22:42:57.000 kid1| 5,4| AsyncCall.cc(97) ScheduleCall: comm.cc(1639) will call Comm::ConnOpener::timeout(conn213 local=20.20.30.5 remote=10.120.125.146:443 FIRSTUP_PARENT flags=1, data="" [call1148]

2024/03/05 22:42:57.000 kid1| 1,7| CodeContext.cc(70) Leaving: master138

2024/03/05 22:42:57.000 kid1| 1,5| CodeContext.cc(60) Entering: master1604

2024/03/05 22:42:57.000 kid1| 5,5| Write.cc(69) HandleWrite: conn3867 local=20.20.30.2:3128 remote=20.20.31.153:49724 FD 753 flags=1: off 0, sz 206.

2024/03/05 22:42:57.001 kid1| 5,5| Write.cc(89) HandleWrite: write() returns 206

2024/03/05 22:42:57.001 kid1| 5,3| IoCallback.cc(112) finish: called for conn3867 local=20.20.30.2:3128 remote=20.20.31.153:49724 FD 753 flags=1 (0, 0)

2024/03/05 22:42:57.001 kid1| 5,5| AsyncCall.cc(97) ScheduleCall: IoCallback.cc(131) will call TunnelBlindCopyWriteHandler(conn3867 local=20.20.30.2:3128 remote=20.20.31.153:49724 FD 753 flags=1, data="" size=206, buf=0x557f30d277a0) [call132705]

2024/03/05 22:42:57.001 kid1| 1,7| CodeContext.cc(70) Leaving: master1604

2024/03/05 22:42:57.001 kid1| 1,5| CodeContext.cc(60) Entering: master87

2024/03/05 22:42:57.001 kid1| 5,4| AsyncCallQueue.cc(59) fireNext: entering Comm::ConnOpener::timeout(conn85 local=20.20.30.5 remote=10.120.125.146:443 FIRSTUP_PARENT flags=1, data="">

2024/03/05 22:42:57.001 kid1| 5,4| AsyncCall.cc(42) make: make call Comm::ConnOpener::timeout [call419]

2024/03/05 22:42:57.001 kid1| 5,4| AsyncJob.cc(124) callStart: Comm::ConnOpener status in: [ job76]

2024/03/05 22:42:57.001 kid1| 5,5| ConnOpener.cc(467) timeout: conn85 local=20.20.30.5 remote=10.120.125.146:443 FIRSTUP_PARENT flags=1: * - ERR took too long to receive response.

2024/03/05 22:42:57.001 kid1| 48,5| AsyncCall.cc(97) ScheduleCall: ConnOpener.cc(160) will call HappyConnOpener::notePrimeConnectDone(conn85 local=20.20.30.5 remote=10.120.125.146:443 FIRSTUP_PARENT flags=1, errno=110, flag=-4, data="" [call403]

2024/03/05 22:42:57.001 kid1| 93,5| AsyncJob.cc(85) mustStop: Comm::ConnOpener will stop, reason: Comm::ConnOpener::timeout

2024/03/05 22:42:57.001 kid1| 93,5| AsyncJob.cc(140) callEnd: Comm::ConnOpener::timeout(conn85 local=20.20.30.5 remote=10.120.125.146:443 FIRSTUP_PARENT flags=1, data="" ends job [Stopped, reason:Comm::ConnOpener::timeout job76]

2024/03/05 22:42:57.001 kid1| 5,4| ConnOpener.cc(176) cleanFd: ; temp FD 45

2024/03/05 22:42:57.001 kid1| 5,5| ModEpoll.cc(118) SetSelect: FD 45, type=2, handler=0, client_data=0, timeout=0

2024/03/05 22:42:57.001 kid1| 5,5| comm.cc(1048) comm_remove_close_handler: comm_remove_close_handler: FD 45, AsyncCall=0x557f2c68ecd0*2

2024/03/05 22:42:57.001 kid1| 5,4| AsyncCall.cc(60) cancel: will not call Comm::ConnOpener::earlyAbort [call418] because comm_remove_close_handler

2024/03/05 22:42:57.001 kid1| 5,3| comm.cc(877) _comm_close: start closing FD 45 by ConnOpener.cc:233

2024/03/05 22:42:57.001 kid1| 5,3| comm.cc(558) commUnsetFdTimeout: Remove timeout for FD 45

2024/03/05 22:42:57.001 kid1| 5,5| comm.cc(739) commCallCloseHandlers: commCallCloseHandlers: FD 45

 

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users