|
Hello All, I apologise for asking another squid performance question, but I have been banging my head against the wall for the better part of three months. Squid is installed and working. However, over time it slows down significantly. I have tried
everything from turning off caching to trying to load balance several squid machines. Would someone mind giving my config a quick look. Hardware: Centos 7 32GB Ram Xeon E5 4 cores 4x 140G SAS 15k (Cache) 2x 64G SSD (OS mirror) 10G Network connection Users: about 10k devices random web traffic squid.conf scrubbed # RADIUS Config auth_param basic program /usr/lib64/squid/basic_radius_auth -h 10.10. -w password auth_param basic children 2 auth_param basic realm BCS External Proxy auth_param basic credentialsttl 24 hour # ACL Lists acl ACL_All src all acl ACL_Password proxy_auth REQUIRED acl ACL_SSL_Ports port 80 443 1443 7446 8443 acl ACL_Safe_Ports port "/etc/squid/ports.conf" acl ACL_Connect method CONNECT acl ACL_Purge method PURGE acl ACL_Do_Not_Cache dstdomain "/etc/squid/lists.conf" acl ACL_Query urlpath_regex cgi-bin \? acl ACL_Deny_Url dstdomain "/etc/squid/deny.conf" acl ACL_Allow_Url dstdomain "/etc/squid/allow.conf" acl ACL_Web_Filter dst 10.10.18.1/32 acl ACL_Beavercreek_Clients src 10.10.0.0/16 172.16.0.0/16 acl ACL_Beavercreek_Networks dst 10.10.0.0/16 172.16.0.0/16 acl ACL_MVECA_Networks dst 10.3.0.0/16 acl ACL_Manager proto cache_object acl ACL_Apple dstdomain .appldnld.apple.com .gspe19.ls.apple.com .init-p01md.apple.com .init-p01st.push.apple.com .init.ess.apple.com .iosapps.itunes.apple.com
.mesu.apple.com .pancake.apple.com .phobos.apple.com .ocsp.apple.com acl ACL_Apple_Dest dst 10.10.18.31/32 10.10.18.32/32 10.10.18.33/32 acl ACL_School_Hours time MTWHF 07:30-16:00 acl ACL_Block_Apps urlpath_regex -i \.ipa(\?.*)?$ \.mobileconfig(\?.*)?$ \.plist(\?.*)?$ #acl ACL_Block_IOS urlpath_regex -i acl ACL_Full_Speed src 10.10.0.0/16 172.16.160.0/20 # Delay Pools delay_pools 2 delay_class 1 2 delay_parameters 1 -1/-1 -1/-1 delay_access 1 allow ACL_Full_Speed delay_access 1 deny ACL_All delay_class 2 2 delay_parameters 2 -1/-1 200000/200000 delay_access 2 allow ACL_Apple ACL_School_Hours delay_access 2 allow ACL_Apple_Dest ACL_School_Hours delay_access 2 deny ACL_All delay_initial_bucket_level 50 # Access Rules http_access allow ACL_Web_Filter #http_access deny ACL_Block_IOS http_access allow ACL_Apple_Dest http_access allow ACL_Allow_Url http_access allow ACL_Apple http_access deny ACL_Deny_Url http_access deny !ACL_Safe_Ports http_access deny ACL_Block_Apps http_access allow ACL_Connect ACL_SSL_Ports http_access deny ACL_Connect !ACL_SSL_Ports http_access allow ACL_Beavercreek_Clients http_access allow ACL_Purge ACL_Beavercreek_Clients http_access allow ACL_Beavercreek_Clients ACL_Manager http_access allow ACL_Password http_access deny !ACL_Password http_access deny ACL_All #Forward forwarded_for truncate via on #Do not cache rules #cache deny all always_direct allow ACL_Do_Not_Cache ACL_Apple_Dest ACL_Apple ACL_Beavercreek_Networks ACL_MVECA_Networks cache deny ACL_Do_Not_Cache ACL_Query ACL_Apple ACL_Apple_Dest ACL_Beavercreek_Networks ACL_MVECA_Networks # Network Info http_port 10.10.:8888 http_port 10.10.:3128 #Worker info workers 4 cpu_affinity_map process_numbers=1,2,3,4 cores=1,3,5,7 # DNS Config dns_v4_first on dns_nameservers 10.10. 10.10. append_domain xxx # Cache config cache_mem 16 GB cache_effective_user squid cache_effective_group squid
memory_cache_mode always
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
max_open_disk_fds 0 maximum_object_size_in_memory 1 MB maximum_object_size 100 MB cache_swap_high 95
cache_swap_low 90 max_filedesc 16384 fqdncache_size 8192 ipcache_size 8192 # MISC Settings visible_hostname xxx cache_mgr xxx logfile_rotate 1 half_closed_clients off dead_peer_timeout 30 second dns_timeout 5 second connect_timeout 30 second shutdown_lifetime 10 second server_persistent_connections off authenticate_ttl 1 hour authenticate_ip_ttl 1 hour #ignore_expect_100 on reply_header_max_size 128 KB # Uncomment and adjust the following to add a disk cache directory. cache_dir rock /var/spool/squid 20480 min-size=1 max-size=31000 max-swap-rate=100 swap-timeout=1000 if ${process_number} = 1 cache_dir diskd /squid/data1/aufs 51200 32 256 min-size=31001 max-size=104857600 endif if ${process_number} = 2 cache_dir diskd /squid/data2/aufs 51200 32 256 min-size=31001 max-size=104857600 endif if ${process_number} = 3 cache_dir diskd /squid/data3/aufs 51200 32 256 min-size=31001 max-size=104857600 endif if ${process_number} = 4 cache_dir diskd /squid/data4/aufs 51200 32 256 min-size=31001 max-size=104857600 endif logformat squid %tl.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %<a %mt access_log stdio:/var/log/squid/access.log squid # Leave coredumps in the first cache dir coredump_dir /var/spool/squid # We recommend you to use at least the following line. #hierarchy_stoplist cgi-bin ? # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 #Custom error URL error_directory /etc/squid/custom Info: HTTP/1.1 200 OK Server: squid/3.5.20 Mime-Version: 1.0 Date: Thu, 11 Jan 2018 17:04:50 GMT Content-Type: text/plain Expires: Thu, 11 Jan 2018 17:04:50 GMT Last-Modified: Thu, 11 Jan 2018 17:04:50 GMT Connection: close Squid Object Cache: Version 3.5.20 Build Info: Service Name: squid Start Time: Thu, 11 Jan 2018 03:26:58 GMT Current Time: Thu, 11 Jan 2018 17:04:50 GMT Connection information for squid: Number of clients accessing cache: 35529 Number of HTTP requests received: 4231669 Number of ICP messages received: 0 Number of ICP messages sent: 0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 5174.0 Average ICP messages per minute since start: 0.0 Select loop called: 299015711 times, 12.376 ms avg Cache information for squid: Hits as % of all requests: 5min: 5.4%, 60min: 3.6% Hits as % of bytes sent: 5min: 2.2%, 60min: 2.6% Memory hits as % of hit requests: 5min: 21.2%, 60min: 15.4% Disk hits as % of hit requests: 5min: 47.1%, 60min: 40.2% Storage Swap size: 60801452 KB Storage Swap capacity: 26.4% used, 73.6% free Storage Mem size: 2005632 KB Storage Mem capacity: 12.0% used, 88.0% free Mean Object Size: 119.80 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min 60 min: HTTP Requests (All): 0.90173 0.47329 I have seen these go as high as 20 seconds
Cache Misses: 0.04781 0.06103 Cache Hits: 0.00000 0.00000 Near Hits: 0.02599 0.04127 Not-Modified Replies: 0.00000 0.00000 DNS Lookups: 0.00019 0.00056 ICP Queries: 0.00000 0.00000 Resource usage for squid: UP Time: 49072.590 seconds CPU Time: 4228.270 seconds CPU Usage: 8.62% CPU Usage, 5 minute avg: 10.12% CPU Usage, 60 minute avg: 15.49% Maximum Resident Size: 25868624 KB Page faults with physical i/o: 3 Memory accounted for: Total accounted: 212706 KB memPoolAlloc calls: 647869201 memPoolFree calls: 656372502 File descriptor usage for squid: Maximum number of file descriptors: 81920 Largest file desc currently in use: 10848 Number of file desc currently in use: 9169 Files queued for open: 0 Available number of file descriptors: 72751 Reserved number of file descriptors: 500 Store Disk files open: 1 Internal Data Structures: 162462 StoreEntries 291 StoreEntries with MemObjects 62657 Hot Object Cache Items 507526 on-disk objects 60Min: HTTP/1.1 200 OK Server: squid/3.5.20 Mime-Version: 1.0 Date: Thu, 11 Jan 2018 17:06:23 GMT Content-Type: text/plain Expires: Thu, 11 Jan 2018 17:06:23 GMT Last-Modified: Thu, 11 Jan 2018 17:06:23 GMT Connection: close sample_start_time = 1515686758.822127 (Thu, 11 Jan 2018 16:05:58 GMT) sample_end_time = 1515690358.901608 (Thu, 11 Jan 2018 17:05:58 GMT) client_http.requests = 184.169353/sec client_http.hits = 7.716049/sec client_http.errors = 13.524058/sec client_http.kbytes_in = 549.705029/sec client_http.kbytes_out = 13344.171474/sec client_http.all_median_svc_time = 0.499589 seconds client_http.miss_median_svc_time = 0.061028 seconds client_http.nm_median_svc_time = 0.000000 seconds client_http.nh_median_svc_time = 0.041120 seconds client_http.hit_median_svc_time = 0.000000 seconds server.all.requests = 164.990063/sec server.all.errors = 0.000000/sec server.all.kbytes_in = 13006.455302/sec server.all.kbytes_out = 510.372012/sec server.http.requests = 50.421262/sec server.http.errors = 0.000000/sec server.http.kbytes_in = 2179.449438/sec server.http.kbytes_out = 69.110279/sec server.ftp.requests = 0.000000/sec server.ftp.errors = 0.000000/sec server.ftp.kbytes_in = 0.000000/sec server.ftp.kbytes_out = 0.000000/sec server.other.requests = 114.568802/sec server.other.errors = 0.000000/sec server.other.kbytes_in = 10827.005308/sec server.other.kbytes_out = 441.261456/sec icp.pkts_sent = 0.000000/sec icp.pkts_recv = 0.000000/sec icp.queries_sent = 0.000000/sec icp.replies_sent = 0.000000/sec icp.queries_recv = 0.000000/sec icp.replies_recv = 0.000000/sec icp.replies_queued = 0.000000/sec icp.query_timeouts = 0.000000/sec icp.kbytes_sent = 0.000000/sec icp.kbytes_recv = 0.000000/sec icp.q_kbytes_sent = 0.000000/sec icp.r_kbytes_sent = 0.000000/sec icp.q_kbytes_recv = 0.000000/sec icp.r_kbytes_recv = 0.000000/sec icp.query_median_svc_time = 0.000000 seconds icp.reply_median_svc_time = 0.000000 seconds dns.median_svc_time = 0.000557 seconds unlink.requests = 0.000000/sec page_faults = 0.000833/sec select_loops = 12880.416435/sec select_fds = 9811.879014/sec average_select_fd_period = 0.000000/fd median_select_fds = 0.000000 swap.outs = 8.875762/sec swap.ins = 8.034935/sec swap.files_cleaned = 0.000000/sec aborted_requests = 1.342211/sec syscalls.disk.opens = 3.490250/sec syscalls.disk.closes = 3.489972/sec syscalls.disk.reads = 76.404351/sec syscalls.disk.writes = 104.594693/sec syscalls.disk.seeks = 0.000000/sec syscalls.disk.unlinks = 0.146943/sec syscalls.sock.accepts = 569.969450/sec syscalls.sock.sockets = 197.239195/sec syscalls.sock.connects = 164.998952/sec syscalls.sock.binds = 0.000000/sec syscalls.sock.closes = 363.784801/sec syscalls.sock.reads = 4398.237408/sec syscalls.sock.writes = 4622.031167/sec syscalls.sock.recvfroms = 118.198756/sec syscalls.sock.sendtos = 61.557552/sec cpu_time = 558.603203 seconds wall_time = 18000.159289 seconds cpu_usage = 3.103324% tail end of cache.log [root@proxy ~]# tail -f /var/log/squid/cache.log 2018/01/11 12:05:39 kid3| urlParse: URL too large (12594 bytes) 2018/01/11 12:05:39 kid3| urlParse: URL too large (12602 bytes) 2018/01/11 12:05:47 kid1| fqdncacheParse: No PTR record for '91.212.150.79' 2018/01/11 12:06:10 kid3| urlParse: URL too large (12720 bytes) 2018/01/11 12:06:20 kid4| urlParse: URL too large (13122 bytes) 2018/01/11 12:06:20 kid3| urlParse: URL too large (13166 bytes) 2018/01/11 12:06:32 kid3| urlParse: URL too large (12599 bytes) 2018/01/11 12:06:34 kid3| urlParse: URL too large (12598 bytes) 2018/01/11 12:06:52 kid3| urlParse: URL too large (12724 bytes) 2018/01/11 12:06:53 kid3| urlParse: URL too large (12636 bytes) I should also mention this proxy is behind a web content filter which I believe may be running proxy services. Any help would be appreciated. Thanks, Brian Snyder |
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
