- Josh
-----Original Message-----
From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of Amos Jeffries
Sent: Monday, May 6, 2024 12:59 PM
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Linux Noob - Squid Config
Caution: This email originated from outside of Hexcel. Do not click links or open attachments unless you recognize the sender and know the content is safe.
[ please keep responses on-list to assist any others who encounter the same issues in future ]
On 4/05/24 08:51, Piana, Josh wrote:
Hey Amos,
Thank you so much for getting back to me so quickly!
To answer your question about NTLM, I meant to say NTLMv2. We're trying to become compliant with newer security standards and this old box in depersate need of some love and updating.
Hmm. My question was more aiming as a yes/no answer.
Squid can certainly still support NTLM. But if possible going to just Negotiate/Kerberos auth would be a simpler config.
The /usr/bin/ntlm_auth authenticator you have been using is provided by Samba. So you will need to have Samba installed (yum install samba) and configured the same (or equivalent for its upgrade) as before Squid authentication is usable.
FYI; Modern Squid start helpers only as-needed. Meaning Squid will startup and run fine without a working auth helper ... until the point where a helper lookup is needed. So you can test Squid with some trivial requests before needing Samba fully working.
--------------------------------------------------------------
Current squid.conf file Output:
max_filedesc 4096
I advise changing this to at least:
max_filedescriptors 65536
Why? Modern web pages can cause clients to open up to a hundred connections to various servers to display a single web page. Each client of those connection consumes 3-4 file descriptors.
You will also need to check the OS limitation to ensure
cache_mgr itadmin@...
cache_effective_user squid
cache_effective_group squid
coredump_dir /opt/squid/var
pid_filename /var/run/squid.pid
shutdown_lifetime 5 seconds
error_directory /usr/local/share/squid/errors/English_CUSTOM
Check what customizations have been done to the files inside that directory.
If it is just the new templates for the deny_info lines later in your config; then you can copy those templates to the new machine.
And create symlnks from the
I suggest placing the custom error templates in a directory such as /etc/squid/errors/ and a symlink from the /usr/local/share/squid/errors/templates/ directory (or wherever the templates are put by yum install).
[ This way upgrades that change the default templates will not erase your ones. At worst you should only have to re-create the symlinks manually. ]
(If you need it; to learn how to create symlinks type "man ln".)
logfile_rotate 0
debug_options ALL,1
You can remove the above line. It is a default setting.
buffered_logs on > cache_log /var/log/squid/general> cache_access_log
/var/log/squid/access
The above two lines should be more like:
cache_log /var/log/squid/cache.log
access_log daemon:/var/log/squid/access.log
cache_store_log none
log_mime_hdrs off
The above two lines can be removed. They are default settings.
log_fqdn off
Remove this line. It is not supported in modern Squid.
strip_query_terms off
http_port 10.46.11.20:8080
http_port 127.0.0.1:3128
icp_port 0
The above line can be removed. It is a default setting.
forwarded_for off
Change that "off" to;
* "delete" for complete removal of the header), or
* "transparent" for Squid to not add the header.
ftp_user anonftpuser@...
ftp_list_width 32
ftp_passive on
connect_timeout 30 seconds
peer_connect_timeout 20 seconds
read_timeout 2 minutes
request_timeout 2 minutes
persistent_request_timeout 30 seconds
cache_dir ufs /var/cache/squid/ufs/squid 16000 64 64
cache_replacement_policy heap LFUDA memory_replacement_policy lru
cache_mem 200 MB maximum_object_size 32 MB
maximum_object_size_in_memory 128 KB quick_abort_min 16 KB
quick_abort_max 1 MB quick_abort_pct 90 range_offset_limit 64 KB acl
no_cache_url url_regex -i "/etc/squid/no_cache_url"
cache deny no_cache_url
Modern Squid define a set of "refresh_pattern" to fixup messages cacheability inline with HTTP/1.1 caching requirements.
Please add these lines:
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
acl blows_chunks dstdomain .blr.com
header_access Accept-Encoding deny blows_chunks
This is something to look into. Be aware of how exactly that server is broken when it receives "Accept-Encoding" headers.
The old Squid only supported HTTP/1.0. Your new Squid supports HTTP/1.1 which may work better with whatever that server was doing to be considered bad.
auth_param ntlm program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-ntlmssp
auth_param ntlm children 30
auth_param ntlm keep_alive on
auth_param basic program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-basic auth_param basic children 5
auth_param basic realm "..."
auth_param basic credentialsttl 2 hours acl authenticated proxy_auth
REQUIRED acl all src 0.0.0.0/0.0.0.0
Remove the above line. The "all" ACL is now built-in.
acl src_self src 127.0.0.0/255.0.0.0
acl src_self src 10.46.11.20
Modern Squid provide these as a built-in "localhost" ACL.
To add the machines global IP address as part of localhost, replace the above two lines with:
acl localhost src 10.46.11.20/32
Then replace all uses of "src_self" with "localhost".
acl dst_self dst 127.0.0.0/255.0.0.0
acl dst_self dst 10.46.11.20
Modern Squid provide these as a built-in "to_localhost" ACL.
To add the machines global IP address as part of localhost, replace the above two lines with:
acl to_localhost dst 10.46.11.20/32
Then replace all uses of "dst_self" with "to_localhost".
acl from_arc src 10.0.0.0/255.255.0.0
acl from_arc src 10.46.0.0/255.254.0.0 acl local_dst_addr dst
10.0.0.0/255.0.0.0
FYI; while the above is acceptible to Squid. Prefer writing those as CIDR masks instead of NetMask syntax.
eg. 10.0.0.0/8
acl local_dst_addr dst bldg3.*.com.
acl local_dst_addr dst bldg5.*.com.
acl local_dst_dom dstdomain arcgate
The above is not a domain name, nor wildcard domain.
FYI; Squid will attempt to apply a domain search using the "domain" or "search" as configured in /etc/resolv.conf and resolve arcgate.* in DNS.
acl proto_FTP proto FTP
acl proto_HTTP proto HTTP
acl http_ports port 80
acl http_ports port 81
acl http_ports port 82
acl http_ports port 807
acl http_ports port 8000
acl http_ports port 8001
acl http_ports port 8080
acl http_ports port 8081
acl http_ports port 9000
acl ssl_ports port 443
acl ssl_ports port 818
acl ssl_ports port 4435
acl ssl_ports port 9571
acl ssl_ports port 9030
acl ssl_ports port 4502
acl ssl_ports port 8080
acl ssl_ports port 8081
acl ssh_ports port 22 1776 8217
acl ftp_ports port 21
acl method_CONNECT method CONNECT
The "CONNECT" ACL is provided as a built-in.
Replace all uses of "method_CONNECT" with just "CONNECT" and remove the above line.
acl methods_std method GET HEAD POST PUT DELETE acl methods_std method
TRACE OPTIONS acl purge method PURGE http_access allow purge src_self
http_access deny purge
This purge is something to consider carefully. You should not need to ever use it on a properly working cache that obeys HTTP standard.
If possible prefer to remove the http_access and acl lines relating to the curstom "PURGE" method. This allows modern Squid to improve performance by not tracking a lot of things.
acl cache_manager proto cache_object
The "manager" ACL is provided as built-in by modern Squids, and has a different definition for the latest access mechanism(s).
Replace all uses of "cache_manager" with "manager" and remove the above line.
cachemgr_passwd disabled shutdown offline_toggle cachemgr_passwd none
all
http_access allow cache_manager src_self http_access deny
cache_manager
Per earlier change requests this should become:
http_access allow manager localhost
http_access deny manager
http_access deny dst_self
http_access deny src_self
Per earlier change requests this should become:
http_access deny localhost
http_access deny to_localhost
http_access deny !from_arc
http_access allow local_dst_dom
http_reply_access allow local_dst_dom
http_access allow local_dst_addr
http_reply_access allow local_dst_addr
acl authless_src src "/etc/squid/authless_src"
http_access allow authless_src
http_reply_access allow authless_src
acl authless_dst dstdomain "/etc/squid/authless_dst"
http_access allow authless_dst
http_reply_access allow authless_dst
acl bad_domains_preauth dstdomain "/etc/squid/bad_domains_preauth"
http_access deny bad_domains_preauth
http_access deny !authenticated
acl block_user proxy_auth_regex -i "/etc/squid/block_user"
http_access deny block_user
acl bad_exception_urls url_regex -i "/etc/squid/bad_exception_urls"
acl exec_files url_regex -i "/etc/squid/exec_files"
acl exec_users proxy_auth_regex -i "/etc/squid/exec_users"
http_access deny !bad_exception_urls !exec_users exec_files deny_info
ERR_BLOCK_TYPE exec_files
acl mmedia_users proxy_auth_regex -i "/etc/squid/mmedia_users"
acl mmedia_sites dstdomain "/etc/squid/mmedia_sites"
http_access allow methods_std proto_HTTP http_ports mmedia_sites mmedia_users
http_reply_access allow methods_std proto_HTTP http_ports mmedia_sites mmedia_users
http_access allow method_CONNECT ssl_ports mmedia_sites mmedia_users
http_reply_access allow method_CONNECT ssl_ports mmedia_sites mmedia_users
acl bad_domains dstdomain "/etc/squid/bad_domains"
http_access deny !bad_exception_urls bad_domains
deny_info ERR_BLOCK_DST bad_domains
acl bad_domains_regex dstdom_regex -i "/etc/squid/bad_domains_regex"
http_access deny !bad_exception_urls bad_domains_regex
deny_info ERR_BLOCK_DST bad_domains_regex
acl bad_urls url_regex -i "/etc/squid/bad_urls"
http_access deny !bad_exception_urls bad_urls
deny_info ERR_BLOCK_DST bad_urls
acl bad_files urlpath_regex -i "/etc/squid/bad_files"
http_access deny !bad_exception_urls bad_files deny_info
ERR_BLOCK_TYPE bad_files
acl bad_types rep_mime_type -i "/etc/squid/bad_types"
http_reply_access deny bad_types !bad_exception_urls deny_info
ERR_BLOCK_TYPE bad_types
acl fsoguest_user proxy_auth_regex -i fsoguest acl fsoguest_dst
dstdomain .opm.gov acl fsoguest_dst dstdomain .google-analytics.com
acl fsoguest_dst dstdomain pki.google.com acl fsoguest_dst dstdomain
ajax.googleapis.com acl fsoguest_dst dstdomain fonts.googleapis.com
acl fsoguest_dst dstdomain html5shiv.googlecode.com acl fsoguest_dst
dstdomain fonts.gstatic.com acl fsoguest_dst dstdomain
clients1.google.com acl fsoguest_dst dstdomain ajax.microsoft.com acl
fsoguest_dst dstdomain ajax.aspnetcdn.com acl fsoguest_dst dstdomain
.geotrust.com acl fsoguest_dst dstdomain .akamaihd.net acl
fsoguest_dst dstdomain symcd.com http_access allow methods_std
proto_HTTP http_ports fsoguest_dst fsoguest_user
http_access allow method_CONNECT ssl_ports fsoguest_dst fsoguest_user
http_access deny fsoguest_user
http_access allow http_ports proto_HTTP methods_std
acl ssh_users proxy_auth -i PCADMIN
acl ssh_users proxy_auth -i BSCOTT
http_access allow method_CONNECT ssh_ports ssh_users
acl ssh_dst dst vthm.com
http_access allow method_CONNECT ssh_ports ssh_dst
acl ftp_dst dst
http://www.a/
rc-tech.com%2F&data=05%7C02%7Cjosh.piana%40hexcel.com%7C69861412949747
94b68b08dc6deddb54%7C4248050df19546d5ac9c0c7c52b04cae%7C0%7C0%7C638506
115567977267%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=Rj6hxBCYCNqITDlpJ
QiS%2B8TlfOX6z6DbmVNOfHeD7sU%3D&reserved=0
I hoep that above line is a typo. A full URL is not valid for a "dst"
ACL type.
The "dst" ACL values can be;
* a raw-IP, or
* a CIDR range, or
* a NetMask range (eg A-B/mask), or
* a hostname which is resolvable in DNS at Squid startup/reconfigure time.
acl ftp_dst dst ftp.telemeter.de
acl ftp_dst dst ftp.lucasindustries.com acl ftp_dst dst
ftp-upload.trendmicro.com http_access allow method_CONNECT ftp_dst
http_access allow method_CONNECT ssl_ports http_access deny
method_CONNECT
http_access allow ftp_ports proto_FTP
http_access deny all
http_reply_access allow all
After making the above adjustments with Squid installed you should be aboe to run the command "squid -k parse -f /etc/squid/squid.conf.old" to see what the new Squid thinks of the old configuration.
Any "ERROR" lines that are displayed by by that command will need fixing before you can use Squid.
Any "WARNING" or "NOTICE" or "UPGRADE" are things that should be looked into fixing. But not urgent enough to make Squid unusable.
You may need to run "squid -z -f /etc/squid/squid.conf.old" to create the cache directory.
When that is fine you can move the /etc/squid/squid.conf.old to replace the /etc/squid/squid.conf and use the OS method for starting Squid.
Which for RHEL 9 I believe should be:
service squid start
... then start testing that what Squid does actually meets your expectations.
HTH
Amos
--------------------------------------------------------------
Current Box squid -v Output:
Squid Cache: Version 2.6.STABLE21
configure options: '--build=i386-redhat-linux-gnu' '--host=i386-redhat-linux-gnu' '--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--includedir=/usr/include' '--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--sharedstatedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' '--datadir=/usr/share' '--sysconfdir=/etc/squid' '--enable-arp-acl' '--enable-epoll' '--enable-snmp' '--enable-removal-policies=heap,lru' '--enable-storeio=aufs,coss,diskd,null,ufs' '--enable-ssl' '--with-openssl=/usr/kerberos' '--enable-delay-pools' '--enable-linux-netfilter' '--with-pthreads' '--enable-ntlm-auth-helpers=SMB,fakeauth' '--enable-external-acl-helpers=ip_user,ldap_group,unix_group,wbinfo_group' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-digest-auth-helpers=password' '--with-winbind-auth-challenge' '--enable-useragent-log' '--enable-referer-log' '--disable-dependency-tracking' '--enable-cachemgr-hostname=localhost' '--enable-underscores' '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL' '--enable-cache-digests' '--enable-ident-lookups' '--with-large-files' '--enable-follow-x-forwarded-for' '--enable-wccpv2' '--enable-fd-config' '--with-maxfd=16384' 'build_alias=i386-redhat-linux-gnu' 'host_alias=i386-redhat-linux-gnu' 'target_alias=i386-redhat-linux-gnu' 'CFLAGS=-D_FORTIFY_SOURCE=2 -fPIE -Os -g -pipe -fsigned-char' 'LDFLAGS=-pie'
--------------------------------------------------------------
New Box squid -v Output:
Squid Cache: Version 5.5
Service Name: squid
This binary uses OpenSSL 3.0.7 1 Nov 2022. For legal restrictions on
distribution see
https://www/.
openssl.org%2Fsource%2Flicense.html&data=05%7C02%7Cjosh.piana%40hexcel
.com%7C6986141294974794b68b08dc6deddb54%7C4248050df19546d5ac9c0c7c52b0
4cae%7C0%7C0%7C638506115567977267%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sd
ata=SYACR6XE7IVCwGsCWdigcKjas80B5luo%2BHi8F6s5SFo%3D&reserved=0
configure options: '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--libexecdir=/usr/lib64/squid' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/run/squid.pid' '--disable-dependency-tracking' '--enable-eui' '--enable-follow-x-forwarded-for' '--enable-auth' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,PAM,POP3,RADIUS,SASL,SMB,SMB_LM' '--enable-auth-ntlm=SMB_LM,fake' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos' '--enable-external-acl-helpers=LDAP_group,time_quota,session,unix_group,wbinfo_group,kerberos_ldap_group' '--enable-storeid-rewrite-helpers=file' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,ufs,rock' '--enable-diskio' '--enable-wccpv2' '--enable-esi' '--enable-ecap' '--with-aio' '--with-default-user=squid' '--with-dl' '--with-openssl' '--with-pthreads' '--disable-arch-native' '--disable-security-cert-validators' '--disable-strict-error-checking' '--with-swapdir=/var/spool/squid' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CC=gcc' 'CFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-prot
