On 24/07/2013 6:29 p.m., mdecheser@xxxxxxxxxxx wrote:
Hello Squid Users --
I'm very new to running squid. I've quickly learned that there are many parameters to configure and therefore many places problems can occur.
FYI: quite a few will be picked up by squid -k parse.
I am prototyping a squid environment on a CentOS 6.4 32-bit system with 256MB RAM. Presently, I'm observing strange behavior with the proxy, and the most notable symptom is that either the proxy server becomes unavailable after a period of time or DNS names stop resolving. I should mention that I'm also sending the traffic over a Poptop (pptpd) VPN tunnel, but these issues persist even without the tunnel up as far as I can tell.
Info from the environment:
# squid -v
Squid Cache: Version 3.1.10
3.1 is quite old now. A lot has improved since then. Please consider
upgrading if you can.
configure options: '--build=i386-redhat-linux-gnu' '--host=i386-redhat-linux-gnu' '--target=i686-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-internal-dns' '--disable-strict-error-checking' '--exec_prefix=/usr' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-arp-acl' '--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL,DB,POP3,squid_radius_auth' '--enable-ntlm-auth-helpers=smb_lm,no_check,fakeauth' '--enable-digest-auth-helpers=password,ldap,eDirectory' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--with-large-files' '--enable-linux-netfilter' '--enable-referer-log' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl' '--enable-storeio=aufs,diskd,ufs' '--enable-useragent-log' '--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=squid' '--with-filedescriptors=16384' '--with-dl' '--with-openssl' '--with-pthreads' 'build_alias=i386-redhat-linux-gnu' 'host_alias=i386-redhat-linux-gnu' 'target_alias=i686-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -fpie' 'LDFLAGS=-pie' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -fpie' --with-squid=/builddir/build/BUILD/squid-3.1.10
squid.conf:
http_port 3128 intercept
http_port 4334
http_port 127.0.0.1:33699
http_port 127.0.0.1:32623
http_port 127.0.0.1:26226
http_port 127.0.0.1:26499
http_port 127.0.0.1:18108
http_port 127.0.0.1:49236
http_port MY.PUB.IP.ADR:3128
You have already opened port 3128 on all IP addresses the box has, up
there with "intercept" flag on it. That is what the commBind error in
your log is about.
Your main forward-proxy port is 4334 on all IP addresses.
FWIW: I suggest that you swap those around so 3128 is your main port and
4334 is the one receiving the intercepted traffic.
icp_port 0
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
acl apache rep_header Server ^Apache
cache_mem 128 MB
maximum_object_size 1024 KB
maximum_object_size_in_memory 16 KB
cache_dir aufs /opt/squid/cache 16384 64 256
error_directory /opt/squid/logs
Er. You have your error page templates in a directory called "logs" ??
Probably remove that line completely for now. The defaults should be
able to work well enough until you read up on it a bit.
allow_underscore off
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl Safe_ports port 80 # http
acl Safe_ports port 443 # https
acl Safe_ports port 21 # ftp
acl Safe_ports port 554 # rtsp
http_access deny !Safe_ports
## authentication
auth_param basic program /usr/lib/squid/squid_db_auth --user ******** --password ******** --plaintext --persist --dsn DBI:mysql:database=the_database
auth_param basic children 5
auth_param basic realm Web-Proxy
auth_param basic credentialsttl 1 minute
auth_param basic casesensitive off
acl db-auth proxy_auth REQUIRED
http_access allow db-auth
http_access deny all
cache_mgr thisisme@xxxxxxxxx
httpd_suppress_version_string on
visible_hostname myserver
dns_nameservers 8.8.8.8 8.8.4.4
cache.log:
After a service start, the daemon will run for a while, and then I see the following events (note the first line):
2013/07/24 09:29:02| assertion failed: errorpage.cc:1064: "(size_t)content->contentSize() == strlen(content->content())"
2013/07/24 09:29:05| Starting Squid Cache version 3.1.10 for i386-redhat-linux-gnu...
Interesting assertion. I dont recall seeing it before. Can you run a 3.3
or later release and see if it is still occuring in the recent code?
Aside from this issue, my main goal is to eventually build a config that handles dynamic content (YouTube and its CDNs, for example). Right now, content coming from YouTube or its CDNs is not loading through my cache, however other sites display content rapidly. Example: content from cnn.com displays perfectly, including video content from the Turner CDN.
Then you really need to upgrade. Probably to 3.4 when its available.
FWIW your above config in 3.1 is perfectly capable of handling and
caching dynamic content. It will simply not get as high a HIT rate in
those older versions as the newer ones can achieve.
My take on all this is that it appears the server may be starved for memory. I have seen other strange behaviors such mysqld, squid, and pptpd daemons dying randomly. I've seen squid daemon restart for reasons unknown (though the above cache.log suggests a cache digest rebuild every 3600 seconds).
Your Squid requires ~128MB for its memory cached HTTP objects and their
index. Plus all the usual operational memory. plus the OS memory. Then
quite a bit more for the memory leaks which exist in the 3.1 releases.
Are there any obvious mistakes I've made here? I do plan to run my production proxy with more memory and am not in objection to adding more memory to this environment, but would like to do so with an understanding of the issues before moving forward.
Squid is fully capable of servicing a few hundred concurrent clients in
32MB or less provided you are happy with a very small or no cache. I
suspect your problem is the memory leaks, or something like the OS
requiring more memory than is spare from the Squid requirements.
Amos
