On Mon, 21 Nov 2011 11:37:43 +0000, Alex Sharaz wrote:
Hi,
I've just move my squid 3.1.16 web caches over to using rsyslog (
Ubuntu 10.4 LTS OS) to move log files over to a centralised syslog
server for storage in a mysql database. Most of the time it works
just
fine. Unfortunately I do seem to be seeing some blocking occurring
where a cache isn't accepting new inbound client connections. I've
got
6 webcaches configured in 2 clusters of 3. When a problem occurs, I
can see about 50 - 100 concurrent connections on caches with the
problem, and 10 - 12K connections on the remaining normaly operating
ones. A restart of the rsyslog daemon on a problematic cache cures
the
problem for a while, but it can come back.
Um,
* take a measure of the typical (mean) log line length produced by
your format.
* take the count of *requests* per second (not connections,
*requests*) across all of your Squid instances which are logging to this
rsyslog.
* multiply those numbers together.
What that gives you is the UDP bandwidth requirements rsyslog is being
expected to handle.
Between modern hardware speeds, squid req/sec capacity, and long URLs
this could be resulting in a great many MB per second of UDP packets
being thrown at rsyslog for disk storage.
From my squid.conf file
logformat hsyslog
%tg,%ts.%tu,%>a,%la,150.237.199.249,%ul,%rm,HTTP/%rv,
%>Hs,%<st,%tr,%ru,%Ss:%Sh
<snip standard documentation>
#access_log /logs/access.log hcommon
access_log syslog:local0.info hsyslog
and from the rsyslog.d directory
$WorkDirectory /logs/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 5g # space limit (use as much as
possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* @@150.237.85.216:514
Squid build with
#!/bin/bash
ulimit -SHn 49152
./configure --enable-snmp --enable-basic-auth-helpers="PAM" --
enable-cachemgr-hostname=wwwcache2-west.hull.ac.uk --enable-htcp --
enable-cache-digests --enable-async-io --prefix=/usr/local/squid --
with-pthreads --enable-removal-policies --enable-ssl -with-openssl=/
usr/local/ssl --disable-linux-netfilter -with-large-files --with-
maxfd=49152 --with-dl --enable-icmp --enable-poll --disable-ident-
lookups --enable-truncate --disable-delay-pools --disable-ipv6 --
disable-loadable-modules
root@wwwcache2-west:/usr/local/src/squid-3.1.16#
Anything I can change in the build to stop this blocking from
happennig?
Squid uses the system syslog() call. It is up to your OS whether that
is a blocking or non-blocking call. It sounds to me like your OS is
blocking, although Debian based OS are not supposed to be blocking.
Amos
