Search squid archive

Re: logfileHandleWrite: daemon:/var/log/squid/access.log: error writing ((32) Broken pipe)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



> pid_filename /var/run/squid2.pid

>   postrotate
>   test ! -e /var/run/squid.pid || ... /usr/sbin/squid -k rotate
>   endscript

I spotted one more (potentially critical) problem: Your Squid configuration sets pid_filename to /var/run/squid2.pid but your logrotate configuration assumes Squid uses /var/run/squid.pid.

IMHO, in general, it is best not to guess where Squid has its PID if you are using "squid -k ...". If you want to test whether Squid is currently running, try using "squid -k check" instead.


HTH,

Alex.



On 9/6/22 20:45, Alex Rousskov wrote:
On 9/6/22 18:02, roee klinger wrote:
it seems that the logs has filled over 100GB of log data, since I made a configuration mistake (I think?) by setting this:

    logfile_rotate 0

This is correct setting when using an external log rotation tool like the logrotate daemon. More on that below.


If I remember and read correctly, this means that the rotation of the files is disabled and they will just keeping increasing
in size if left unchecked.

To be more precise, this means that you are relying on an external tool to rename the log files. With this setting, Squid rotate command closes the access log and opens a new one (under the same name). While that might sound useless, it is the right (and necessary) thing for Squid to do when combined with the correct external log rotation setup.


I have now gone ahead and changed all the configuration file to this setting:

    logfile_rotate 1

So now it should rotate once daily, and on the next rotation it should be deleted, and this is all handled by logrotate on Debian-based machines?

AFAIK, if you are using an external (to Squid) tool like logrotate, you should be setting logfile_rotate to zero.


This is my / cat /etc/logrotate.d/squid:
➜ / cat /etc/logrotate.d/squid
#
# Logrotate fragment for squid.
#
/var/log/squid/*.log {
  daily
  compress
  delaycompress
  rotate 2
  missingok
  nocreate
  sharedscripts
  prerotate
  test ! -x /usr/sbin/sarg-reports || /usr/sbin/sarg-reports daily
  endscript
  postrotate
  test ! -e /var/run/squid.pid || test ! -x /usr/sbin/squid || /usr/sbin/squid -k rotate
  endscript
}

This is not my area of expertise, but the above configuration does not look 100% correct to me: sarg-reports execution failures should have no effect on log rotation but does (AFAICT). There may be other problems (e.g., I do not know whether your /usr/sbin/squid finds the right Squid configuration file). I hope sysadmin experts on this mailing list will help you polish this.

You should be able to test whether the above is working (e.g., by asking logrotate to rotate). Testing is critical even if you do end up getting expert log rotation help on this list (this email is not it!).


HTH,

Alex.


Is there a way for me to set it so it just get deleted every 24 or 12 hours without the archive first?

Thanks,
Roee
On 6 Sep 2022, 16:28 +0300, Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>, wrote:
On 9/6/22 07:41, roee klinger wrote:

It is also important to know that I am running multiple Squid instances
on the same machine, they are all getting the error at the same time

What external event(s) happen at that time? Something is probably
sending a signal to the logging daemon process. It would be good to know
what that something (and that signal) is. Your syslog or cache.log might
contain more info. Analyzing the timing/schedule of these problems may
also be helpful in identifying the trigger.


Is a possible workaround that might be just replacing the line with
this?

access_log /var/log/squid/access2.log

As you know, this configuration (in this deprecated spelling or with and
explicit "stdio:" prefix) will result in Squid workers writing to the
log file directly instead of asking the logging daemon. This will,
naturally, get rid of the pipe between workers and their daemons, and
the associated broken pipe error.

or will this cause a problem?

Impossible to say for sure without knowing whether your workers benefit
from the anticipated performance advantages of avoiding blocking file
I/O _and_ whether those advantages are real (in your environment). Too
many variables and too many unknowns. I would treat this as an important
(and potentially disruptive) configuration change and carefully test the
outcome.


HTH,

Alex.


INFO -
Versions:

Squid Cache: Version 4.10
Ubuntu 20.04.4 LTS


Example squid.conf:

visible_hostname squid2

access_log daemon:/var/log/squid/access2.log squid

cache_log /var/log/squid/cache2.log

pid_filename /var/run/squid2.pid


acl localnet src 0.0.0.1-0.255.255.255# RFC 1122 "this" network (LAN)

acl localnet src 10.0.0.0/8 <http://10.0.0.0/8> # RFC 1918 local
private network (LAN)

acl localnet src 100.64.0.0/10 <http://100.64.0.0/10># RFC 6598
shared address space (CGN)

acl localnet src 169.254.0.0/16 <http://169.254.0.0/16> # RFC 3927
link-local (directly plugged) machines

acl localnet src 172.16.0.0/12 <http://172.16.0.0/12># RFC 1918
local private network (LAN)

acl localnet src 192.168.0.0/16 <http://192.168.0.0/16> # RFC 1918
local private network (LAN)

acl localnet src fc00::/7 # RFC 4193 local private network range

acl localnet src fe80::/10# RFC 4291 link-local (directly plugged)
machines

acl SSL_ports port 443

acl Safe_ports port 80# http

acl Safe_ports port 21# ftp

acl Safe_ports port 443 # https

acl Safe_ports port 70# gopher

acl Safe_ports port 210 # wais

acl Safe_ports port 1025-65535# unregistered ports

acl Safe_ports port 280 # http-mgmt

acl Safe_ports port 488 # gss-http

acl Safe_ports port 591 # filemaker

acl Safe_ports port 777 # multiling http

acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager

http_access deny manager

# include /etc/squid/conf.d/*

http_access allow localhost

acl aws src *censored*

http_access allow aws

# http_access deny all

tcp_outgoing_address *censored*

http_port 10002

coredump_dir /var/spool/squid

refresh_pattern ^ftp: 144020% 10080

refresh_pattern ^gopher:14400%1440

refresh_pattern -i (/cgi-bin/|\?) 0 0%0

refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0
refresh-ims

refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims

refresh_pattern \/InRelease$ 0 0% 0 refresh-ims

refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims

refresh_pattern . 0 20% 4320


shutdown_lifetime 1 seconds

logfile_rotate 0

max_filedescriptors 16384

dns_nameservers 8.8.8.8 8.8.4.4 1.1.1.1

cache deny all

cache_dir null /tmp

via off

forwarded_for off

request_header_access From deny all

request_header_access Server deny all

request_header_access WWW-Authenticate deny all

request_header_access Link deny all

request_header_access Cache-Control deny all

request_header_access Proxy-Connection deny all

request_header_access X-Cache deny all

request_header_access X-Cache-Lookup deny all

request_header_access Via deny all

request_header_access X-Forwarded-For deny all

request_header_access Pragma deny all

request_header_access Keep-Alive deny all

dns_v4_first on


Example service file:

## Copyright (C) 1996-2020 The Squid Software Foundation and
contributors

##

## Squid software is distributed under GPLv2+ license and includes

## contributions from numerous individuals and organizations.

## Please see the COPYING and CONTRIBUTORS files for details.

##


[Unit]

Description=Squid Web Proxy Server

Documentation=man:squid(8)

After=network.target network-online.target nss-lookup.target


[Service]

Type=forking

PIDFile=/var/run/squid2.pid

ExecStartPre=/usr/sbin/squid --foreground -z -f /etc/squid/squid2.conf

ExecStart=/usr/sbin/squid -sYC -f /etc/squid/squid2.conf

ExecReload=/bin/kill -HUP $MAINPID

KillMode=mixed


[Install]

WantedBy=multi-user.target



Permissions:

➜  ls -alt /etc/squid/
total 128
drwxr-xr-x   2 root root 4096 Sep  6 11:33 .
-rw-r--r--   1 root root 2831 Sep  6 11:33 squid7.conf
drwxr-xr-x 116 root root 4096 Sep  6 11:33 ..
-rw-r--r--   1 root root 2830 Sep  6 11:33 squid2.conf
-rw-r--r--   1 root root 2836 Sep  6 11:33 squid13.conf
-rw-r--r--   1 root root 2836 Sep  6 11:32 squid23.conf
-rw-r--r--   1 root root 2836 Sep  6 11:32 squid19.conf
-rw-r--r--   1 root root 2832 Sep  6 11:32 squid1.conf
-rw-r--r--   1 root root 2836 Sep  6 11:32 squid17.conf
-rw-r--r--   1 root root 2832 Sep  6 11:31 squid4.conf
-rw-r--r--   1 root root 2834 Sep  6 11:31 squid21.conf
-rw-r--r--   1 root root 2833 Sep  6 11:31 squid25.conf
-rw-r--r--   1 root root 2834 Sep  6 11:31 squid12.conf
-rw-r--r--   1 root root 2832 Sep  6 11:31 squid3.conf
-rw-r--r--   1 root root 2836 Sep  6 11:30 squid10.conf
-rw-r--r--   1 root root 2835 Sep  6 11:30 squid11.conf
-rw-r--r--   1 root root 2833 Sep  6 11:30 squid18.conf
-rw-r--r--   1 root root 2830 Sep  6 11:30 squid8.conf
-rw-r--r--   1 root root 2830 Sep  6 11:30 squid6.conf
-rw-r--r--   1 root root 2833 Sep  6 11:30 squid28.conf
-rw-r--r--   1 root root 2830 Sep  6 11:25 squid9.conf
-rw-r--r--   1 root root 2836 Sep  6 11:25 squid24.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid22.conf
-rw-r--r--   1 root root 2837 Sep  6 11:25 squid20.conf
-rw-r--r--   1 root root 2836 Sep  6 11:25 squid16.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid15.conf
-rw-r--r--   1 root root 2836 Sep  6 11:25 squid14.conf
-rw-r--r--   1 root root 2831 Sep  6 11:25 squid5.conf
-rw-r--r--   1 root root 2833 Sep  6 11:25 squid27.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid26.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid30.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid29.conf


_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users




[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux