Re: logfileHandleWrite: daemon:/var/log/squid/access.log: error writing ((32) Broken pipe)

[roee klinger <roeeklinger60@xxxxxxxxx>]

Hey Alex, 

Thank you for your reply.

Your syslog or cache.log might
contain more info. Analyzing the timing/schedule of these problems may
also be helpful in identifying the trigger.

After following this advice, I could easily see that this happens due to disk space running out on my server,
it seems that the logs has filled over 100GB of log data, since I made a configuration mistake (I think?) by setting this:

logfile_rotate 0

If I remember and read correctly, this means that the rotation of the files is disabled and they will just keeping increasing
in size if left unchecked.

I have now gone ahead and changed all the configuration file to this setting:

logfile_rotate 1

So now it should rotate once daily, and on the next rotation it should be deleted, and this is all handled by logrotate on Debian-based machines?

This is my / cat /etc/logrotate.d/squid:
➜ / cat /etc/logrotate.d/squid 
#
# Logrotate fragment for squid.
#
/var/log/squid/*.log {
 daily
 compress
 delaycompress
 rotate 2
 missingok
 nocreate
 sharedscripts
 prerotate
 test ! -x /usr/sbin/sarg-reports || /usr/sbin/sarg-reports daily
 endscript
 postrotate
 test ! -e /var/run/squid.pid || test ! -x /usr/sbin/squid || /usr/sbin/squid -k rotate
 endscript
}

Is there a way for me to set it so it just get deleted every 24 or 12 hours without the archive first?

Thanks,
Roee

On 6 Sep 2022, 16:28 +0300, Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>, wrote:

On 9/6/22 07:41, roee klinger wrote:

It is also important to know that I am running multiple Squid instances
on the same machine, they are all getting the error at the same time

What external event(s) happen at that time? Something is probably
sending a signal to the logging daemon process. It would be good to know
what that something (and that signal) is. Your syslog or cache.log might
contain more info. Analyzing the timing/schedule of these problems may
also be helpful in identifying the trigger.

Is a possible workaround that might be just replacing the line with
this?

access_log /var/log/squid/access2.log

As you know, this configuration (in this deprecated spelling or with and
explicit "stdio:" prefix) will result in Squid workers writing to the
log file directly instead of asking the logging daemon. This will,
naturally, get rid of the pipe between workers and their daemons, and
the associated broken pipe error.

or will this cause a problem?

Impossible to say for sure without knowing whether your workers benefit
from the anticipated performance advantages of avoiding blocking file
I/O _and_ whether those advantages are real (in your environment). Too
many variables and too many unknowns. I would treat this as an important
(and potentially disruptive) configuration change and carefully test the
outcome.


HTH,

Alex.

INFO -
Versions:

Squid Cache: Version 4.10
Ubuntu 20.04.4 LTS


Example squid.conf:

visible_hostname squid2

access_log daemon:/var/log/squid/access2.log squid

cache_log /var/log/squid/cache2.log

pid_filename /var/run/squid2.pid


acl localnet src 0.0.0.1-0.255.255.255# RFC 1122 "this" network (LAN)

acl localnet src 10.0.0.0/8 <http://10.0.0.0/8> # RFC 1918 local
private network (LAN)

acl localnet src 100.64.0.0/10 <http://100.64.0.0/10># RFC 6598
shared address space (CGN)

acl localnet src 169.254.0.0/16 <http://169.254.0.0/16> # RFC 3927
link-local (directly plugged) machines

acl localnet src 172.16.0.0/12 <http://172.16.0.0/12># RFC 1918
local private network (LAN)

acl localnet src 192.168.0.0/16 <http://192.168.0.0/16> # RFC 1918
local private network (LAN)

acl localnet src fc00::/7 # RFC 4193 local private network range

acl localnet src fe80::/10# RFC 4291 link-local (directly plugged)
machines

acl SSL_ports port 443

acl Safe_ports port 80# http

acl Safe_ports port 21# ftp

acl Safe_ports port 443 # https

acl Safe_ports port 70# gopher

acl Safe_ports port 210 # wais

acl Safe_ports port 1025-65535# unregistered ports

acl Safe_ports port 280 # http-mgmt

acl Safe_ports port 488 # gss-http

acl Safe_ports port 591 # filemaker

acl Safe_ports port 777 # multiling http

acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager

http_access deny manager

# include /etc/squid/conf.d/*

http_access allow localhost

acl aws src *censored*

http_access allow aws

# http_access deny all

tcp_outgoing_address *censored*

http_port 10002

coredump_dir /var/spool/squid

refresh_pattern ^ftp: 144020% 10080

refresh_pattern ^gopher:14400%1440

refresh_pattern -i (/cgi-bin/|\?) 0 0%0

refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0
refresh-ims

refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims

refresh_pattern \/InRelease$ 0 0% 0 refresh-ims

refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims

refresh_pattern . 0 20% 4320


shutdown_lifetime 1 seconds

logfile_rotate 0

max_filedescriptors 16384

dns_nameservers 8.8.8.8 8.8.4.4 1.1.1.1

cache deny all

cache_dir null /tmp

via off

forwarded_for off

request_header_access From deny all

request_header_access Server deny all

request_header_access WWW-Authenticate deny all

request_header_access Link deny all

request_header_access Cache-Control deny all

request_header_access Proxy-Connection deny all

request_header_access X-Cache deny all

request_header_access X-Cache-Lookup deny all

request_header_access Via deny all

request_header_access X-Forwarded-For deny all

request_header_access Pragma deny all

request_header_access Keep-Alive deny all

dns_v4_first on


Example service file:

## Copyright (C) 1996-2020 The Squid Software Foundation and
contributors

##

## Squid software is distributed under GPLv2+ license and includes

## contributions from numerous individuals and organizations.

## Please see the COPYING and CONTRIBUTORS files for details.

##


[Unit]

Description=Squid Web Proxy Server

Documentation=man:squid(8)

After=network.target network-online.target nss-lookup.target


[Service]

Type=forking

PIDFile=/var/run/squid2.pid

ExecStartPre=/usr/sbin/squid --foreground -z -f /etc/squid/squid2.conf

ExecStart=/usr/sbin/squid -sYC -f /etc/squid/squid2.conf

ExecReload=/bin/kill -HUP $MAINPID

KillMode=mixed


[Install]

WantedBy=multi-user.target



Permissions:

➜  ls -alt /etc/squid/
total 128
drwxr-xr-x   2 root root 4096 Sep  6 11:33 .
-rw-r--r--   1 root root 2831 Sep  6 11:33 squid7.conf
drwxr-xr-x 116 root root 4096 Sep  6 11:33 ..
-rw-r--r--   1 root root 2830 Sep  6 11:33 squid2.conf
-rw-r--r--   1 root root 2836 Sep  6 11:33 squid13.conf
-rw-r--r--   1 root root 2836 Sep  6 11:32 squid23.conf
-rw-r--r--   1 root root 2836 Sep  6 11:32 squid19.conf
-rw-r--r--   1 root root 2832 Sep  6 11:32 squid1.conf
-rw-r--r--   1 root root 2836 Sep  6 11:32 squid17.conf
-rw-r--r--   1 root root 2832 Sep  6 11:31 squid4.conf
-rw-r--r--   1 root root 2834 Sep  6 11:31 squid21.conf
-rw-r--r--   1 root root 2833 Sep  6 11:31 squid25.conf
-rw-r--r--   1 root root 2834 Sep  6 11:31 squid12.conf
-rw-r--r--   1 root root 2832 Sep  6 11:31 squid3.conf
-rw-r--r--   1 root root 2836 Sep  6 11:30 squid10.conf
-rw-r--r--   1 root root 2835 Sep  6 11:30 squid11.conf
-rw-r--r--   1 root root 2833 Sep  6 11:30 squid18.conf
-rw-r--r--   1 root root 2830 Sep  6 11:30 squid8.conf
-rw-r--r--   1 root root 2830 Sep  6 11:30 squid6.conf
-rw-r--r--   1 root root 2833 Sep  6 11:30 squid28.conf
-rw-r--r--   1 root root 2830 Sep  6 11:25 squid9.conf
-rw-r--r--   1 root root 2836 Sep  6 11:25 squid24.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid22.conf
-rw-r--r--   1 root root 2837 Sep  6 11:25 squid20.conf
-rw-r--r--   1 root root 2836 Sep  6 11:25 squid16.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid15.conf
-rw-r--r--   1 root root 2836 Sep  6 11:25 squid14.conf
-rw-r--r--   1 root root 2831 Sep  6 11:25 squid5.conf
-rw-r--r--   1 root root 2833 Sep  6 11:25 squid27.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid26.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid30.conf
-rw-r--r--   1 root root 2835 Sep  6 11:25 squid29.conf


_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users