Hi, I hope that someone can explain what happened, why squid stopped working. The problem is related to memory/swap handling. After we changed vm.swappiness parameter from 60 to 10 (tuning attempt, to lower a disk usage, because we have only 4 disks in a RAID10, so disk subsystem is a weak link), we got a lot of errors in cache.log. The problems started after scheduled logrotate after 2AM. Squid ran out of memory, auth helpers stopped working. It's weird because we didn't disable swap, but behavior is like we did. After an error, we increased parameter from 10 to 40. The server has 24GB DDR3 memory, disk swap set to 24GB, 12 CPU (24HT cores). We have 2800 users, using kerberos authentication, squidguard for filtering, ldap authorization. When problem appeared memory was still 3GB free (free column), ram (caching) was filled to 15GB, so 21 GB ram filled, 3GB free. Thanks for help, errors from cache.log. 2017/11/08 02:55:27| Set Current Directory to /var/log/squid/ 2017/11/08 02:55:27 kid1| storeDirWriteCleanLogs: Starting... 2017/11/08 02:55:27 kid1| Finished. Wrote 0 entries. 2017/11/08 02:55:27 kid1| Took 0.00 seconds ( 0.00 entries/sec). 2017/11/08 02:55:27 kid1| logfileRotate: daemon:/var/log/squid/access.log 2017/11/08 02:55:27 kid1| logfileRotate: daemon:/var/log/squid/access.log 2017/11/08 02:55:28 kid1| Pinger socket opened on FD 30 2017/11/08 02:55:28 kid1| helperOpenServers: Starting 1/1000 'squidGuard' processes 2017/11/08 02:55:28 kid1| ipcCreate: fork: (12) Cannot allocate memory 2017/11/08 02:55:28 kid1| WARNING: Cannot run '/usr/bin/squidGuard' process. 2017/11/08 02:55:28 kid1| helperOpenServers: Starting 300/3000 'negotiate_kerberos_auth' processes 2017/11/08 02:55:28 kid1| ipcCreate: fork: (12) Cannot allocate memory 2017/11/08 02:55:28 kid1| WARNING: Cannot run '/usr/lib/squid/negotiate_kerberos_auth' process. 2017/11/08 02:55:28 kid1| ipcCreate: fork: (12) Cannot allocate memory 2017/11/08 02:55:28 kid1| WARNING: Cannot run '/usr/lib/squid/negotiate_kerberos_auth' process. 2017/11/08 02:55:28 kid1| ipcCreate: fork: (12) Cannot allocate memory 2017/11/08 02:55:28 kid1| WARNING: Cannot run '/usr/lib/squid/negotiate_kerberos_auth' process. external ACL 'memberof' queue overload. Using stale result. _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
