>>On 11/08/2017 02:26 AM, Bike dernikov1 wrote: >> I hope that someone can explain what happened, why squid stopped working. >I can suggest a working theory: You did not have enough RAM before >vm.swappiness changes and the same insufficient RAM problem led to >failed system calls after you told the OS to free RAM (into swap) less >aggressively. The changes effectively decreased the amount of >immediately available RAM. >From "free -m output" (available column) i thought that we have enough memory, because as I can remember, available never dropped under 2GB. As I understood that system with vm.swappines set to 10, will start draining ram to swap after 90% memory used. So why didn't swap hop in ? That we set vm.swappines to 0 i would understand error. Because errors started with logrotate, i suspect that problem exploded because of logrotate and swap started to write to a disk at same time, and because of slow disks (small IOPS) it could not drain memory fast enough, so available memory vanished and hell lose from there. >If Squid is the primary service on your server, then consider: It isn't only service but use over 90-95 percent memory resources. >1. Removing RAID (for Squid disk cache). We don't use disk cache because of slow disks, compared to "high" bandwith (we have 500Mbit/s) >2. Removing swap (or at least not counting on it and treating > any non-trivial swap usage as a system misconfiguration). What to search ? Key words, vm.swappines got us in problem, as we tried to tune ? >3. Identify the primary RAM consumers and > the factors that influence their memory usage. Squid, we suspect on kerberos negotiator, squid helpers, we will tune up down setting over two weeks. We have problems with squid.conf reconfiguration, because users lose connection (they got squid error screen for moment) so it is nightmare if we changing configuration in work time. >The above actions may not solve your "insufficient RAM" problem (unless >RAID was the primary culprit) but they will give you a better starting >point for finding the solution. Others on the list can help you get >there, especially if you have specific questions. > > >Good luck, > >Alex. Thanks for theory, and help. On Wed, Nov 8, 2017 at 4:08 PM, Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> wrote: > On 11/08/2017 02:26 AM, Bike dernikov1 wrote: > >> I hope that someone can explain what happened, why squid stopped working. > > I can suggest a working theory: You did not have enough RAM before > vm.swappiness changes and the same insufficient RAM problem led to > failed system calls after you told the OS to free RAM (into swap) less > aggressively. The changes effectively decreased the amount of > immediately available RAM. > > If Squid is the primary service on your server, then consider: > > 1. Removing RAID (for Squid disk cache). > > 2. Removing swap (or at least not counting on it and treating > any non-trivial swap usage as a system misconfiguration). > > 3. Identify the primary RAM consumers and > the factors that influence their memory usage. > > The above actions may not solve your "insufficient RAM" problem (unless > RAID was the primary culprit) but they will give you a better starting > point for finding the solution. Others on the list can help you get > there, especially if you have specific questions. > > > Good luck, > > Alex. > >> The server has 24GB DDR3 memory, disk swap set to 24GB, 12 CPU (24HT cores). >> We have 2800 users, using kerberos authentication, squidguard for >> filtering, ldap authorization. _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
