Hello Kaya, first, don't forget to look at sysctl kern.maxfiles values. Also improve daemon FD values in login.conf for squid. Don't forget each connection is a FD (1 connection for the client, 1 for the transaction to remote site, somes for the caching). Also to improve performances of squidguard, i stored all blacklists DB to a memory fs (mfs) this improve massively squidguard performance I have wrote an article to improve squid perfs on OpenBSD: http://www.unix-experience.fr/2013/monter-un-proxy-cache-performant-avec-squid-et-openbsd/ -- Best regards, Loïc BLOT, UNIX systems, security and network engineer http://www.unix-experience.fr Le samedi 09 novembre 2013 à 19:39 +0000, Kaya Saman a écrit : > Just found this is Squid cache log: > > 2013/11/09 19:28:25 kid1| /var/squid/cache/04/7A: (24) Too many open files > 2013/11/09 19:31:31 kid1| WARNING: All 20/20 redirector processes are busy. > 2013/11/09 19:31:31 kid1| WARNING: 20 pending requests queued > 2013/11/09 19:31:31 kid1| WARNING: Consider increasing the number of > redirector processes in your config file. > > > The cache size is 2GB.... though that shouldn't affect performance as > far as I understand. > > On 11/09/2013 05:23 PM, Eliezer Croitoru wrote: > > Hey, > > > > Notes inside. > > > > On 11/09/2013 05:58 PM, Kaya Saman wrote: > >> > >> What can I do to improve performance with this? > >> > >> > >> Is this line too high: url_rewrite_children 500 > > YES!! > > > >> or simply have a misconfigured something? > > > > > >> I additionally have 'c-icap' running with squidclamav coupled to clamd > >> in case that is of importance - not using the squidGuard line in the > >> squidclamav.conf file!!! > >> > >> Basically how can I get the IO usage down and get the system to work > >> again? > > For how many users exactly? > > Just a note that I am not in a favor of any OS by default but I would > > feel better Using Linux. > > > >> > >> - the logs don't indicate anything outside of 'starting squidGuard > >> process' many times. > > The basic assumption of using 500 child process is that you have > > atleast 100 CPUs. > > SquidGuard was design for performance which is lots of urls per sec. > > It can be tested just to clear the point out. > > for example in a rate of 1500k requests per second you should not have > > a need in more then 40-50 children. > > In practice it works a bit different speed since there is a speed > > limit on STDIN and STDOUT which slows down the speed of squid and > > squidguard communication blocking the whole squid instance(in a way). > > > > If you need basic url filtering you can use ICAP which has an option > > to run as a standalone service outside of squid settings and machine. > > > > I have written in the past a small ICAP service for the favor of > > requests manipulation and filtering. > > I have never finished it in a level I was happy with but the basic > > code can be seen here: > > https://github.com/elico/echelon > > > > I know for a fact that ICAP interface adds concurrency by the "nature" > > of it using TCP. > > > > This is not the place to ask about concurrency in squidguard which can > > allow the usage of square less processes(children) for more requests. > > > > In order to find the right number of children start with 40 and see if > > it fits you and then see what is the bottle neck in the whole setup. > > > > Eliezer >
Attachment:
signature.asc
Description: This is a digitally signed message part
