On Mon, 1 Nov 2010 15:00:21 +0000, declanw@xxxxxxxxxxxx wrote: > Greetings! > > I am poking a potential squid upgrade from squid 2 to 3.1.8 with a new > config, but it's added around 40% more CPU load, and I'm looking for > tune-ups. > > One thing I notice it doing when I truss is read()ing HTTP responses with > only a 2046 byte buffer, whereas squid2 used 24Kb. This is making for a > lot of > unnecessary system calls and go-arounds in the main polling loop. > > I went for a rummage in the code for the buffer size decisions, but got > very > very lost in the OO abstractions very quickly. Can anyone point me at > anything I can tweak to fix this? It's a global macro defined by auto-probing your operating systems TCP receiving buffer when building. Default is 16KB and max is 64KB. There may also be auto-probing done at run time. It is tunable at run-time with http://www.squid-cache.org/Doc/config/tcp_recv_bufsize/ > > Besides that, I have a laaarge url_regexp file to process, and I was > wondering if there was any benefit to trying to break this regexp out to a > perl helper process (and if anyone has a precooked setup doing this I can > borrow) The others have already covered the main points of this. ufdbGuard is probably the way to go once you have restricted the size down by elminiating all the entries which can be done with dstdomain and other faster ACL types. > > Beyond that, I assume, to get the most out of a multi-cpu system I should > be running one squid per CPU, which means I need more IP's and that they > can't share their memory or disk caches with each other directly, and I > would need to switch on HTCP to try and re-merge them? Possibly. You may want to test out 3.2 with SMP support. Reports have been good so far (for a beta). > > While I'm musing here, is there any way to make an ACL construct that makes > a decision based on whether something is already cached? I have a lot of > heavy ACL's, and if something is cached, in my case, it will prove that it > has previously passed all the ACL's and I can just return the cached copy. Not as such. The "acl source hier_code NONE" ACL could do this in the http_reply_access tests where the source is known. The cache index lookup is one of the heavier operations and depends on all request adaptation being done first. It may be possible to make a cache_status ACL or such which works in adapted_http_access and later tests. All ACL with external data sources (DNS, external_acl_type, ident, etc) cache their results for faster re-use. Tuning the TTL for each of these can reduce their impact. > > Build: Sun Solaris 9 > PATH=~/sunstudio12.0/bin:$PATH ./configure CC=cc CXX=CC CFLAGS="-fast > -xtarget=ultra3i -m64 -xipo" CXXFLAGS="-fast -xtarget=ultra3i -m64 -xipo" > --enable-cache-digests --enable-removal-policies=lru,heap > --enable-storeio=aufs,ufs --enable-devpoll Ah. You will definitely be wanting 3.1.9. /dev/poll support is included and several ACL problems specific to the S9 are fixed. > > Tuney bits of Config: > htcp_port 0 > icp_port 0 > digest_generation off > quick_abort_min 0 KB > quick_abort_max 0 KB > read_ahead_gap 64 KB > store_avg_object_size 16 KB > read_timeout 5 minutes > request_timeout 30 seconds > persistent_request_timeout 30 seconds > pconn_timeout 3 seconds NOTE: pconn_timeout tuning can no longer be done based on info from older versions. There have been a LOT of fixes that make 3.1.8+ pconn support HTTP compliant, used more often and less resources hungry than older versions. > cache_mem 512 MB > maximum_object_size_in_memory 64 KB NP: It's worth noting that 3.x has fixed the large file in memory problems which 2.x suffers from. 3.x will handle them in linear time instead of with exponential CPU load. > memory_replacement_policy heap GDSF > ignore_expect_100 on If this is actually a problem you may benefit extra from 3.2 beta here as well. Amos
