Amos Jeffries-2 wrote: > > On Mon, 27 Jul 2009 12:14:29 -0700 (PDT), jotacekm <minuzzo@xxxxxxxxxxxx> > wrote: >> Hello. >> Recently we have added a lot more clientes behind a squid proxy, and now >> cpu >> utilizitaion is usually 70-95%. The processor is a intel dual core 2160 > @ >> 1.80GHz. Users started complaing about the speed on accessing pages, and >> the >> link is fine. >> >> Here is squidclient mgr:info: >> >> Squid Object Cache: Version 2.6.STABLE5 >> Start Time: Fri, 24 Jul 2009 20:19:16 GMT >> Current Time: Mon, 27 Jul 2009 19:09:07 GMT >> Connection information for squid: >> Number of clients accessing cache: 1561 >> Number of HTTP requests received: 8590404 >> Number of ICP messages received: 0 >> Number of ICP messages sent: 0 >> Number of queued ICP replies: 0 >> Number of HTCP messages received: 0 >> Number of HTCP messages sent: 0 >> Request failure ratio: 0.00 >> Average HTTP requests per minute since start: 2021.3 > > 2k req/min is getting close to what people report for saturation of older > 2.6 forward proxies. > First performance improvement will be simply an upgrade to 2.7. > > >> Average ICP messages per minute since start: 0.0 >> Select loop called: 122560206 times, 2.081 ms avg >> Cache information for squid: >> Request Hit Ratios: 5min: 23.0%, 60min: 21.9% >> Byte Hit Ratios: 5min: 13.3%, 60min: 14.7% >> Request Memory Hit Ratios: 5min: 16.3%, 60min: 17.0% >> Request Disk Hit Ratios: 5min: 26.3%, 60min: 29.9% >> Storage Swap size: 4609784 KB >> Storage Mem size: 65692 KB >> Mean Object Size: 15.67 KB >> Requests given to unlinkd: 455490 >> Median Service Times (seconds) 5 min 60 min: >> HTTP Requests (All): 1.24267 1.46131 >> Cache Misses: 1.54242 1.81376 >> Cache Hits: 0.28853 0.30459 >> Near Hits: 1.31166 1.62803 >> Not-Modified Replies: 0.23230 0.23230 >> DNS Lookups: 0.29097 0.31806 >> ICP Queries: 0.00000 0.00000 >> Resource usage for squid: >> UP Time: 254991.358 seconds >> CPU Time: 56029.670 seconds >> CPU Usage: 21.97% >> CPU Usage, 5 minute avg: 94.38% >> CPU Usage, 60 minute avg: 93.36% >> Process Data Segment Size via sbrk(): 193008 KB >> Maximum Resident Size: 0 KB >> Page faults with physical i/o: 70 >> Memory usage for squid via mallinfo(): >> Total space in arena: 193008 KB >> Ordinary blocks: 179871 KB 6058 blks >> Small blocks: 0 KB 0 blks >> Holding blocks: 1080 KB 2 blks >> Free Small blocks: 0 KB >> Free Ordinary blocks: 13136 KB >> Total in use: 180951 KB 93% >> Total free: 13136 KB 7% >> Total size: 194088 KB >> Memory accounted for: >> Total accounted: 111416 KB >> memPoolAlloc calls: 973288506 >> memPoolFree calls: 972077571 >> File descriptor usage for squid: >> Maximum number of file descriptors: 4096 >> Largest file desc currently in use: 1604 >> Number of file desc currently in use: 1308 >> Files queued for open: 0 >> Available number of file descriptors: 2788 >> Reserved number of file descriptors: 100 >> Store Disk files open: 30 >> IO loop method: epoll >> Internal Data Structures: >> 300182 StoreEntries >> 14733 StoreEntries with MemObjects >> 14414 Hot Object Cache Items >> 294106 on-disk objects >> >> And here is part of of squid.conf: >> >> http_port 3128 >> visible_hostname xxx >> hierarchy_stoplist cgi-bin ? >> acl QUERY urlpath_regex cgi-bin \? >> cache deny QUERY > > _after_ upgrading to the most recent 2.6 or a 2.7. Remove the QUERY > entries > above. > >> acl apache rep_header Server ^Apache >> broken_vary_encoding allow apache >> access_log /var/log/squid/access.log squid >> cache_store_log none >> hosts_file /etc/hosts >> >> >> # >> > ---------------------------------------------------------------------------------------------------------- >> cache_mem 64 MB >> cache_dir ufs /var/spool/squid 5000 50 256 > > #1 change that to aufs if using a Linux or variants. diskd if using BSD > or > variants. Change to Linux or BSD if using Windows. > > #2 up the cache size as much as you can. Local objects are much faster to > retrieve than external ones. > The more local objects you can store the faster Squid responds. > From my experience its around an order of magnitude or more req/sec > handled > from cache as from network. > > Memory cache is better for speed than disk, but can be lost more easily > and > you can get bigger cache total spending RAM on indexes and disk on object > storage. > > After the upgrade to 2.7 you have access to a stable COSS system which is > very much more efficient for the small files. It's worth a try. > >> cache_replacement_policy heap LFUDA >> maximum_object_size 51200 KB > > There is good evidence that the web now has a lot of traffic in the 2MB to > 250MB range from multimedia objects. > This will bear investigating in your traffic and see if the limit above is > still good. > > >> maximum_object_size_in_memory 64 KB >> memory_replacement_policy heap GDSF >> logfile_rotate 3 >> >> >> # >> > ---------------------------------------------------------------------------------------------------------- >> refresh_pattern ^ftp: 1440 20% 10080 >> refresh_pattern ^gopher: 1440 0% 1440 > > Add: > refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 > >> refresh_pattern . 0 20% 4320 >> acl all src 0.0.0.0/0.0.0.0 >> acl manager proto cache_object >> acl localhost src 127.0.0.1/255.255.255.255 >> acl to_localhost dst 127.0.0.0/8 >> >> >> >> max_open_disk_fds 2046 > > Definitely raise that. Your stats show that many _new_ requests arriving > every second. > Each single request eats 2-3 FD and may last from seconds to hours. > >> >> # timeouts >> connect_timeout 30 seconds >> shutdown_lifetime 5 seconds >> forward_timeout 2 minutes >> pconn_timeout 30 seconds > > You _want_ pconn to last a fairly long while. There is no CPU or bandwidth > overhead. > Possibly also enabling client and server persistent connections. > >> persistent_request_timeout 1 minute >> request_timeout 2 minute >> >> Is there anything that i can do to lower the cpu utilization, or do i > have >> to upgrade the hardware? > > There are a number of other perfromance optimizations possible within the > build options and in other squid.conf settings. You don't say what they > are > set at, so we can't suggest any right now. > > Amos > > Hello, Thanks for you help. I'll try the modifications you mentioned and report here if they worked. About the build options, this squid was not compiled, it was installed via apt-get in a debian etch. And the other squid.conf options are just acls and allows / denies. -- View this message in context: http://www.nabble.com/High-CPU-utilization-tp24686629p24697787.html Sent from the Squid - Users mailing list archive at Nabble.com.
