On 4/08/2013 7:13 p.m., John Joseph wrote:
Thanks Augustus for the email my information is ------------------- [root@proxy squid]# squidclient -h 127.0.0.1 mgr:storedir HTTP/1.0 200 OK Server: squid/3.1.10 Mime-Version: 1.0 Date: Sun, 04 Aug 2013 07:01:30 GMT Content-Type: text/plain Expires: Sun, 04 Aug 2013 07:01:30 GMT Last-Modified: Sun, 04 Aug 2013 07:01:30 GMT X-Cache: MISS from proxy X-Cache-Lookup: MISS from proxy:3128 Via: 1.0 proxy (squid/3.1.10) Connection: close Store Directory Statistics: Store Entries : 13649421 Maximum Swap Size : 583680000 KB Current Store Swap Size: 250112280 KB Current Capacity : 43% used, 57% free Store Directory #0 (aufs): /opt/var/spool/squid FS Block Size 4096 Bytes First level subdirectories: 32 Second level subdirectories: 256 Maximum Size: 583680000 KB Current Size: 250112280 KB Percent Used: 42.85% Filemap bits in use: 13649213 of 16777216 (81%) Filesystem Space in use: 264249784/854534468 KB (31%) Filesystem Inodes in use: 13657502/54263808 (25%) Flags: SELECTED Removal policy: lru LRU reference age: 44.69 days
You appear to have a good case there for upgrading to squid-3.2 or later and adding a rock cache_dir.
As you can see 81% of the Filemap is full. That is the file number codes Squid uses to internally reference stored objects. There is an absolute limit of 2^24 (or "1677216" in the above report). That will require an average object size of 35KB to fill your 557 GB storage area. Your details earlier said the mean object size actually stored so far was 18KB.
If you add a 50GB rock store alongside that UFS directory you should be able to double the cached object count.
-------------- and my squid.conf is as ---------------------------------------------- always_direct allow all cache_log /opt/var/log/squid/cache.log cache_access_log /opt/var/log/squid/access.log cache_swap_low 90 cache_swap_high 95 acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 acl localnet src 172.16.5.0/24 # RFC1918 possible internal network acl localnet src 172.17.0.0/22 # RFC1918 possible internal network acl localnet src 192.168.20.0/24 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines always_direct allow local-servers
You are using always_direct allow all above. This line is never even being checked.
Also, always_direct has no meaning when there are no cache_peer lines to be overridden (which is the purpose of always_direct). You can remove both the always_direct lines to make things a bit faster.
acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl ipgroup src 172.16.5.1-172.16.5.255/32 acl ipgroup src 172.17.0.10-172.17.3.254/32 delay_pools 1 delay_class 1 2 delay_parameters 1 2560000/3860000 140000/180000 delay_access 1 allow ipgroup delay_access 1 deny all http_access allow localnet http_access allow localhost http_access allow localnet http_access allow localhost
You have doubled these rules up.
http_access deny all http_port 3128 transparent
It is a good idea to always have 3128 listing for regular proxy traffic and redirecting the intercepted traffic to a separate port. The interception port is a private detail only relevant to teh NAT infrastructure doing the redirection and Squid. It can be firewalled to prevent any access directly to the port.
hierarchy_stoplist cgi-bin ? cache_dir aufs /opt/var/spool/squid 570000 32 256 coredump_dir /opt/var/spool/squid maximum_object_size 4 GB
Can you try placing this above the cache_dir line please and see if it makes any difference?
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
ignore-private and ignore-no-store are actually VERY bad ideas. No matter that it looks okay for innocent things like images and archives. Even those types are used in critical systems from time to time (think security captchas using images, security certificates exchanged in compressed archive formats, etc, etc).
Please remove them from the above lines. If you need them at all (eg to fix a specific identifiable problem URL) it is best to target the regex pattern to the specific domain or URLs.
In general the CMS systems and dynamic page frameworks use no-cache and Expires to prevent unnecessary caching and force revalidation - 3.1 is not fully capable of that but an upgrade to recent 3.2 or later releases Squid can manage no-cache properly.
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080 refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 40% 40320 visible_hostname proxy
This should be an FQDN if possible. The error page icons and similar things will be reference at an HTTP:// URL using the visible hostname as domain and Squid forward-proxy port as port number. Your configuration is probably sending traffic to "http://proxy:3128/";, whatever that resolves to in the client machine(s).
icap_enable on icap_preview_enable on icap_preview_size 4096 icap_persistent_connections on icap_send_client_ip on icap_send_client_username on icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod adaptation_access qlproxy1 allow all adaptation_access qlproxy2 allow all ------------------------------------------------------------------------ Guidance and advice requested Thanks for the reply Joseph John
Amos
