Hi, I identify that it is not cached ( or maybe cached but not served from the cache !) when looking at the logs : it is always a TCP_MISS when getting this object + it is reported in the X-Cache header whenever I get the object with wget : wget -e http_proxy=X.X.X.X:3128 -S http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/22.0/update/win32/fr/firefox-22.0.complete.mar -O /dev/null --2013-07-08 14:54:09-- http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/22.0/update/win32/fr/firefox-22.0.complete.mar Connecting to X.X.X.X:3128... connected. Proxy request sent, awaiting response... HTTP/1.1 200 OK Last-Modified: Tue, 18 Jun 2013 15:38:37 GMT ETag: "dcfc8c-19e4648-4df6f81aff940" Server: Apache X-Backend-Server: ftp3.dmz.scl3.mozilla.com Content-Type: application/octet-stream Accept-Ranges: bytes Access-Control-Allow-Origin: * X-Cache-Info: caching Content-Length: 27149896 Cache-Control: max-age=164515 Expires: Wed, 10 Jul 2013 10:37:05 GMT Date: Mon, 08 Jul 2013 12:55:10 GMT X-Cache: MISS from prxpusx104 X-Cache: MISS from prxpusx001 Via: 1.1 prxpusx104 (squid/3.2.11), 1.1 prxpusx001 (squid/3.2.11) Connection: keep-alive Length: 27149896 (26M) [application/octet-stream] Saving to: ‘/dev/null’ 100%[=========================================================================================>] 27,149,896 8.57MB/s in 3.0s 2013-07-08 14:54:12 (8.57 MB/s) - ‘/dev/null’ saved [27149896/27149896] More info. Here is the squid.conf file ( squid v 3.2.11 ). I striped down some ACLs so that it is more readable ( I hope ). So yes SMP with 4 workers. # pour le debogage (ne pas mettre plus de 2) #debug_options ALL,2 # Pas de limitation sur les FD max_filedescriptors 65535 # Utilisateurs cache_effective_user nobody cache_effective_group nobody # Format access.log strip_query_terms off #logformat Squid %ts.%03tu %6tr %>a %Ss/%>Hs %<st %rm %ru %un %Sh/%<A %mt logformat PAS-Bdx %ts.%03tu %6tr %>a %Ss/%>Hs %<st %rm %ru %un %Sh/%<A %mt %rv %tl "%{Referer}>h" "%{User-Agent}>h" # chemins coredump_dir /var/cache/squid pid_filename /var/run/squid/squid.pid access_log stdio:/var/log/squid/access.log PAS-Bdx cache_log /var/log/squid/cache.log cache_store_log none mime_table /etc/squid/mime.conf error_directory /etc/squid/errors error_default_language fr err_page_stylesheet /etc/squid/errorpage.css # Fichier hosts hosts_file /etc/hosts # SNMP acl snmpcommunity snmp_community read_only_user snmp_access allow snmpcommunity snmp_port 3401 ########################### # FONCTIONNEMENT DU PROXY # ########################### #SMP workers 4 #Ports d'ecoute http_port 3128 http_access allow localhost manager http_access deny manager #définition de la VIP des squid Père cache_peer X.X.X.X parent 3128 0 default no-query no-digest # Filtrage avec squidGuard url_rewrite_program /usr/local/squidGuard/bin/squidGuard url_rewrite_children 1000 startup=10 idle=5 concurrency=0 # Time Out / Time To Live negative_ttl 1 seconds read_timeout 15 minutes request_timeout 5 minutes client_lifetime 4 hours positive_dns_ttl 2 hours negative_dns_ttl 5 minutes shutdown_lifetime 5 seconds dns_nameservers 127.0.0.1 # Divers ftp_passive on ftp_epsv off logfile_rotate 2 request_header_access Via deny all request_header_access X-Forwarded-For allow all refresh_all_ims on ########################### # FONCTIONNEMENT DU CACHE # ########################### #definition du cache #8Gb of shared rock cache, for 32Ko objects max cache_dir rock /var/cache/squid/mem/ 8192 max-size=32768 if ${process_number} =1 cache_dir aufs /var/cache/squid/mem/W${process_number} 3000 16 256 min-size=32768 max-size=131072 cache_dir aufs /var/cache/squid/W${process_number} 9000 16 256 min-size=131072 endif if ${process_number} =2 cache_dir aufs /var/cache/squid/mem/W${process_number} 3000 16 256 min-size=32768 max-size=131072 cache_dir aufs /var/cache/squid/W${process_number} 9000 16 256 min-size=131072 endif if ${process_number} =3 cache_dir aufs /var/cache/squid/mem/W${process_number} 3000 16 256 min-size=32768 max-size=131072 cache_dir aufs /var/cache/squid/W${process_number} 9000 16 256 min-size=131072 endif if ${process_number} =4 cache_dir aufs /var/cache/squid/mem/W${process_number} 3000 16 256 min-size=32768 max-size=131072 cache_dir aufs /var/cache/squid/W${process_number} 9000 16 256 min-size=131072 endif # pages dynamiques non mises en cache acl QUERY urlpath_regex cgi-bin \? \.fcgi \.cgi \.pl \.php3 \.asp \.php \.do no_cache deny QUERY #Rafraichissement du cache memory_cache_shared on cache_mem 4 GB max_filedesc 65535 maximum_object_size 512 MB maximum_object_size_in_memory 2048 KB ipcache_size 8192 fqdncache_size 8192 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 # Least frequently used with dynamic aging cache_replacement_policy heap LFUDA memory_replacement_policy heap LFUDA # pas de redirecteur server_persistent_connections on client_persistent_connections on half_closed_clients off #tout le trafic passe par le squid père sauf les pages d'erreur always_direct allow to_localhost never_direct allow all # Full internet acl fullinternet src "/etc/squid/acl/src/full.src" http_access allow fullinternet http_access deny all __________________________________________ here is the output of the mgr:storedir : [root] # /usr/local/squid/bin/squidclient mgr:storedir HTTP/1.1 200 OK Server: squid/3.2.11 Mime-Version: 1.0 Date: Mon, 08 Jul 2013 12:57:27 GMT Content-Type: text/plain Expires: Mon, 08 Jul 2013 12:57:27 GMT Last-Modified: Mon, 08 Jul 2013 12:57:27 GMT Connection: close by kid1 { Store Directory Statistics: Store Entries : 69552 Maximum Swap Size : 12288000 KB Current Store Swap Size: 11058996.00 KB Current Capacity : 90.00% used, 10.00% free Shared Memory Cache Maximum Size: 4194304 KB Maximum entries: 131072 Current entries: 125817 95.99% Store Directory #1 (aufs): /var/cache/squid/mem/W1 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 3072000 KB Current Size: 2764880.00 KB Percent Used: 90.00% Filemap bits in use: 45166 of 65536 (69%) Filesystem Space in use: 18200988/20971520 KB (87%) Filesystem Inodes in use: 197750/4627610 (4%) Flags: Removal policy: lru LRU reference age: 5.14 days Store Directory #2 (aufs): /var/cache/squid/W1 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 9216000 KB Current Size: 8294116.00 KB Percent Used: 90.00% Filemap bits in use: 23235 of 32768 (71%) Filesystem Space in use: 39747760/59850496 KB (66%) Filesystem Inodes in use: 117081/15451392 (1%) Flags: Removal policy: lru LRU reference age: 7.19 days } by kid1 by kid2 { Store Directory Statistics: Store Entries : 71099 Maximum Swap Size : 12288000 KB Current Store Swap Size: 11059168.00 KB Current Capacity : 90.00% used, 10.00% free Shared Memory Cache Maximum Size: 4194304 KB Maximum entries: 131072 Current entries: 125817 95.99% Store Directory #1 (aufs): /var/cache/squid/mem/W2 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 3072000 KB Current Size: 2764784.00 KB Percent Used: 90.00% Filemap bits in use: 45277 of 65536 (69%) Filesystem Space in use: 18200988/20971520 KB (87%) Filesystem Inodes in use: 197750/4627610 (4%) Flags: Removal policy: lru LRU reference age: 4.95 days Store Directory #2 (aufs): /var/cache/squid/W2 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 9216000 KB Current Size: 8294384.00 KB Percent Used: 90.00% Filemap bits in use: 24949 of 32768 (76%) Filesystem Space in use: 39747760/59850496 KB (66%) Filesystem Inodes in use: 117081/15451392 (1%) Flags: Removal policy: lru LRU reference age: 7.03 days } by kid2 by kid3 { Store Directory Statistics: Store Entries : 71223 Maximum Swap Size : 12288000 KB Current Store Swap Size: 11057428.00 KB Current Capacity : 89.99% used, 10.01% free Shared Memory Cache Maximum Size: 4194304 KB Maximum entries: 131072 Current entries: 125817 95.99% Store Directory #1 (aufs): /var/cache/squid/mem/W3 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 3072000 KB Current Size: 2764776.00 KB Percent Used: 90.00% Filemap bits in use: 45360 of 65536 (69%) Filesystem Space in use: 18200988/20971520 KB (87%) Filesystem Inodes in use: 197750/4627610 (4%) Flags: Removal policy: lru LRU reference age: 4.18 days Store Directory #2 (aufs): /var/cache/squid/W3 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 9216000 KB Current Size: 8292652.00 KB Percent Used: 89.98% Filemap bits in use: 24888 of 32768 (76%) Filesystem Space in use: 39747760/59850496 KB (66%) Filesystem Inodes in use: 117081/15451392 (1%) Flags: Removal policy: lru LRU reference age: 7.14 days } by kid3 by kid4 { Store Directory Statistics: Store Entries : 71236 Maximum Swap Size : 12288000 KB Current Store Swap Size: 11058892.00 KB Current Capacity : 90.00% used, 10.00% free Shared Memory Cache Maximum Size: 4194304 KB Maximum entries: 131072 Current entries: 125817 95.99% Store Directory #1 (aufs): /var/cache/squid/mem/W4 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 3072000 KB Current Size: 2764760.00 KB Percent Used: 90.00% Filemap bits in use: 45489 of 65536 (69%) Filesystem Space in use: 18200988/20971520 KB (87%) Filesystem Inodes in use: 197750/4627610 (4%) Flags: Removal policy: lru LRU reference age: 5.20 days Store Directory #2 (aufs): /var/cache/squid/W4 FS Block Size 4096 Bytes First level subdirectories: 16 Second level subdirectories: 256 Maximum Size: 9216000 KB Current Size: 8294132.00 KB Percent Used: 90.00% Filemap bits in use: 24771 of 32768 (76%) Filesystem Space in use: 39747760/59850496 KB (66%) Filesystem Inodes in use: 117081/15451392 (1%) Flags: Removal policy: lru LRU reference age: 10.99 days } by kid4 by kid5 { Store Directory Statistics: Store Entries : 52 Maximum Swap Size : 8388608 KB Current Store Swap Size: 8387184.00 KB Current Capacity : 99.98% used, 0.02% free Store Directory #0 (rock): /var/cache/squid/mem/ FS Block Size 1024 Bytes Maximum Size: 8388608 KB Current Size: 8387184.00 KB 99.98% Maximum entries: 262143 Current entries: 262099 99.98% Pending operations: 1 out of 0 Flags: } by kid5 Thanks! Alex 2013/7/5 Amos Jeffries <squid3@xxxxxxxxxxxxx>: > On 5/07/2013 3:33 a.m., Alexandre Chappaz wrote: >> >> Hi, >> >> I have this object not being cached, and I can't understand why. > > > Lets start with... how are you identifying that it is not being cached? > >> >> http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/22.0/update/win32/fr/firefox-22.0.complete.mar > > redbot.org confirms this is a cacheable resource with no protocol problems > visible. > > >> I have this cachedir set for objects of size > 128Ko : >> >> cache_dir aufs /var/cache/squid/W${process_number} 9000 16 256 >> min-size=131072 > > > SMP macros in use I see. How many workers do you have? > > Also, what *other* size limits on object sizes are in your configuration > file? > please list those *and* all cache_dir lines in your configuration file in > exactly the order they appear in the config. > > > >> >> >> the headers of the object : >> HTTP/1.1 200 OK >> Last-Modified: Tue, 18 Jun 2013 15:38:37 GMT >> ETag: "dcfc8c-19e4648-4df6f81aff940" >> Server: Apache >> X-Backend-Server: ftp3.dmz.scl3.mozilla.com >> Content-Type: application/octet-stream >> Accept-Ranges: bytes >> Access-Control-Allow-Origin: * >> X-Cache-Info: caching >> Content-Length: 27149896 >> Cache-Control: max-age=266561 >> Expires: Sat, 06 Jul 2013 18:26:03 GMT >> Date: Wed, 03 Jul 2013 16:23:22 GMT >> >> >> the storedir is full, but I understand that since the object is >> requested very often, it should replace anotherone in the cachedir. >> (cache_replacement_policy heap LFUDA ) > > > You say *the storedir* ... but there are multiple store directories yes? one > for each Squid worker process. > > > >> Do you have any hint on how to make sure th object gets cached? >> Thanks > > > More info needed. > > Please also list your refresh_pattern rules. > > > Amos
