Hi Amos, On Mon, Feb 21, 2011 at 5:37 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote: > On Mon, 21 Feb 2011 11:52:11 -0500, John Craws wrote: >> >> Hi, >> >> Thank you for the clarification. Maybe I'm just not correctly >> interpreting whether the object is in the cache or not. >> Here's the info you asked for, based on the config I posted >> previously. I'm downloading a +- 16M file. >> >> 1. Before downloading the object: >> >> john.craws@jjj:~/wget$ curl -I http://172.16.199.150/popeye.mp4 >> HTTP/1.0 200 OK >> Date: Mon, 21 Feb 2011 16:46:56 GMT >> Server: Apache/2.2.3 (Red Hat) >> Last-Modified: Thu, 24 Sep 2009 19:22:32 GMT >> ETag: "e2800c-1013726-47457c0c5ae00" >> Accept-Ranges: bytes >> Content-Length: 16856870 >> Content-Type: video/mp4 >> X-Cache: MISS from jnk >> Via: 1.0 jnk (squid/3.1.11) >> Connection: keep-alive >> >> >> john.craws@jjj:~/wget$ /opt/squid/bin/squidclient mgr:objects >> HTTP/1.0 200 OK >> Server: squid/3.1.11 >> Mime-Version: 1.0 >> Date: Mon, 21 Feb 2011 15:51:02 GMT >> Content-Type: text/plain >> Expires: Mon, 21 Feb 2011 15:51:02 GMT >> Last-Modified: Mon, 21 Feb 2011 15:51:02 GMT >> X-Cache: MISS from jnk >> Via: 1.0 jnk (squid/3.1.11) >> Connection: close >> >> (lists cached objects, no trace of the object -- normal). >> >> 2. Downloading the object >> >> john.craws@jjj:~/wget$ curl http://172.16.199.150/popeye.mp4 -o popeye.mp4 >> % Total % Received % Xferd Average Speed Time Time >> Time Current >> Dload Upload Total Spent Left >> Speed >> 100 16.0M 100 16.0M 0 0 2446k 0 0:00:06 0:00:06 >> --:--:-- 1634k >> >> 3. This time the object appears in the list >> >> john.craws@jjj:~/wget$ /opt/squid/bin/squidclient mgr:objects >> HTTP/1.0 200 OK >> Server: squid/3.1.11 >> Mime-Version: 1.0 >> Date: Mon, 21 Feb 2011 15:51:18 GMT >> Content-Type: text/plain >> Expires: Mon, 21 Feb 2011 15:51:18 GMT >> Last-Modified: Mon, 21 Feb 2011 15:51:18 GMT >> X-Cache: MISS from jnk >> Via: 1.0 jnk (squid/3.1.11) >> Connection: close >> >> (...) >> KEY 669AB801B7640FA80E4BA73193FDAC2A >> STORE_OK IN_MEMORY SWAPOUT_NONE PING_DONE >> CACHABLE,DISPATCHED,VALIDATED >> LV:1298303469 LU:1298303469 LM:1253820152 EX:-1 >> 0 locks, 0 clients, 1 refs >> Swap Dir -1, File 0XFFFFFFFF >> GET http://172.16.199.150/popeye.mp4 >> inmem_lo: 0 >> inmem_hi: 16857134 >> swapout: 0 bytes queued >> (...) >> >> >> 4. This time it's a HIT >> >> john.craws@jjj:~/wget$ curl -I http://172.16.199.150/popeye.mp4 >> HTTP/1.0 200 OK >> Date: Mon, 21 Feb 2011 15:51:09 GMT >> Server: Apache/2.2.3 (Red Hat) >> Last-Modified: Thu, 24 Sep 2009 19:22:32 GMT >> ETag: "e2800c-1013726-47457c0c5ae00" >> Accept-Ranges: bytes >> Content-Length: 16856870 >> Content-Type: video/mp4 >> Age: 16 >> X-Cache: HIT from jnk >> Via: 1.0 jnk (squid/3.1.11) >> Connection: keep-alive >> >> 5. access.log >> >> 1298304218.303 19 127.0.0.1 TCP_MISS/200 322 HEAD >> http://172.16.199.150/popeye.mp4 - DIRECT/172.16.199.150 video/mp4 >> 1298304222.694 0 127.0.0.1 TCP_MEM_HIT/200 329 HEAD >> http://172.16.199.150/popeye.mp4 - NONE/- video/mp4 >> >> I also notice the difference the major time difference between the two >> curl operations. >> > > Thanks for the detail. You are right about it being in the memory cache. > > What I expect to see with your config is that file pushed to disk, since it > is within the 17MB but over the 32KB. But you have no on-disk cache right? You are correct. I have (intentionally) no disk cache (no cache_dir directive). I expected the object to be discarded. > > Something funky is going on with the swapout. > > I think there are 2 bugs visible here, the easy one is that the config > parser is not detecting and warning about the global limit being larger than > the biggest specific limit. Second being the object not discarded when over > memory size and push to disk not possible. That's what I expected also. Let me know if I can do anything to help. Is it reasonable to open a bug? Thanks, John > > Amos > >> Thanks! >> >> John >> >> >> On Fri, Feb 18, 2011 at 5:56 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx> >> wrote: >>> >>> On 19/02/11 07:28, John Craws wrote: >>>> >>>> Hi, >>>> >>>> I have a squid 3.1.11 instance configured with no disk cache. >>>> Stripped down configuration below. >>>> >>>> >>>> >>>> >>>> #-------------------------------------------------------------------------------- >>>> # squid.conf >>>> >>>> >>>> >>>> #-------------------------------------------------------------------------------- >>>> >>>> shutdown_lifetime 0 seconds >>>> http_port 3128 >>>> http_access allow all >>>> forwarded_for transparent >>>> >>>> acl VIDEO-CONTENT rep_header Content-Type video/.+ >>>> >>>> maximum_object_size_in_memory 32 KB >>>> maximum_object_size 17 MB >>>> cache_mem 4 GB >>>> cache allow all >>>> debug_options ALL,1 >>>> >>>> >>>> >>>> >>>> #-------------------------------------------------------------------------------- >>>> # end squid.conf >>>> >>>> >>>> >>>> #-------------------------------------------------------------------------------- >>>> >>>> I use curl to request a file through the proxy. >>>> >>>> What I understood from the docs is that maximum_object_size limits the >>>> size of objects to disk cache and maximum_object_size_in_memory does >>>> so for RAM cache. >>>> >>>> I have no disk cache, so I tried to apply a limit using >>>> maximum_object_size_in_memory. It didn't work. However, it does work >>>> when I specify a limit using maximum_object_size. >>>> >>>> What am I doing wrong? >>> >>> You are almost correct. >>> maximum_object_size is a global limit which nothing stored anywhere can >>> evade. >>> maximum_object_size_in_memory is a RAM-only limit. The smaller of memory >>> and global limits is used for cache_mem. >>> >>> There are matching *-size limits on individual cache_dir in the latest >>> Squid that do the same for each disk dir. >>> >>> >>> It should be preventing long-term storage of anything over 32KB, provided >>> the size info is known in the headers. >>> Note that this does not limit objects which are currently in transit. >>> They >>> are stored for as long as that transit use needs, then discarded. >>> >>> Do you have a copy of the headers and a display of how you are >>> identifying >>> the failure please? >>> >>> Amos >>> -- >>> Please be using >>> Current Stable Squid 2.7.STABLE9 or 3.1.11 >>> Beta testers wanted for 3.2.0.5 >>> > >
