On Mon, 21 Feb 2011 11:52:11 -0500, John Craws wrote:
Hi,
Thank you for the clarification. Maybe I'm just not correctly
interpreting whether the object is in the cache or not.
Here's the info you asked for, based on the config I posted
previously. I'm downloading a +- 16M file.
1. Before downloading the object:
john.craws@jjj:~/wget$ curl -I http://172.16.199.150/popeye.mp4
HTTP/1.0 200 OK
Date: Mon, 21 Feb 2011 16:46:56 GMT
Server: Apache/2.2.3 (Red Hat)
Last-Modified: Thu, 24 Sep 2009 19:22:32 GMT
ETag: "e2800c-1013726-47457c0c5ae00"
Accept-Ranges: bytes
Content-Length: 16856870
Content-Type: video/mp4
X-Cache: MISS from jnk
Via: 1.0 jnk (squid/3.1.11)
Connection: keep-alive
john.craws@jjj:~/wget$ /opt/squid/bin/squidclient mgr:objects
HTTP/1.0 200 OK
Server: squid/3.1.11
Mime-Version: 1.0
Date: Mon, 21 Feb 2011 15:51:02 GMT
Content-Type: text/plain
Expires: Mon, 21 Feb 2011 15:51:02 GMT
Last-Modified: Mon, 21 Feb 2011 15:51:02 GMT
X-Cache: MISS from jnk
Via: 1.0 jnk (squid/3.1.11)
Connection: close
(lists cached objects, no trace of the object -- normal).
2. Downloading the object
john.craws@jjj:~/wget$ curl http://172.16.199.150/popeye.mp4 -o
popeye.mp4
% Total % Received % Xferd Average Speed Time Time
Time Current
Dload Upload Total Spent
Left Speed
100 16.0M 100 16.0M 0 0 2446k 0 0:00:06 0:00:06
--:--:-- 1634k
3. This time the object appears in the list
john.craws@jjj:~/wget$ /opt/squid/bin/squidclient mgr:objects
HTTP/1.0 200 OK
Server: squid/3.1.11
Mime-Version: 1.0
Date: Mon, 21 Feb 2011 15:51:18 GMT
Content-Type: text/plain
Expires: Mon, 21 Feb 2011 15:51:18 GMT
Last-Modified: Mon, 21 Feb 2011 15:51:18 GMT
X-Cache: MISS from jnk
Via: 1.0 jnk (squid/3.1.11)
Connection: close
(...)
KEY 669AB801B7640FA80E4BA73193FDAC2A
STORE_OK IN_MEMORY SWAPOUT_NONE PING_DONE
CACHABLE,DISPATCHED,VALIDATED
LV:1298303469 LU:1298303469 LM:1253820152 EX:-1
0 locks, 0 clients, 1 refs
Swap Dir -1, File 0XFFFFFFFF
GET http://172.16.199.150/popeye.mp4
inmem_lo: 0
inmem_hi: 16857134
swapout: 0 bytes queued
(...)
4. This time it's a HIT
john.craws@jjj:~/wget$ curl -I http://172.16.199.150/popeye.mp4
HTTP/1.0 200 OK
Date: Mon, 21 Feb 2011 15:51:09 GMT
Server: Apache/2.2.3 (Red Hat)
Last-Modified: Thu, 24 Sep 2009 19:22:32 GMT
ETag: "e2800c-1013726-47457c0c5ae00"
Accept-Ranges: bytes
Content-Length: 16856870
Content-Type: video/mp4
Age: 16
X-Cache: HIT from jnk
Via: 1.0 jnk (squid/3.1.11)
Connection: keep-alive
5. access.log
1298304218.303 19 127.0.0.1 TCP_MISS/200 322 HEAD
http://172.16.199.150/popeye.mp4 - DIRECT/172.16.199.150 video/mp4
1298304222.694 0 127.0.0.1 TCP_MEM_HIT/200 329 HEAD
http://172.16.199.150/popeye.mp4 - NONE/- video/mp4
I also notice the difference the major time difference between the
two
curl operations.
Thanks for the detail. You are right about it being in the memory
cache.
What I expect to see with your config is that file pushed to disk,
since it is within the 17MB but over the 32KB. But you have no on-disk
cache right?
Something funky is going on with the swapout.
I think there are 2 bugs visible here, the easy one is that the config
parser is not detecting and warning about the global limit being larger
than the biggest specific limit. Second being the object not discarded
when over memory size and push to disk not possible.
Amos
Thanks!
John
On Fri, Feb 18, 2011 at 5:56 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx>
wrote:
On 19/02/11 07:28, John Craws wrote:
Hi,
I have a squid 3.1.11 instance configured with no disk cache.
Stripped down configuration below.
#--------------------------------------------------------------------------------
# squid.conf
#--------------------------------------------------------------------------------
shutdown_lifetime 0 seconds
http_port 3128
http_access allow all
forwarded_for transparent
acl VIDEO-CONTENT Â Â Â Â Â rep_header Content-Type video/.+
maximum_object_size_in_memory 32 KB
maximum_object_size 17 MB
cache_mem 4 GB
cache allow all
debug_options ALL,1
#--------------------------------------------------------------------------------
# end squid.conf
#--------------------------------------------------------------------------------
I use curl to request a file through the proxy.
What I understood from the docs is that maximum_object_size limits
the
size of objects to disk cache and maximum_object_size_in_memory
does
so for RAM cache.
I have no disk cache, so I tried to apply a limit using
maximum_object_size_in_memory. It didn't work. However, it does
work
when I specify a limit using maximum_object_size.
What am I doing wrong?
You are almost correct.
Âmaximum_object_size is a global limit which nothing stored anywhere
can
evade.
Âmaximum_object_size_in_memory is a RAM-only limit. The smaller of
memory
and global limits is used for cache_mem.
ÂThere are matching *-size limits on individual cache_dir in the
latest
Squid that do the same for each disk dir.
It should be preventing long-term storage of anything over 32KB,
provided
the size info is known in the headers.
ÂNote that this does not limit objects which are currently in
transit. They
are stored for as long as that transit use needs, then discarded.
Do you have a copy of the headers and a display of how you are
identifying
the failure please?
Amos
--
Please be using
ÂCurrent Stable Squid 2.7.STABLE9 or 3.1.11
ÂBeta testers wanted for 3.2.0.5
