BUI18 wrote:
Hi - Thanks for responding. URL for video file never changes.
What release of Squid?
Did you check the Expires header properly from the transfer rather than
from the (apparently untrustworthy) info in the store log?
I did some more checking in the Squid logs and this is what I noticed:
File Properties of video file (Pacific Daylight Time (PDT))
Created On: Monday, September 22, 2008, 8:59:35 AM
Modified On: Monday, September 22, 2008, 8:59:35 AM
Accessed On: Today, September 24, 2008, 3:53:12 AM
*******************************************************************
Wget Grabs File (Time in India Standard Time (IST))
--04:38:35-- http://ftp.mydomain.com/websites/data/myvideofile.vid
=> `/WGET/Temp/myvideofile.vid'
04:38:54 (93.91 KB/s) - `/WGET/Temp/myvideofile.vid' saved [1791244/1791244]
The access.log confirms initial pre-fetch by wget.
1222124934.241 18968 192.168.200.4 TCP_MISS/200 1791684 GET http://ftp.mydomain.com/websites/data/myvideofile.vid - DIRECT/69.43.136.41 video/jpeg
UTC = Mon, 22 Sep 2008 23:08:54 GMT
The store.log shows a write from memory to disk:
1222124934.241 SWAPOUT 00 00057B65 1E18E35BDC9307C6BC3FBEFD5B4120A3 200 1222124765 1222099175 -1 video/jpeg 1791244/1791244 GET http://ftp.mydomain.com/websites/data/myvideofile.vid
UTC = Mon, 22 Sep 2008 23:08:54 GMT
*******************************************************************
Then Store.log shows release or removal from cache:
1222253725.068 RELEASE 00 00057B65 605FAC36E93B0CDE81902BBC6C5EC71A 200 1222124765 1222099175 -1 video/jpeg 1791244/-279 GET http://ftp.mydomain.com/websites/data/myvideofile.vid
UTC = Wed, 24 Sep 2008 10:55:25 GMT
Notice the -1 for expiration header (I do not set one on the object). My min age is 5 days so I'm not sure why the object would be released from cache in less than 2 days.
If the object was released from cache, when the user tried to access file, Squid reports TCP_REFRESH_MISS, which to me means that it was found in cache but when it sends a If-Modified-Since request, it thinks that the file has been modified (which it was not as seen by the lastmod date indicated in the store.log below.
*******************************************************************
User accessed file (access.log):
1222253742.005 17275 192.168.200.52 TCP_REFRESH_MISS/200 1791688 GET http://ftp.mydomain.com/websites/data/myvideofile.vid - DIRECT/69.43.136.41 video/jpeg
UTC = Wed, 24 Sep 2008 10:55:42 GMT
Then store.log shows a write to disk
1222253742.005 SWAPOUT 00 00088336 1E18E35BDC9307C6BC3FBEFD5B4120A3 200 1222253575 1222099175 -1 video/jpeg 1791244/1791244 GET http://ftp.mydomain.com/websites/data/myvideofile.vid
UTC = Wed, 24 Sep 2008 10:55:42 GMT
datehdr: Wed, 24 Sep 2008 10:55:55 GMT
lastmod: Mon, 22 Sep 2008 15:59:35 GMT
Anyone with ideas on why this behavior occurs?
thanks
----- Original Message ----
From: Itzcak Pechtalt <itzcak@xxxxxxxxx>
To: Squid Users <squid-users@xxxxxxxxxxxxxxx>
Sent: Wednesday, September 24, 2008 4:35:59 AM
Subject: Re: Object becomes STALE: refresh_pattern min and max
On Wed, Sep 24, 2008 at 1:39 PM, BUI18 <lbui18@xxxxxxxxx> wrote:
Hi -
I have squid box with tons of disk for the cache_dir
(hundreds of GB). I use wget to perform some pre-fetching of large
video files. I've set the min and max age to 5 days and 7 days (in
minutes). And although I have plenty of disk space available, I still
receive TCP_REFRESH_MISS for files that had been pre-fetched and later
accessed the same day. Does anyone know why Squid would consider it as
STALE? I thought that by setting the min value for refresh_pattern for
the video file would guarantee freshness. Not only does the cache
consider it STALE, it then goes and pre-fetches a new copy even though
I know that the video file has not changed. Any help would be greatly
appreciated. Thanks.
Hi,
Check if the video URL changes from request to request. In YouTube
video even if the main URL is the same, there is request ID in URL who
changes per request.
Itzcak
--
Please use Squid 2.7.STABLE4 or 3.0.STABLE9
