Hi - I went through your same thinking as you described below. I checked the Expires header from the server and we do not set one. I checked via Fiddler web debug tool. I also verified with the dev guys here regarding no Expires header. I have set the min and max via refresh_pattern because of the absence of the Expires header thinking that Squid would keep it FRESH. I recently posted the details of the Squid logs, but will re-post it here so that the thread will follow properly (it may help others when search). Hoping someone could explain the behavior I see below. File Properties of video file (Pacific Daylight Time (PDT)) Created On: Monday, September 22, 2008, 8:59:35 AM Modified On: Monday, September 22, 2008, 8:59:35 AM Accessed On: Today, September 24, 2008, 3:53:12 AM ******************************************************************* Wget Grabs File (Time in India Standard Time (IST)) --04:38:35-- http://ftp.mydomain.com/websites/data/myvideofile.vid => `/WGET/Temp/myvideofile.vid' 04:38:54 (93.91 KB/s) - `/WGET/Temp/myvideofile.vid' saved [1791244/1791244] The access.log confirms initial pre-fetch by wget. 1222124934.241 18968 192.168.200.4 TCP_MISS/200 1791684 GET http://ftp.mydomain.com/websites/data/myvideofile.vid - DIRECT/69.43.136.41 video/jpeg UTC = Mon, 22 Sep 2008 23:08:54 GMT The store.log shows a write from memory to disk: 1222124934.241 SWAPOUT 00 00057B65 1E18E35BDC9307C6BC3FBEFD5B4120A3 200 1222124765 1222099175 -1 video/jpeg 1791244/1791244 GET http://ftp.mydomain.com/websites/data/myvideofile.vid UTC = Mon, 22 Sep 2008 23:08:54 GMT ******************************************************************* Then Store.log shows release or removal from cache: 1222253725.068 RELEASE 00 00057B65 605FAC36E93B0CDE81902BBC6C5EC71A 200 1222124765 1222099175 -1 video/jpeg 1791244/-279 GET http://ftp.mydomain.com/websites/data/myvideofile.vid UTC = Wed, 24 Sep 2008 10:55:25 GMT Notice the -1 for expiration header (I do not set one on the object). My min age is 5 days so I'm not sure why the object would be released from cache in less than 2 days. If the object was released from cache, when the user tried to access file, Squid reports TCP_REFRESH_MISS, which to me means that it was found in cache but when it sends a If-Modified-Since request, it thinks that the file has been modified (which it was not as seen by the lastmod date indicated in the store.log below. ******************************************************************* User accessed file (access.log): 1222253742.005 17275 192.168.200.52 TCP_REFRESH_MISS/200 1791688 GET http://ftp.mydomain.com/websites/data/myvideofile.vid - DIRECT/69.43.136.41 video/jpeg UTC = Wed, 24 Sep 2008 10:55:42 GMT Then store.log shows a write to disk 1222253742.005 SWAPOUT 00 00088336 1E18E35BDC9307C6BC3FBEFD5B4120A3 200 1222253575 1222099175 -1 video/jpeg 1791244/1791244 GET http://ftp.mydomain.com/websites/data/myvideofile.vid UTC = Wed, 24 Sep 2008 10:55:42 GMT datehdr: Wed, 24 Sep 2008 10:55:55 GMT lastmod: Mon, 22 Sep 2008 15:59:35 GMT ----- Original Message ---- From: Michael Alger <squid@xxxxxxxxxxx> To: squid-users@xxxxxxxxxxxxxxx Sent: Wednesday, September 24, 2008 4:49:38 AM Subject: Re: Object becomes STALE: refresh_pattern min and max On Wed, Sep 24, 2008 at 03:39:16AM -0700, BUI18 wrote: > I have squid box with tons of disk for the cache_dir > (hundreds of GB). I use wget to perform some pre-fetching of large > video files. I've set the min and max age to 5 days and 7 days (in > minutes). And although I have plenty of disk space available, I still > receive TCP_REFRESH_MISS for files that had been pre-fetched and later > accessed the same day. Does anyone know why Squid would consider it as > STALE? I thought that by setting the min value for refresh_pattern for > the video file would guarantee freshness. Not only does the cache > consider it STALE, it then goes and pre-fetches a new copy even though > I know that the video file has not changed. Any help would be greatly > appreciated. Thanks. The fact that it's doing TCP_REFRESH_xxx means squid does have a cached copy which it considers potentially stale. So it's sending an If-Modified-Since request to the origin server. The origin is then either saying "yes, it's been modified since you retrieved it -- here's a new one"; or it has no idea how to handle IMS and is sending the whole object regardless. What Expires: header is the server sending? You can use the -S switch with wget to show the server response headers when you're doing your pre-fetch. If they look fine, maybe keep a copy of them and compare later in the day when squid decides it needs a new one. I assume you're setting the min and max age via refresh_pattern lines? Remember that these are only used in the absence of an Expires header.
