Search squid archive

Re: squid won't return cached even with refresh_pattern extra options override-lastmod override-expire ignore-reload ignore-no-store ignore-private store-stale

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 3/24/21 3:34 PM, Miroslaw Malinowski wrote:
> I thought about upper service but as is not required at the moment,
> introducing extra hop just to remove the header looks a bit like a
> hammer approach. I'll look into how easily I can amend the code as the
> other option is to introduce a proxy like a feature to the application,
> so either way, it is a code change. The only problem here is that it's
> an OPNSense squid service so I have to compile from source on BSD and
> then keep adding in manually each time they do the update.

At the risk of stating the obvious: If your feature is officially
accepted into Squid sources, then you would not have to keep adding it
manually (once the changes reach your Squid packaging source).

Alex.


> On Wed, Mar 24, 2021 at 7:11 PM Alex Rousskov wrote:
> 
>     On 3/24/21 2:49 PM, Miroslaw Malinowski wrote:
> 
>     > looking at the code and reading carefully your response, you're saying
>     > there is no way you can do it with squid.
> 
>     With Squid, your options include:
> 
>     1. Squid source code changes. Should not be too difficult and, IMO, a
>     high-quality implementation would deserve official acceptance because it
>     is a generally useful feature in line with existing control knobs.
>     https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F
> 
>     2. An adaptation service that removes Cache-Control:no-cache from the
>     response before Squid processes it:
>     https://wiki.squid-cache.org/SquidFaq/ContentAdaptation
> 
> 
>     HTH,
> 
>     Alex.
> 
>     > On Wed, Mar 24, 2021 at 6:28 PM Miroslaw Malinowski wrote:
>     >
>     >     Hi,
>     >
>     >     You've right yes it's revalidating as API server I'm
>     requesting data
>     >     is setting Cache-Control: no-cache. My question is how I can force
>     >     squid to cache and not validate as I know it's safe to do so. As
>     >     I've explained earlier we are making the same request and
>     receiving
>     >     the same response from 100+ server so as to reduce number of
>     >     requests to the external server we would like squid to cache the
>     >     response and issue a cached version.
>     >
>     >     2021/03/24 18:00:54.867 kid1| 22,3| refresh.cc(351) refreshCheck:
>     >     YES: Must revalidate stale object (origin set no-cache or private)
>     >
>     >     Mirek
>     >
>     >     On Wed, Mar 24, 2021 at 6:15 PM Alex Rousskov
>     >     <rousskov@xxxxxxxxxxxxxxxxxxxxxxx
>     <mailto:rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
>     >     <mailto:rousskov@xxxxxxxxxxxxxxxxxxxxxxx
>     <mailto:rousskov@xxxxxxxxxxxxxxxxxxxxxxx>>> wrote:
>     >
>     >         On 3/24/21 12:48 PM, Miroslaw Malinowski wrote:
>     >
>     >         > Probably, me missing on something silly or it can't be done
>     >         but I don't
>     >         > know why but squid won't return the cached version even
>     when I
>     >         turn all
>     >         > override options ON in refresh_pattern.
>     >
>     >         AFAICT, no configuration options that can disable
>     revalidation of
>     >         Cache-Control:no-cache responses. refresh_pattern does not
>     have an
>     >         (equivalent of) "ignore-no-cache-in-responses" option.
>     >
>     >         IIRC, older Squids were violating an HTTP MUST by
>     forgetting to
>     >         revalidate Cache-Control:no-cache responses, but that was
>     fixed
>     >         in [1].
>     >         Your Squid version has that fix.
>     >
>     >         [1]
>     >       
>      https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa
>     <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa>
>     >       
>      <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa
>     <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa>>
>     >
>     >
>     >         > With debug, I can see the rule is matched and the cache is
>     >         fresh but
>     >         > still in access.log is TCP_REFRESH_MODIFIED
>     >
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 11,3| http.cc(982)
>     >         > haveParsedReplyHeaders: decided: cache positively and share
>     >         because
>     >
>     >         FYI: You are looking at cache.log lines logged _after_
>     Squid has
>     >         already
>     >         decided to refresh the cached version. If you want to analyze
>     >         why Squid
>     >         decided to refresh the cached version, you should look
>     _before_
>     >         Squid
>     >         logged the request to the server (and before any FwdState.cc
>     >         lines). I
>     >         have not checked the details, but I bet that your Squid
>     revalidates
>     >         because of Cache-Control:no-cache in the response. Look for
>     >         "YES: Must
>     >         revalidate stale object".
>     >
>     >
>     >         HTH,
>     >
>     >         Alex.
>     >
>     >         > squid conf:
>     >         > refresh_pattern -i <URL> 4320 80% 129600 override-lastmod
>     >         > override-expire ignore-reload ignore-no-store ignore-private
>     >         store-stale
>     >         >
>     >         > curl headers:
>     >         > curl --insecure --verbose --request GET --url 'URL'
>     >/dev/null
>     >         > * TCP_NODELAY set
>     >         > * ALPN, offering h2
>     >         > * ALPN, offering http/1.1
>     >         > * successfully set certificate verify locations:
>     >         > *   CAfile: /etc/ssl/certs/ca-certificates.crt
>     >         >  CApath: /etc/ssl/certs
>     >         > } [5 bytes data]
>     >         > * TLSv1.3 (OUT), TLS handshake, Client hello (1):
>     >         > } [512 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, Server hello (2):
>     >         > { [122 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
>     >         > { [6 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, Certificate (11):
>     >         > { [1956 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, CERT verify (15):
>     >         > { [78 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, Finished (20):
>     >         > { [52 bytes data]
>     >         > * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
>     >         > } [1 bytes data]
>     >         > * TLSv1.3 (OUT), TLS handshake, Finished (20):
>     >         > } [52 bytes data]
>     >         > * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
>     >         >
>     >         >> GET URL HTTP/1.1
>     >         >> Host: URL
>     >         >> User-Agent: curl/7.68.0
>     >         >> Accept: */*
>     >         >>
>     >         > { [5 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
>     >         > { [217 bytes data]
>     >         > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
>     >         > { [217 bytes data]
>     >         > * old SSL session ID is stale, removing
>     >         > { [5 bytes data]
>     >         > * Mark bundle as not supporting multiuse
>     >         > < HTTP/1.1 200 OK
>     >         > < Cache-Control: no-cache
>     >         > < Content-Type: application/json
>     >         > < X-Cloud-Trace-Context: d3c27833b8b4312ce31a2dbae7e12fd0
>     >         > < Date: Wed, 24 Mar 2021 15:04:34 GMT
>     >         > < Server: Google Frontend
>     >         > < Content-Length: 7950
>     >         > < X-Cache: MISS from server
>     >         > < X-Cache-Lookup: HIT from server
>     >         > < Via: 1.1 server (squid/4.14)
>     >         > < Connection: keep-alive
>     >         >
>     >         > access log:
>     >         > 243 172.16.230.249 TCP_REFRESH_MODIFIED/200 8328 GET URL -
>     >         > ORIGINAL_DST/IP application/json
>     >         >
>     >         > cache log:
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 11,3| http.cc(982)
>     >         > haveParsedReplyHeaders: decided: cache positively and share
>     >         because
>     >         > refresh check returned cacheable; HTTP status 200
>     >         e:=p2V/0x34868914670*3       
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(470)
>     >         refreshCheck:
>     >         > returning FRESH_MIN_RULE       
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(455)
>     >         refreshCheck:
>     >         > Object isn't stale..   
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(327)
>     >         refreshCheck:
>     >         > Staleness = -1         
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(199)
>     >         > refreshStaleness: FRESH: age (60 sec) is less than
>     configured
>     >         minimum
>     >         > (259200 sec)   
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(166)
>     >         > refreshStaleness: No explicit expiry given, using
>     heuristics to
>     >         > determine freshness    
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(307)
>     >         refreshCheck:
>     >         > entry->timestamp: Wed, 24 Mar 2021 15:04:34 GMT        
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(305)
>     >         refreshCheck:
>     >         > check_time: Wed, 24 Mar 2021 15:05:34 GMT      
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(303)
>     >         refreshCheck:
>     >         > age: 60        
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(301)
>     >         refreshCheck:
>     >         > Matched 'URL 259200 80%% 7776000'      
>     >         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3|
>     refresh.cc(279)
>     >         refreshCheck:
>     >         > checking freshness of URI: https://URL <https://URL>
>     <https://URL <https://URL>>
>     >         <https://URL <https://URL> <https://URL <https://URL>>>
>     >         >
>     >         >
>     >         > _______________________________________________
>     >         > squid-users mailing list
>     >         > squid-users@xxxxxxxxxxxxxxxxxxxxx
>     <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx>
>     >         <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx
>     <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx>>
>     >         > http://lists.squid-cache.org/listinfo/squid-users
>     <http://lists.squid-cache.org/listinfo/squid-users>
>     >         <http://lists.squid-cache.org/listinfo/squid-users
>     <http://lists.squid-cache.org/listinfo/squid-users>>
>     >         >
>     >
> 

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users




[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux