On 3/24/21 3:34 PM, Miroslaw Malinowski wrote: > I thought about upper service but as is not required at the moment, > introducing extra hop just to remove the header looks a bit like a > hammer approach. I'll look into how easily I can amend the code as the > other option is to introduce a proxy like a feature to the application, > so either way, it is a code change. The only problem here is that it's > an OPNSense squid service so I have to compile from source on BSD and > then keep adding in manually each time they do the update. At the risk of stating the obvious: If your feature is officially accepted into Squid sources, then you would not have to keep adding it manually (once the changes reach your Squid packaging source). Alex. > On Wed, Mar 24, 2021 at 7:11 PM Alex Rousskov wrote: > > On 3/24/21 2:49 PM, Miroslaw Malinowski wrote: > > > looking at the code and reading carefully your response, you're saying > > there is no way you can do it with squid. > > With Squid, your options include: > > 1. Squid source code changes. Should not be too difficult and, IMO, a > high-quality implementation would deserve official acceptance because it > is a generally useful feature in line with existing control knobs. > https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F > > 2. An adaptation service that removes Cache-Control:no-cache from the > response before Squid processes it: > https://wiki.squid-cache.org/SquidFaq/ContentAdaptation > > > HTH, > > Alex. > > > On Wed, Mar 24, 2021 at 6:28 PM Miroslaw Malinowski wrote: > > > > Hi, > > > > You've right yes it's revalidating as API server I'm > requesting data > > is setting Cache-Control: no-cache. My question is how I can force > > squid to cache and not validate as I know it's safe to do so. As > > I've explained earlier we are making the same request and > receiving > > the same response from 100+ server so as to reduce number of > > requests to the external server we would like squid to cache the > > response and issue a cached version. > > > > 2021/03/24 18:00:54.867 kid1| 22,3| refresh.cc(351) refreshCheck: > > YES: Must revalidate stale object (origin set no-cache or private) > > > > Mirek > > > > On Wed, Mar 24, 2021 at 6:15 PM Alex Rousskov > > <rousskov@xxxxxxxxxxxxxxxxxxxxxxx > <mailto:rousskov@xxxxxxxxxxxxxxxxxxxxxxx> > > <mailto:rousskov@xxxxxxxxxxxxxxxxxxxxxxx > <mailto:rousskov@xxxxxxxxxxxxxxxxxxxxxxx>>> wrote: > > > > On 3/24/21 12:48 PM, Miroslaw Malinowski wrote: > > > > > Probably, me missing on something silly or it can't be done > > but I don't > > > know why but squid won't return the cached version even > when I > > turn all > > > override options ON in refresh_pattern. > > > > AFAICT, no configuration options that can disable > revalidation of > > Cache-Control:no-cache responses. refresh_pattern does not > have an > > (equivalent of) "ignore-no-cache-in-responses" option. > > > > IIRC, older Squids were violating an HTTP MUST by > forgetting to > > revalidate Cache-Control:no-cache responses, but that was > fixed > > in [1]. > > Your Squid version has that fix. > > > > [1] > > > https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa > <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa> > > > <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa > <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa>> > > > > > > > With debug, I can see the rule is matched and the cache is > > fresh but > > > still in access.log is TCP_REFRESH_MODIFIED > > > > > 2021-03-24T15:04:34 squid .710 kid1| 11,3| http.cc(982) > > > haveParsedReplyHeaders: decided: cache positively and share > > because > > > > FYI: You are looking at cache.log lines logged _after_ > Squid has > > already > > decided to refresh the cached version. If you want to analyze > > why Squid > > decided to refresh the cached version, you should look > _before_ > > Squid > > logged the request to the server (and before any FwdState.cc > > lines). I > > have not checked the details, but I bet that your Squid > revalidates > > because of Cache-Control:no-cache in the response. Look for > > "YES: Must > > revalidate stale object". > > > > > > HTH, > > > > Alex. > > > > > squid conf: > > > refresh_pattern -i <URL> 4320 80% 129600 override-lastmod > > > override-expire ignore-reload ignore-no-store ignore-private > > store-stale > > > > > > curl headers: > > > curl --insecure --verbose --request GET --url 'URL' > >/dev/null > > > * TCP_NODELAY set > > > * ALPN, offering h2 > > > * ALPN, offering http/1.1 > > > * successfully set certificate verify locations: > > > * CAfile: /etc/ssl/certs/ca-certificates.crt > > > CApath: /etc/ssl/certs > > > } [5 bytes data] > > > * TLSv1.3 (OUT), TLS handshake, Client hello (1): > > > } [512 bytes data] > > > * TLSv1.3 (IN), TLS handshake, Server hello (2): > > > { [122 bytes data] > > > * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): > > > { [6 bytes data] > > > * TLSv1.3 (IN), TLS handshake, Certificate (11): > > > { [1956 bytes data] > > > * TLSv1.3 (IN), TLS handshake, CERT verify (15): > > > { [78 bytes data] > > > * TLSv1.3 (IN), TLS handshake, Finished (20): > > > { [52 bytes data] > > > * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): > > > } [1 bytes data] > > > * TLSv1.3 (OUT), TLS handshake, Finished (20): > > > } [52 bytes data] > > > * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 > > > > > >> GET URL HTTP/1.1 > > >> Host: URL > > >> User-Agent: curl/7.68.0 > > >> Accept: */* > > >> > > > { [5 bytes data] > > > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): > > > { [217 bytes data] > > > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): > > > { [217 bytes data] > > > * old SSL session ID is stale, removing > > > { [5 bytes data] > > > * Mark bundle as not supporting multiuse > > > < HTTP/1.1 200 OK > > > < Cache-Control: no-cache > > > < Content-Type: application/json > > > < X-Cloud-Trace-Context: d3c27833b8b4312ce31a2dbae7e12fd0 > > > < Date: Wed, 24 Mar 2021 15:04:34 GMT > > > < Server: Google Frontend > > > < Content-Length: 7950 > > > < X-Cache: MISS from server > > > < X-Cache-Lookup: HIT from server > > > < Via: 1.1 server (squid/4.14) > > > < Connection: keep-alive > > > > > > access log: > > > 243 172.16.230.249 TCP_REFRESH_MODIFIED/200 8328 GET URL - > > > ORIGINAL_DST/IP application/json > > > > > > cache log: > > > 2021-03-24T15:04:34 squid .710 kid1| 11,3| http.cc(982) > > > haveParsedReplyHeaders: decided: cache positively and share > > because > > > refresh check returned cacheable; HTTP status 200 > > e:=p2V/0x34868914670*3 > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(470) > > refreshCheck: > > > returning FRESH_MIN_RULE > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(455) > > refreshCheck: > > > Object isn't stale.. > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(327) > > refreshCheck: > > > Staleness = -1 > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(199) > > > refreshStaleness: FRESH: age (60 sec) is less than > configured > > minimum > > > (259200 sec) > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(166) > > > refreshStaleness: No explicit expiry given, using > heuristics to > > > determine freshness > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(307) > > refreshCheck: > > > entry->timestamp: Wed, 24 Mar 2021 15:04:34 GMT > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(305) > > refreshCheck: > > > check_time: Wed, 24 Mar 2021 15:05:34 GMT > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(303) > > refreshCheck: > > > age: 60 > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(301) > > refreshCheck: > > > Matched 'URL 259200 80%% 7776000' > > > 2021-03-24T15:04:34 squid .710 kid1| 22,3| > refresh.cc(279) > > refreshCheck: > > > checking freshness of URI: https://URL <https://URL> > <https://URL <https://URL>> > > <https://URL <https://URL> <https://URL <https://URL>>> > > > > > > > > > _______________________________________________ > > > squid-users mailing list > > > squid-users@xxxxxxxxxxxxxxxxxxxxx > <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx> > > <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx > <mailto:squid-users@xxxxxxxxxxxxxxxxxxxxx>> > > > http://lists.squid-cache.org/listinfo/squid-users > <http://lists.squid-cache.org/listinfo/squid-users> > > <http://lists.squid-cache.org/listinfo/squid-users > <http://lists.squid-cache.org/listinfo/squid-users>> > > > > > > _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
