I run a transit prediction web app <https://www.transsee.ca/>. It
connects to a variety of web APIs to collect the real time transit data
it needs. The app's activities are split among many processes. They
currently uses libcurl to connect to squid for caching (often for as
little as 10-30 seconds) and benefits of connection sharing.
There is still cases where data isn't being cached no matter what I do.
It is https data, but I am able to cache other https pages like
https://cdn.mbta.com/realtime/Alerts.pb
The refresh_pattern:
refresh_pattern . 60 99999% 7200 override-expire
override-lastmod reload-into-ims ignore-reload ignore-no-cache
ignore-no-store ignore-private ignore-auth store-stale
The http headers from curl of an example where it is not being cached:
* Trying 127.0.0.1:3128...
* Connected to 127.0.0.1 (127.0.0.1) port 3128 (#0)
> GET https://api.transport.nsw.gov.au/v1/gtfs/alerts/buses HTTP/1.1
Host: 127.0.0.1:3128
User-Agent: curl/7.77.0 (+https://www.transsee.ca/)
Accept: */*
Accept-Encoding: gzip
Authorization: apikey 2eYEqXXxOPEDChnpeF7sZL2aR8moD2DtdNmn
Cache-Control: max-age=60
Content-Encoding: aes128gcm
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Sun, 20 Jun 2021 17:52:14 GMT
< Content-Type: application/protobuf
< Content-Length: 7455
< Set-Cookie:
AWSALB=EEAv0EXNaGzBkgAPS1JzrAiQZi7aKDk063qjM4ApeOuIArbwQ+s17/jimuq7AzzuC+a35t5Lx/y+H5YMqTC9bL6LqS4Y75auumXt7LWnvSFE/SMXQd/OFa6TSCW9;
Expires=Sun, 27 Jun 2021 17:52:14 GMT; Path=/
< Set-Cookie:
AWSALBCORS=EEAv0EXNaGzBkgAPS1JzrAiQZi7aKDk063qjM4ApeOuIArbwQ+s17/jimuq7AzzuC+a35t5Lx/y+H5YMqTC9bL6LqS4Y75auumXt7LWnvSFE/SMXQd/OFa6TSCW9;
Expires=Sun, 27 Jun 2021 17:52:14 GMT; Path=/; SameSite=None; Secure
< Server: Apache-Coyote/1.1
< X-Powered-By: Express
< Access-Control-Allow-Credentials: true
< ETag: W/"ab70-8SI2GdBV4SJG4edSc4E5W8LBJWk"
< Vary: Accept-Encoding
< X-Cache: Hit from cloudfront
< X-Amz-Cf-Pop: SYD1-C1
< X-Amz-Cf-Id: hCoQckLsNONQMpgPr2kXJVdTDHu98jxl-rPXqV_PHB2vTCEomAd-Nw==
< Age: 35
< Access-Control-Allow-Origin: *
< Content-Encoding: gzip
< X-Cache: MISS from transsee
< X-Cache-Lookup: MISS from transsee:3128
< Via: 1.1 359a113ca166631b42f31a0f2e6a1aab.cloudfront.net (CloudFront),
1.1 transsee (squid/4.15)
< Connection: keep-alive
Here is a sample from the Squid access log:
1624212034.891 246 127.0.0.1 59216 TCP_MISS/200 8517 GET
https://api.transport.nsw.gov.au/v1/gtfs/alerts/buses -
HIER_DIRECT/52.65.222.24 application/protobuf
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
