Re: TCP_MISS_ABORTED/502

[Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>]

On 2024-07-12 13:03, Ben Toms wrote:

> So the issue seems to be caching content that requires authentication

The client is getting an error response from Squid. That error is 
probably not related to caching decisions. I do not recommend focusing 
on caching at this stage of triage. I recommend addressing that error first.


> The question here is, can squid cache items that require authentication 
> to access?

Yes, in some cases. To know whether your case qualifies, I asked for the 
response headers. That led to the discovery that there are none (from 
child Squid point of view). If you really want to investigate the 
caching angle in parallel with solving ERR_READ_ERROR/WITH_SERVER, then 
try to obtain HTTP response headers that the origin server responds (to 
the parent cache) with.


HTH,

Alex.


> *From: *Ben Toms <ben@xxxxxxxxxxx>
> *Date: *Friday, 12 July 2024 at 17:56
> *To: *Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>, 
> squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx>
> *Subject: *Re:  TCP_MISS_ABORTED/502
>
> So, with the below config:
>
> https_port 443 accel protocol=HTTPS tls-cert=/usr/local/squid/client.pem 
> tls-key=/usr/local/squid/client.key
>
> cache_peer public.server.fqdn parent 443 0 no-query originserver 
> no-digest no-netdb-exchange tls login=PASSTHRU name=myAccel 
> forceddomain=public.server.fqdn
>
> acl our_sites dstdomain local.server.fqdn
>
> http_access allow our_sites
>
> cache_peer_access myAccel allow our_sites
>
> cache_peer_access myAccel deny all
>
> cache_dir ufs /usr/local/squid/var/cache 100000 16 256
>
> cache_mem 500 MB
>
> maximum_object_size_in_memory 50000 KB
>
> refresh_pattern .               0       20%     4320
>
> debug_options 11,2
>
> I can see the below in /var/log/squid/cache.log
>
> ----------
>
> 2024/07/12 16:49:57.056 kid1| 11,2| http.cc(1263) readReply: conn12 
> local=client.ip:56670 remote=public.ip.of.public.server:443 
> FIRSTUP_PARENT FD 14 flags=1: read failure: (0) No error.
>
> 2024/07/12 16:49:57.056 kid1| 11,2| Stream.cc(273) sendStartOfMessage: 
> HTTP Client conn9 local=client.ip:443 remote=local.server.ip:59158 FD 13 
> flags=1
>
> 2024/07/12 16:49:57.056 kid1| 11,2| Stream.cc(274) sendStartOfMessage: 
> HTTP Client REPLY:
>
> ---------
>
> HTTP/1.1 502 Bad Gateway
>
> Server: squid/6.6
>
> Mime-Version: 1.0
>
> Date: Fri, 12 Jul 2024 16:49:57 GMT
>
> Content-Type: text/html;charset=utf-8
>
> Content-Length: 3629
>
> X-Squid-Error: ERR_READ_ERROR 0
>
> Vary: Accept-Language
>
> Content-Language: en
>
> Cache-Status: local.server;detail=mismatch
>
> Via: 1.1 local.server (squid/6.6)
>
> Connection: keep-alive
>
> ----------
>
> The apache server still shows a 200 for the request:
>
> [12/Jul/2024:17:49:57 +0100] "GET /path/to/file HTTP/1.1" 200 10465 "-" 
> "curl/8.7.1"
>
> And this is when testing via:
>
> curl -D - https://local.server.fqdn/path/to/file 
> <https://local.server.fqdn/path/to/file> -H "Authorization: Basic 
> base64auth" -o /dev/null
>
> Regards,
>
> Ben.
>
> *From: *Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
> *Date: *Friday, 12 July 2024 at 17:36
> *To: *Ben Toms <ben@xxxxxxxxxxx>, squid-users@xxxxxxxxxxxxxxxxxxxxx 
> <squid-users@xxxxxxxxxxxxxxxxxxxxx>
> *Subject: *Re:  TCP_MISS_ABORTED/502
>
> On 2024-07-12 12:14, Ben Toms wrote:
>
> > Which log should those be found?
>
> cache.log (if they are present)
>
>
> > Can’t see “HTTP Server RESPONSE” in the access.log or cache.log.
>
> Sigh. This is one of the reasons I avoid asking folks to study logs
> themselves, even ALL,2 logs...
>
> If that line is not in cache.log, then child Squid probably did not
> receive a response from parent Squid, or could not parse that response.
> A full debugging log should give us more information.
>
> Alex.
>
>
> > *From: *squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> on 
> > behalf of Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
> > *Date: *Friday, 12 July 2024 at 17:11
> > *To: *squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx>
> > *Subject: *Re:  TCP_MISS_ABORTED/502
> >
> > On 2024-07-12 11:38, Ben Toms wrote:
> > > Think I made the changes Alex requested:
> > >
> > > 12/Jul/2024:15:36:31 +0000.640 local.server.ip TCP_MISS_ABORTED/502 3974 
> > > GET https://local.server.fqdn/path/to/file 
> <https://local.server.fqdn/path/to/file>
> > <https://local.server.fqdn/path/to/file 
> <https://local.server.fqdn/path/to/file>> -
> > > FIRSTUP_PARENT/public.ip.of.public.server text/html 
> > > ERR_READ_ERROR/WITH_SERVER
> >
> > Thank you for using Squid v6 for this test.
> >
> > Unfortunately, due to Squid logging bugs, ERR_READ_ERROR/WITH_SERVER
> > does not always mean what it says. For example, parent Squid could have
> > closed the child-parent connection prematurely, but there could be other
> > reasons. A full debugging log should give us more information.
> >
> >
> > > 2024/07/12 14:57:08.678 kid1| 11,2| Stream.cc(274) sendStartOfMessage: 
> > > HTTP Client REPLY:
> >
> > This is a child proxy response to the client. We need parent response to
> > the child proxy. Look for "HTTP Server RESPONSE" lines instead.
> >
> >
> > HTH,
> >
> > Alex.
> >
> >
> >
> > > ---------
> > >
> > > HTTP/1.1 502 Bad Gateway
> > >
> > > Server: squid/6.6
> > >
> > > Mime-Version: 1.0
> > >
> > > Date: Fri, 12 Jul 2024 14:57:08 GMT
> > >
> > > Content-Type: text/html;charset=utf-8
> > >
> > > Content-Length: 3629
> > >
> > > X-Squid-Error: ERR_READ_ERROR 0
> > >
> > > Vary: Accept-Language
> > >
> > > Content-Language: en
> > >
> > > Cache-Status: squid.host;detail=mismatch
> > >
> > > Via: 1.1 squid.host (squid/6.6)
> > >
> > > Connection: keep-alive
> > >
> > > ----------
> > >
> > > Regards,
> > >
> > > Ben.
> > >
> > > *From: *squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> on 
> > > behalf of Amos Jeffries <squid3@xxxxxxxxxxxxx>
> > > *Date: *Friday, 12 July 2024 at 15:22
> > > *To: *squid-users@xxxxxxxxxxxxxxxxxxxxx <squid-users@xxxxxxxxxxxxxxxxxxxxx>
> > > *Subject: *Re:  TCP_MISS_ABORTED/502
> > >
> > >
> > > On 13/07/24 01:52, Alex Rousskov wrote:
> > > > On 2024-07-12 08:06, Ben Toms wrote:
> > > > > Seems that my issue is similar to - 
> > > > > https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication <https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication> <https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication <https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication>> <https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication <https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication <https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication>>>
> > > >
> > > > You are facing up to two problems:
> > > >
> > > > 1. Some authenticated responses are not cachable by Squid. Please share 
> > > > HTTP headers of the response in question.
> > >
> > > FYI, those can be obtained by configuring squid.conf with
> > >
> > >      debug_options 11,2
> > >
> > >
> > > Cheers
> > > Amos
> > >
> > >
> > > > 2. TCP_MISS_ABORTED/502 errors may delete a being-cached response. These 
> > > > can be bogus errors (essentially Squid logging bugs) or real ones (e.g., 
> > > > due to communication bugs, misconfiguration, or compatibility problems). 
> > > > I recommend adding %err_code/%err_detail to your logformat and sharing 
> > > > the corresponding access.log lines (obfuscated as needed).
> > > >
> > > > Sharing (privately if needed) a pointer to compressed ALL,9 cache.log 
> > > > while reproducing the issue using a single transaction may help us 
> > > > resolve all the unknowns:
> > > >
> > > > https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction <https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction> <https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction <https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction>> <https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction <https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction <https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction>>>
> > > >
> > > >
> > > > HTH,
> > > >
> > > > Alex.
> > >
> > >
> > > _______________________________________________
> > > squid-users mailing list
> > > squid-users@xxxxxxxxxxxxxxxxxxxxx
> > > https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>
> > <https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>>
> > > <https://lists.squid-cache.org/listinfo/squid-users 
> > <https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>>>
> > > _______________________________________________
> > > squid-users mailing list
> > > squid-users@xxxxxxxxxxxxxxxxxxxxx
> > > https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>
> > <https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>>
> > _______________________________________________
> > squid-users mailing list
> > squid-users@xxxxxxxxxxxxxxxxxxxxx
> > https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>
> > <https://lists.squid-cache.org/listinfo/squid-users 
> <https://lists.squid-cache.org/listinfo/squid-users>>
> >

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users