Search squid archive

Re: ACL / http_access rules stop work using Squid 6+

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Andre,

I did not receive your new reply, but I did find it in the mailing list archives in time to download the logs. Thank you for sharing them! FWIW, you may want to CC me in future emails (but I do not know whether that will help).


Something really strange is going on. Squid runtime logs (both versions) do not match the two identical "squid -k parse" logs you have shared earlier. For example, at runtime, Squid v5 checks the following rules, in this order:

1. http_access ... ArticaTSTSite ...
2. http_access ... NoCacheMGR ...
3. http_access ... LocalClient ...
4. http_access ... MyIPSrc ...
5. http_access ... internal_icons ...
6. http_access ... MgrInfoUri ...

The first 10 rules in "squid -k parse" output are different! I pasted them below and marked those that are missing in runtime output:

1. http_access deny ArticaTSTSite ArticaTSTSiteAll
2. http_access allow NoCacheMGR ToArticaWWW
3. http_access allow LocalClient NoCacheMGR
4. http_access allow MyIPSrc NoCacheMGR
5. http_access allow internal_icons
6. Missing: http_access allow internal_icons
7. Missing: http_access deny NormalPorts to_localhost  CVEFix all
8. Missing: http_access deny NormalPorts manager  ManagerDeny all
9. Missing: http_access allow MgRPort manager all
10. http_access allow MgRPort MgrInfoUri all

The differences continue...

N.B. Rules 5 and 6 in parsing logs are the same, but Squid does not know that (yet?). Squid will parse, consume, and evaluate both (if they are reached and do not match). In runtime logs, Squid v5 only evaluates the first one (#5) and then goes to a MgrInfoUri rule that is listed as #6 in Squid v5 runtime logs and as #10 in parse logs, skipping four rules.

Runtime logs do not contain just the ACL names. They also contain rule numbers assigned at parsing time. The number should be incremented for every http_access rule Squid parses. Those numbers do not match "squid -k parse" output, reflecting the same "missing rules" problem, and elevating my confidence that other rules are missing at runtime.


Squid v6 runtime logs show the same problem, but the differences start later (and are more profound). For example, Squid v6 logs do not show Group212 ACL being evaluated (rule #151 in parse logs) but do show AnnotateFinalAllow ACL evaluated (rule #168 in parse logs and rule #43 in v6 runtime logs). Group98 (rule #116 in parse logs) is also missing from v6 runtime logs.


Parsing logs show 168 http_access rules.
Running Squid v5 appears to have 163 http_access rules.
Running Squid v6 appears to have 43 http_access rules!

Bugs notwithstanding, Squid does not "skip" http_access rules that it successfully parsed/consumed at startup or reconfiguration. Squid does not evaluate higher-numbered rules after a rule matches, but that is not what it happening here: For example, I see sequences where Squid v6 checks 43 rules from the first (in all logs) ArticaTSTSite rule to the last (in all logs) AnnotateFinalAllow rule; the other 125 rules (seen in "squid -k parse" logs) appear to be "missing" from v6 runtime logs.


I speculate that one of these things have happened (in no particular order):

A. I am confused by these partial logs.


B. Squid failed to parse/consume some http_access rules in your configuration files. The set of rules Squid failed to parse differs across v5 and v6. The logs do not detail parsing, so I cannot validate this theory, but it would explain the missing rules. In the vast majority of cases, there ought to be at least some level-0/1 cache.log errors or warnings about failing to parse/consume an http_access rule!

One way to validate this theory is to compare "squid -k parse" output with http_access lines in mgr:config cache manager report received from a running Squid instance. You may need to modify squid.conf to allow that cache manager query (see cachemgr_passwd). Squids, especially older ones, have various bugs in mgr:config output so a manual comparison may be needed.

Another way is to study "squid -k parse -X -d9" output that should have more details for Squid v6. This is similar to the trick in (D) below, but without runtime debugging.


C. Your "squid -k parse" commands do not match the actual configurations that Squids are parsing and then using at runtime. The same mgr:config trick from (B) above can be used to validate this theory.


D. Both Squids are suffering from some serious bug. The effects of that bug result in different actual/used http_access rule sets in each version.

One way to confirm theory D is to study ALL,9 logs collected from Squid v6 with "squid ... -X -d9" command line option (or a similar trick that will include initial parsing in the log). The logs should detail at least one transaction that should evaluate most http_access rules. Such logs will address concern (A) as well, but you will probably have to share them privately if you are using production configuration/instance.


HTH,

Alex.


On 2024-04-15 19:49, Andre Bolinhas wrote:
Hi Alex,
Thnks for your reply.

Logs uploaded again, you can find it here.

https://we.tl/t-QiSKMgclOb

Best regards

On 15/04/2024 14:12, Alex Rousskov wrote:
On 2024-04-14 17:23, Andre Bolinhas wrote:

Any tip on this matter? I want to upgrade to squid 6.9 but due to this issue, i'm stuck.


Hi Andre,

    Please note that I did _not_ receive your email quoted below. It is in the email archive, so the problem is not on your end, but I just wanted to mention that I was not (knowingly) ignoring you.

> I have re-uploaded the cache.log files.

The files have expired again. I have reviewed the diff you shared, but cannot make further progress without those test logs. Hopefully, your next list post reaches me.

Alex.


On 01/04/2024 11:53, Andre Bolinhas wrote:

Hi Alex

Thanks for your help on the matter.


The logs archive you shared previously has expired, so I cannot double check, but from what I remember, the shared logs did not support the above assertion, so there may be more to the story here. However, to make progress, let's assume that v5 configuration files are identical to v6 configuration files.
If you want, I can run the same test with in a different debug parameters, just tell which ones.

I have re-uploaded the cache.log files.
https://we.tl/t-AB4XuUwuf7

One way to answer all of the above questions is to look at the following output:

    squid -k parse ... |& grep Processing:.http_access
There is no diff between both squid version, you can check it here
DiffNow - Compare Files, URLs, and Clipboard Contents Online <https://www.diffnow.com/report/jsrva>

The logs archive you shared previously has expired, so I cannot double check, but from what I remember, the shared logs did not support the above assertion, so there may be more to the story here. However, to make progress, let's assume that v5 configuration files are identical to v6 configuration files.
The configuration files / folder are the same, the server is the same, the only thing that changes is the Squid version

On 29/03/2024 17:40, Alex Rousskov wrote:
On 2024-03-25 15:13, Bolinhas André wrote:

Yes, the configuration is the same for both versions.

The logs archive you shared previously has expired, so I cannot double check, but from what I remember, the shared logs did not support the above assertion, so there may be more to the story here. However, to make progress, let's assume that v5 configuration files are identical to v6 configuration files.

1. Is there an "http_access allow all AnnotateFinalAllow" rule?

2. Is there an "http_access deny HTTP Group38 AnnotateRule28" rule?

3. Assuming the answers are "yes" and "yes", which rule comes first? If you use include files, this question applies to the imaginary preprocessed squid.conf file with all the include files inlined (recursively if needed). That kind of preprocessed configuration is what Squid effectively sees when compiling http_access rules, one by one. Which of the two rules will Squid see first?

One way to answer all of the above questions is to look at the following output:

    squid -k parse ... |& grep Processing:.http_access

Replace "..." with your regular squid startup command line options and adjust standard error redirection (|&) as needed for your shell. Run the above command for both Squid v5 and v6 binaries. You should see output like this:


2024/03/29 13:31:05| Processing: http_access allow manager
2024/03/29 13:31:05| Processing: http_access deny all


HTH,

Alex.


------------------------------------------------------------------------
*De:* Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx>
*Enviado:* segunda-feira, 25 de março de 2024 19:12
*Para:* squid-users@xxxxxxxxxxxxxxxxxxxxx
*Assunto* Re: ACL / http_access rules stop work using Squid 6+



On 2024-03-22 09:38, Andre Bolinhas wrote:

 > In previous versions of squid, from 3 to 5.9, I use this kind of deny
 > rules and they work like charm
 >
 > acl AnnotateRule28 annotate_transaction accessrule=Rule28
 > http_access deny HTTP Group38 AnnotateRule28
 >
 > This allows me to deny objects without bump / show the error page
 > (deny_info)
 >
 > But using squid 6+ this rules stop to work and everything is allowed.
 >
 > Example:
 > Squid 5.9 (OK)
 > https://ibb.co/YdKgL1Y
 >
 > Squid 6.8 (NOK)
 > https://ibb.co/tbyY2GV
 >
 > Sample of both cache.log in debug mode
 >
 > https://we.tl/t-T7Nz1rVbVu


In you v6 logs, most logged transactions are allowed because a rule
similar to the one reconstructed below is matching:

      http_access allow all AnnotateFinalAllow


There are similar cases in v5 logs as well, but most denied v5
transactions match the following rule instead (i.e. the one you shared
above):

      http_access deny HTTP Group38 AnnotateRule28


In your Squid configuration, v6 allow rule is listed much higher than v5
deny rule (#43 vs #149). I do not see any signs of Group38 or
AnnotateRule28 ACL evaluation in v6 logs, as if the rule sets are
different for two different Squid instances. Are you using the same set
of http_access rules for both Squid versions?

Alex.

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users



_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users


_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users




[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux