On 2023-10-30 13:08, magri@xxxxxx wrote:
Am 27.10.23 um 16:22 schrieb Alex Rousskov:
1. Enhance Squid to resolve transaction destination address once (on
first use/need). Remember/reuse resolved IP addresses. Log them via some
new %resolved_dst and %dst_resolution_detail codes.
This improvement will help address a few use cases unrelated to this
discussion, but it will _not_ tell you which of the resolved addresses
actually matched which ACL. You will be able to guess in many cases, but
there will be exceptions.
2. Add a Squid feature where an evaluated ACL can be configured to
annotate the transaction with the information about that evaluation.
To start with, we can support annotations for _matched_ dst ACLs. For
example:
# When matches, sets the following master transaction annotations:
# * badDestination_input: used destination address (IP or name)
# * badDestination_match: the matched destination IP
# * badDestination_value: the matched ACL parameter value
# * badDestination_ips: resolved destination IP(s)
# * badDestination_errors: DNS resolution and other error(s)
# * badDestination_matches: number of matches (this transaction)
# * badDestination_evals: number of evaluations (this transaction)
acl badDestination dst --on-match=annotate 127.0.0.0/24 10.0.0.1
If the same ACL matches more than once, the last(?) match wins, but the
aclfoo_matches annotation can be used to detect these cases. The
aclfoo_evals annotation can be used to detect whether this ACL was
"reached" at all.
If really needed, we can support turning individual annotations on and
off, but I doubt that complexity is worth associated performance
savings. After all, most ACLs will only match once per transaction
lifetime (for correctly written configurations). Access.log will be
configured to only log annotations of interest to the admin, of course.
The above approach can be extended to provide ACL debugging aid:
# Dumps every mismatch information to cache.log at level 1
acl goodDestination dst --on-mismatch=log 127.0.0.0/24
# Dumps every evaluation information to cache.log at level 1
acl redDestination dst --on-eval=log 127.0.0.0/24
3. Add a Squid feature where Squid (optionally) maintains an internal
database of recent ACL evaluation history and makes that information
accessible via cache manager queries like "which ACLs matched
transaction X?" (where X is logged %master_xaction ID).
The three sketched options are not mutually exclusive, of course. All
require non-trivial code changes.
Let me first ask some questions for clarification:
- Does squid cache all ips from dns responses with multiple ips or only
the one it uses for the request?
All.
- If it caches more than one ip - does squid use more than one of these
ips (e.g. as fallback or round robin) inside a single transaction or for
multiple transactions?
A single transaction will go through several single-name IPs after
certain transaction failures, but not all failures are treated the same.
Multiple transactions may use multiple single-name IPs due to
round-robin rotation of cached IPs and other factors.
Supposing squid uses only a single dst ip inside a single transaction,
your first option would be sufficient for our purpose!
In my first option, logged %resolved_dst (and the corresponding master
transaction metadata) may contain multiple IPs (i.e. all resolved ones).
AFAICT, if any one of those IPs match a "dst" ACL address, then that
"dst" ACL should match, even if the transaction does not actually use
the matched address: The "dst" ACL matches based on request-target info,
not the _use_ of that info.
One of the difficulties here is to decide whether a transaction should
wait for both IPv4 and IPv6 addresses to be resolved:
If a transaction proceeds with one family of addresses, then there will
be no delays associated with waiting for the second one. However, in
that case, Squid may declare a "dst" ACL mismatch even though there
would have been a match if we waited for the second DNS answer! It feels
like for correctness sake, we must wait if a "dst" ACL needs checking or
a similar decision has to be made...
Resolving the ip only once and storing it inside the transaction would
also avoid ambiguous cases where different ips could theoretically be
used in different acls or worse in acls and the real connection.
Yes, that is one of the positive side effects of that option.
It could also improve the performance of acl evaluation because after
the resolution of the dst ip all following dst acls would be evaluated
fast(?).
A speedup is possible in some corner cases, but, in most cases, there
will be no difference in this area because "all following dst ACLs"
should "immediately" use _cached_ IP(s) in the current code AFAICT.
Your second option sounds really nice for debugging purposes but IMHO
these annotations should be optional because even if the performance
penalty may(?) be neglectable they increase the size of the transaction
object and this could accumulate if many acls are in use.
Hence I would propose this as optional extension to the first option.
Yes, the second feature is optional (and orthogonal): If one does not
add "--on-match=annotate" to a "dst" ACL declaration, then they do not
get those annotations (and their overheads).
Your third option seems not feasible for us, because the delay between a
failed request and reporting of the failure often takes more than a day.
The needed history would be quite excessive (with over 80 million
requests a day).
Agreed.
Alex.
Is there any way to get the ip logged that was used in the dst-acl
aside
from debug logging? Maybe through some annotation mechanism?
Squid version is 6.2, as 6.4 crashes with assertion errors here, too.
thanks,
Martin
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
