Am 27.10.23 um 16:22 schrieb Alex Rousskov:
On 2023-10-27 07:14, magri@xxxxxx wrote:
Am 26.10.23 um 21:11 schrieb Alex Rousskov:
On 2023-10-26 08:37, magri@xxxxxx wrote:
TL;DR: is there a way to get/log the resolved ip of a denied request?
TLDR: Bugs notwithstanding, use %<a.
%<a doesn't work :-(
Sorry, my first response was wrong: As you have correctly explained, %<a
is the destination address of the used next hop connection while dst ACL
uses an address derived from the request-target. The two addresses do
not have to match and, more importantly, it is wrong to expect a dst ACL
to update/store that "used next hop address" information when the ACL
itself does not open or reuse a connection to that address.
acl matchDst1 dst 127.0.0.1
acl markDst1 note matched=127.0.0.1
http_access deny matchDst1 markDst1
acl matchDst2 dst 127.0.0.2
acl markDst2 note matched=127.0.0.2
http_access deny matchDst2 markDst2
logformat myFormat ... matched_dst=%note{matched}
access_log ...
For long dst lists, the above approach will require scripting the
generation of the corresponding squid.conf portions or include files, of
course.
I don't think this scales to blacklists with 6-digit count sizes
I think that depends on the definition of "to scale".
and it also doesn't work for blacklisted networks :-(
Agreed. This approach can only log which dst ACL data matched, and that
data may be different from the IP.
I hoped there would be a way to get the ip as some kind of variable like
the header fields in logformat.
Yes, I know. I was just documenting an existing workaround for cases
where it is usable.
Any ideas?
I have three:
1. Enhance Squid to resolve transaction destination address once (on
first use/need). Remember/reuse resolved IP addresses. Log them via some
new %resolved_dst and %dst_resolution_detail codes.
This improvement will help address a few use cases unrelated to this
discussion, but it will _not_ tell you which of the resolved addresses
actually matched which ACL. You will be able to guess in many cases, but
there will be exceptions.
2. Add a Squid feature where an evaluated ACL can be configured to
annotate the transaction with the information about that evaluation.
To start with, we can support annotations for _matched_ dst ACLs. For
example:
# When matches, sets the following master transaction annotations:
# * badDestination_input: used destination address (IP or name)
# * badDestination_match: the matched destination IP
# * badDestination_value: the matched ACL parameter value
# * badDestination_ips: resolved destination IP(s)
# * badDestination_errors: DNS resolution and other error(s)
# * badDestination_matches: number of matches (this transaction)
# * badDestination_evals: number of evaluations (this transaction)
acl badDestination dst --on-match=annotate 127.0.0.0/24 10.0.0.1
If the same ACL matches more than once, the last(?) match wins, but the
aclfoo_matches annotation can be used to detect these cases. The
aclfoo_evals annotation can be used to detect whether this ACL was
"reached" at all.
If really needed, we can support turning individual annotations on and
off, but I doubt that complexity is worth associated performance
savings. After all, most ACLs will only match once per transaction
lifetime (for correctly written configurations). Access.log will be
configured to only log annotations of interest to the admin, of course.
The above approach can be extended to provide ACL debugging aid:
# Dumps every mismatch information to cache.log at level 1
acl goodDestination dst --on-mismatch=log 127.0.0.0/24
# Dumps every evaluation information to cache.log at level 1
acl redDestination dst --on-eval=log 127.0.0.0/24
3. Add a Squid feature where Squid (optionally) maintains an internal
database of recent ACL evaluation history and makes that information
accessible via cache manager queries like "which ACLs matched
transaction X?" (where X is logged %master_xaction ID).
The three sketched options are not mutually exclusive, of course. All
require non-trivial code changes.
Would any of the above options address your needs? Any preferences or
spec adjustments?
Very nice and detailed options!
Let me first ask some questions for clarification:
- Does squid cache all ips from dns responses with multiple ips or only
the one it uses for the request?
- If it caches more than one ip - does squid use more than one of these
ips (e.g. as fallback or round robin) inside a single transaction or for
multiple transactions?
Supposing squid uses only a single dst ip inside a single transaction,
your first option would be sufficient for our purpose!
Resolving the ip only once and storing it inside the transaction would
also avoid ambiguous cases where different ips could theoretically be
used in different acls or worse in acls and the real connection.
It could also improve the performance of acl evaluation because after
the resolution of the dst ip all following dst acls would be evaluated
fast(?).
Before the introduction of annotations we hat to use some acls twice
(for http_access and dedicated logs for this acl) but that's not
necessary anymore. We still have several dst acls that could benefit
from storing the once resolved dst ip.
Your second option sounds really nice for debugging purposes but IMHO
these annotations should be optional because even if the performance
penalty may(?) be neglectable they increase the size of the transaction
object and this could accumulate if many acls are in use.
Hence I would propose this as optional extension to the first option.
Your third option seems not feasible for us, because the delay between a
failed request and reporting of the failure often takes more than a day.
The needed history would be quite excessive (with over 80 million
requests a day).
Thank you,
Martin
Thank you,
Alex.
Is there any way to get the ip logged that was used in the dst-acl
aside
from debug logging? Maybe through some annotation mechanism?
Squid version is 6.2, as 6.4 crashes with assertion errors here, too.
thanks,
Martin
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
