Hi Alex,
Am 26.10.23 um 21:11 schrieb Alex Rousskov:
On 2023-10-26 08:37, magri@xxxxxx wrote:
TL;DR: is there a way to get/log the resolved ip of a denied request?
TLDR: Bugs notwithstanding, use %<a.
%<a doesn't work :-(
Tested with attached minimal-squid.conf:
- config blacklists the ip of www.example.org
- logformat uses %<a
Test-Call:
http_proxy=http://127.0.0.1:3128 https_proxy=http://127.0.0.1:3128 wget
http://www.example.org
--2023-10-27 09:52:27-- http://www.example.org/
Connecting to 127.0.0.1:3128... connected.
Proxy request sent, awaiting response... 403 Forbidden
2023-10-27 09:52:31 ERROR 403: Forbidden.
Log contains:
1698393151.066 3903 127.0.0.1 TCP_DENIED/- 3889 GET
http://www.example.org/ - HIER_NONE/- text/html HTTP/1.1 37030 182 "-"
"Wget/1.21.3" ERR_ACCESS_DENIED "-" 53
We have a rather large ip based malware blacklist (dst acl) and
sometimes a destination is blocked inadvertantly because of a false
positive entry in this list.
This happens most often with CDNs where the ips of a destination change
often and even move between different sites.
Because of this rapid change it's difficult to determine the blocked ip
in hindsight when analyzing access problems and makes it impossible to
correct the blacklist.
For normal requests the resolved and accessed ip is be logged with %<a,
but that doesn't happen when the request is denied.
If a request was denied by a dst ACL based on its successfully resolved
destination IP address but %<a was logged as "-", then it is a Squid bug
that should be fixed IMO. Meanwhile, you can annotate every dst match
and log that annotation. Here is an untested sketch:
acl matchDst1 dst 127.0.0.1
acl markDst1 note matched=127.0.0.1
acl all-of dst1 matchDst1 markDst1
http_access deny dst1
acl matchDst2 dst 127.0.0.2
acl markDst2 note matched=127.0.0.2
acl all-of dst2 matchDst2 markDst2
http_access deny dst2
logformat myFormat ... matched_dst=%note{matched}
access_log ...
The same thing with fewer lines (but with fewer ways to group dst1 and
dst2 with other ACLs):
acl matchDst1 dst 127.0.0.1
acl markDst1 note matched=127.0.0.1
http_access deny matchDst1 markDst1
acl matchDst2 dst 127.0.0.2
acl markDst2 note matched=127.0.0.2
http_access deny matchDst2 markDst2
logformat myFormat ... matched_dst=%note{matched}
access_log ...
For long dst lists, the above approach will require scripting the
generation of the corresponding squid.conf portions or include files, of
course.
I don't think this scales to blacklists with 6-digit count sizes and it
also doesn't work for blacklisted networks :-(
I hoped there would be a way to get the ip as some kind of variable like
the header fields in logformat.
If a request was denied by a dst ACL because its destination IP address
could not be resolved, then %<a should be logged as "-". I cannot think
of a way to distinguish this case from other cases where %<a is "-". It
feels like address resolution failures should be available via
%err_detail, but I doubt Squid code populates that information in these
cases. Another problem to fix!
I'm not familiar with the code but from staring at it I get that '%<a'
is fetched from hier.tcpServer->remote and this isn't necessarily the ip
of the resolved URI FQDN (e.g. when using a parent proxy).
There doesn't seem to be any code that stores or reuses a once resolved
ip in acl/DestinationIP.cc (at least if a request exists).
As far as I understand it for every dst ACL the ip is fetched
(asyncronously) from the ipcache and compared to the ACL but never
stored for later use.
Any ideas?
Martin
HTH,
Alex.
Is there any way to get the ip logged that was used in the dst-acl aside
from debug logging? Maybe through some annotation mechanism?
Squid version is 6.2, as 6.4 crashes with assertion errors here, too.
thanks,
Martin
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
http_port 3128
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl www_example_org dst 93.184.216.34
http_access deny www_example_org
http_access allow all
never_direct deny all
always_direct allow all
logformat debug_squid %ts.%03tu %6tr %>a %Ss/%<Hs %<st %rm %ru %un %Sh/%<a %mt HTTP/%rv %>p %>st "%{Referer}>h" "%{User-Agent}>h" %err_code "%err_detail" %master_xaction
access_log daemon:/var/log/squid/test.log logformat=debug_squid
cache_log /var/log/squid/cache.log
debug_options ALL,2
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.squid-cache.org/listinfo/squid-users
