On 8/07/21 11:44 am, David Mills wrote:
Hi Eliezer,
We have:
/etc/apt/apt.conf:
Acquire::http::proxy
"http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/
<http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/>";
Acquire::https::proxy
"http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/
<http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/>";
/etc/apt/sources.list (comment lines removed for brevity)
deb https://mirror.aarnet.edu.au/ubuntu/
<https://mirror.aarnet.edu.au/ubuntu/> focal main restricted
deb https://mirror.aarnet.edu.au/ubuntu/
<https://mirror.aarnet.edu.au/ubuntu/> focal-updates main restricted
deb https://mirror.aarnet.edu.au/ubuntu/
<https://mirror.aarnet.edu.au/ubuntu/> focal-updates universe
deb https://mirror.aarnet.edu.au/ubuntu/
<https://mirror.aarnet.edu.au/ubuntu/> focal multiverse
deb https://mirror.aarnet.edu.au/ubuntu/
<https://mirror.aarnet.edu.au/ubuntu/> focal-updates multiverse
deb https://mirror.aarnet.edu.au/ubuntu/
<https://mirror.aarnet.edu.au/ubuntu/> focal-backports main
restricted universe multiverse
deb https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal-security main restricted
deb https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal-security universe
deb https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal-security multiverse
squid.conf
...
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# Redirect HTTP to HTTPS
acl port_80 port 80
acl gstatic dstdomain www.gstatic.com <http://www.gstatic.com>
http_access deny port_80 gstatic
deny_info 301:https://%H%R gstatic
acl avpc dstdomain crop-assessment.acusensus-vpc
http_access deny port_80 avpc
deny_info 302:<company url> avpc
# Deny HTTP
http_access deny port_80
# Whitelist of allowed sites
acl allowed_http_sites dstdomain "/etc/squid/squid.allowed.sites.txt"
http_access allow allowed_http_sites vpc_cidr
Is the "mirror.aarnet.edu.au" or a wildcard matching it listed in file
squid.allowed.sites.txt ?
(I assume so, but checking in case it is that simple).
# And finally deny all other access to this proxy
http_access deny all
# Squid normally listens to port 3128
http_port 3128 ssl-bump cert=/etc/squid/cert.pem
acl allowed_https_sites ssl::server_name
"/etc/squid/squid.allowed.sites.txt"
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump peek step2 allowed_https_sites
ssl_bump splice step3 allowed_https_sites
ssl_bump terminate step2 all
# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
Squid 3.5 is running on an EC2 instance running Amazon Linux 2. I'll
answer the questions you asked Ben for extra info.
ip address:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state
UP group default qlen 1000
link/ether 02:1b:15:b8:9a:06 brd ff:ff:ff:ff:ff:ff
inet 10.0.12.111/24 <http://10.0.12.111/24> brd 10.0.12.255
scope global dynamic eth0
valid_lft 2393sec preferred_lft 2393sec
inet6 fe80::1b:15ff:feb8:9a06/64 scope link
valid_lft forever preferred_lft forever
ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
ip route show
default via 10.0.12.1 dev eth0
10.0.12.0/24 <http://10.0.12.0/24> dev eth0 proto kernel scope link
src 10.0.12.111
169.254.169.254 dev eth0
The traffic from Squid to the AArnet server is apparently using IPv6. Is
that routing setup properly too?
...
From: squid-users On Behalf Of David Mills
Sent: Wednesday, July 7, 2021 2:26 AM
...
We have tried upgrading one to 20.04. Same setup. From the command
line curl or wget can happily download an Ubuntu package from the
Ubuntu Mirror site we use. But "apt update" gets lots of "IGN:"
timeouts and errors.
The package we test curl with is
https://mirror.aarnet.edu.au/ubuntu/pool/main/c/curl/curl_7.68.0-1ubuntu2.5_amd64.deb
<https://mirror.aarnet.edu.au/ubuntu/pool/main/c/curl/curl_7.68.0-1ubuntu2.5_amd64.deb>
The Squid log shows a line the doesn't occur for the successful
18.04 "apt updates":
1625190959.233 81 10.0.11.191 TAG_NONE/200 0 CONNECT
http://mirror.aarnet.edu.au:443 <http://mirror.aarnet.edu.au:443> -
HIER_DIRECT/2001:388:30bc:cafe::beef -
With Ubuntu 20.04 you should have received Squid-4 (v4.10 or later).
Which logs a few things differently from Squid-3.5, particularly for
SSL-Bump activity and client connections that lack HTTP messages.
The above log line shows SSL-Bump activity. At least step2 was reached,
possibly also step3. Looking at this a little closer to see if it
completes fine or has unseen issues would be my next point of approach.
To debug what is happening with SSL-Bump use "debug_options ALL1, 11,2
83,5" and check the resulting cache.log.
The full output of an attempt to update is:
Ign:1 https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal InRelease
Ign:2 https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal-updates InRelease
Ign:3 https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal-backports InRelease
Ign:4 https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal-security InRelease
These "Ign" are fine. They just mean that apt has determined those files
it has cached are up-to-date and do not need to be re-fetched right now.
The below "Err" are the problem:
Err:5 https://mirror.aarnet.edu.au/ubuntu
<https://mirror.aarnet.edu.au/ubuntu> focal Release
Could not wait for server fd - select (11: Resource temporarily
unavailable) [IP: 10.0.11.82 3128]...
While running, the line
0% [Connecting to HTTP proxy
(http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128
<http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128>)]
appears often and hang for a while.
I've tried upping the squid logging and allowing all, but they
didn't offer any additional information about the issue.
Your squid.conf looks fine, assuming the same http_access rules were
used in your working version.
I suspect the issue is related to one or more of:
* IPv6 routing
* ICMP config issues (maybe outside your network)
* SSL-Bump issues processing the client or server handshake traffic
typically seen with OpenSSL library version or config mismatches
between Squid, client and server.
* network timeouts affecting Squid
HTH
Amos
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
