Hi Eliezer,
We have:
/etc/apt/apt.conf:
Acquire::http::proxy "http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/";
Acquire::https::proxy "http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/";
/etc/apt/sources.list (comment lines removed for brevity)
deb https://mirror.aarnet.edu.au/ubuntu/ focal main restricted
deb https://mirror.aarnet.edu.au/ubuntu/ focal-updates main restricted
deb https://mirror.aarnet.edu.au/ubuntu/ focal-updates universe
deb https://mirror.aarnet.edu.au/ubuntu/ focal multiverse
deb https://mirror.aarnet.edu.au/ubuntu/ focal-updates multiverse
deb https://mirror.aarnet.edu.au/ubuntu/ focal-backports main restricted universe multiverse
deb https://mirror.aarnet.edu.au/ubuntu focal-security main restricted
deb https://mirror.aarnet.edu.au/ubuntu focal-security universe
deb https://mirror.aarnet.edu.au/ubuntu focal-security multiverse
squid.conf
# Debugging for your ACLsdebug_options ALL,1# temp option for full debug logs#debug_options 28,2# Example rule allowing access from your local networks.# Adapt to list your (internal) IP networks from where browsing# should be allowedacl vpc_cidr src 10.0.0.0/16 # VPC CIDRacl vpc_cidr src 127.0.0.1# technician VPN source cidracl technician_vpn src 10.0.104.0/22acl SSL_ports port 443acl Safe_ports port 80 # http#acl Safe_ports port 21 # ftpacl Safe_ports port 443 # https#acl Safe_ports port 70 # gopher#acl Safe_ports port 210 # waisacl Safe_ports port 1025-65535 # unregistered ports#acl Safe_ports port 280 # http-mgmt#acl Safe_ports port 488 # gss-http#acl Safe_ports port 591 # filemaker#acl Safe_ports port 777 # multiling httpacl CONNECT method CONNECT## Recommended minimum Access Permission configuration:## Deny requests to certain unsafe portshttp_access deny !Safe_ports# Deny CONNECT to other than secure SSL portshttp_access deny CONNECT !SSL_ports# Only allow cachemgr access from localhosthttp_access allow localhost managerhttp_access deny manager# We strongly recommend the following be uncommented to protect innocent# web applications running on the proxy server who think the only# one who can access services on "localhost" is a local user#http_access deny to_localhost## INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS## Redirect HTTP to HTTPSacl port_80 port 80acl gstatic dstdomain www.gstatic.comhttp_access deny port_80 gstaticdeny_info 301:https://%H%R gstaticacl avpc dstdomain crop-assessment.acusensus-vpchttp_access deny port_80 avpcdeny_info 302:<company url> avpc# Deny HTTPhttp_access deny port_80# Whitelist of allowed sitesacl allowed_http_sites dstdomain "/etc/squid/squid.allowed.sites.txt"http_access allow allowed_http_sites vpc_cidr# And finally deny all other access to this proxyhttp_access deny all# Squid normally listens to port 3128http_port 3128 ssl-bump cert=/etc/squid/cert.pemacl allowed_https_sites ssl::server_name "/etc/squid/squid.allowed.sites.txt"acl step1 at_step SslBump1acl step2 at_step SslBump2acl step3 at_step SslBump3ssl_bump peek step1 allssl_bump peek step2 allowed_https_sitesssl_bump splice step3 allowed_https_sitesssl_bump terminate step2 all# Uncomment and adjust the following to add a disk cache directory.#cache_dir ufs /var/spool/squid 100 16 256# Leave coredumps in the first cache dircoredump_dir /var/spool/squid## Add any of your own refresh_pattern entries above these.#refresh_pattern ^ftp: 1440 20% 10080refresh_pattern ^gopher: 1440 0% 1440refresh_pattern -i (/cgi-bin/|\?) 0 0% 0refresh_pattern . 0 20% 4320
Squid 3.5 is running on an EC2 instance running Amazon Linux 2. I'll answer the questions you asked Ben for extra info.
ip address:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
link/ether 02:1b:15:b8:9a:06 brd ff:ff:ff:ff:ff:ff
inet 10.0.12.111/24 brd 10.0.12.255 scope global dynamic eth0
valid_lft 2393sec preferred_lft 2393sec
inet6 fe80::1b:15ff:feb8:9a06/64 scope link
valid_lft forever preferred_lft forever
ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
ip route show
default via 10.0.12.1 dev eth0
10.0.12.0/24 dev eth0 proto kernel scope link src 10.0.12.111
169.254.169.254 dev eth0
ip route show table 100
iptables-save
squid -v
Squid Cache: Version 3.5.20
Service Name: squid
configure options: '--build=x86_64-koji-linux-gnu' '--host=x86_64-koji-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-strict-error-checking' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-eui' '--enable-follow-x-forwarded-for' '--enable-auth' '--enable-auth-basic=DB,LDAP,MSNT-multi-domain,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB,SMB_LM,getpwnam' '--enable-auth-ntlm=smb_lm,fake' '--enable-auth-digest=file,LDAP,eDirectory' '--enable-auth-negotiate=kerberos' '--enable-external-acl-helpers=file_userip,LDAP_group,time_quota,session,unix_group,wbinfo_group,kerberos_ldap_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,rock,ufs' '--enable-wccpv2' '--enable-esi' '--enable-ecap' '--with-aio' '--with-default-user=squid' '--with-dl' '--with-openssl' '--with-pthreads' '--disable-arch-native' 'build_alias=x86_64-koji-linux-gnu' 'host_alias=x86_64-koji-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fpie' 'LDFLAGS=-Wl,-z,relro -pie -Wl,-z,relro -Wl,-z,now' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fpie' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
uname -a
Linux ip-10-0-12-111.ap-southeast-2.compute.internal 4.14.231-173.361.amzn2.x86_64 #1 SMP Mon Apr 26 20:57:08 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Regards,
On Wed, 7 Jul 2021 at 20:53, Eliezer Croitoru <ngtech1ltd@xxxxxxxxx> wrote:
Hey David,
Just wondering if you have seen the apt related docs at:
https://help.ubuntu.com/community/AptGet/Howto/#Setting_up_apt-get_to_use_a_http-proxy
Eliezer
From: squid-users <squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx> On Behalf Of David Mills
Sent: Wednesday, July 7, 2021 2:26 AM
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Ubuntu 20.04 "apt update" issues behind a VPN and Squid proxy
Hi,
We've got a collection of Ubuntu 18.04 boxes out in the field. They connect to an AWS OpenVPN VPN and use a Squid 3.5 AWS hosted Proxy. They work fine.
We have tried upgrading one to 20.04. Same setup. From the command line curl or wget can happily download an Ubuntu package from the Ubuntu Mirror site we use. But "apt update" gets lots of "IGN:" timeouts and errors.
The package we test curl with is https://mirror.aarnet.edu.au/ubuntu/pool/main/c/curl/curl_7.68.0-1ubuntu2.5_amd64.deb
The Squid log shows a line the doesn't occur for the successful 18.04 "apt updates":
1625190959.233 81 10.0.11.191 TAG_NONE/200 0 CONNECT http://mirror.aarnet.edu.au:443 - HIER_DIRECT/2001:388:30bc:cafe::beef -
The full output of an attempt to update is:
Ign:1 https://mirror.aarnet.edu.au/ubuntu focal InRelease
Ign:2 https://mirror.aarnet.edu.au/ubuntu focal-updates InRelease
Ign:3 https://mirror.aarnet.edu.au/ubuntu focal-backports InRelease
Ign:4 https://mirror.aarnet.edu.au/ubuntu focal-security InRelease
Err:5 https://mirror.aarnet.edu.au/ubuntu focal Release
Could not wait for server fd - select (11: Resource temporarily unavailable) [IP: 10.0.11.82 3128]
Err:6 https://mirror.aarnet.edu.au/ubuntu focal-updates Release
Could not wait for server fd - select (11: Resource temporarily unavailable) [IP: 10.0.11.82 3128]
Err:7 https://mirror.aarnet.edu.au/ubuntu focal-backports Release
Could not wait for server fd - select (11: Resource temporarily unavailable) [IP: 10.0.11.82 3128]
Err:8 https://mirror.aarnet.edu.au/ubuntu focal-security Release
Could not wait for server fd - select (11: Resource temporarily unavailable) [IP: 10.0.1.26 3128]
Reading package lists... Done
N: Ignoring file 'microsoft-prod.list-keep' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
E: The repository 'https://mirror.aarnet.edu.au/ubuntu focal Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://mirror.aarnet.edu.au/ubuntu focal-updates Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://mirror.aarnet.edu.au/ubuntu focal-backports Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: The repository 'https://mirror.aarnet.edu.au/ubuntu focal-security Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
While running, the line
0% [Connecting to HTTP proxy (http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128)]
appears often and hang for a while.
I've tried upping the squid logging and allowing all, but they didn't offer any additional information about the issue.
Any advice would be greatly appreciated.
Regards,
David Mills
Senior DevOps Engineer
E: mailto:david.mills@xxxxxxxxxxxxx
M: +61 411 513 404
W:http://acusensus.com/
DISCLAIMER: Acusensus puts the privacy and security of its clients, its data and information at the core of everything we do. The information contained in this email (including attachments) is intended only for the use of the person(s) to whom it is addressed to, as it may be confidential and contain legally privileged information. If you have received this email in error, please delete all copies and notify the sender immediately. Any views or opinions presented are solely those of the author and do not necessarily represent the views of Acusensus Pty Ltd. Please consider the environment before printing this email.
DISCLAIMER: Acusensus puts the privacy and security of its clients, its data and information at the core of everything we do. The information contained in this email (including attachments) is intended only for the use of the person(s) to whom it is addressed to, as it may be confidential and contain legally privileged information. If you have received this email in error, please delete all copies and notify the sender immediately. Any views or opinions presented are solely those of the author and do not necessarily represent the views of Acusensus Pty Ltd. Please consider the environment before printing this email.
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
