On Mon, Feb 27, 2017 at 2:53 AM, Eliezer Croitoru <eliezer@xxxxxxxxxxxx> wrote: > Let me know if you need some help.. > > Eliezer > > ---- > Eliezer Croitoru > Linux System Administrator > Mobile: +972-5-28704261 > Email: eliezer@xxxxxxxxxxxx > > > -----Original Message----- > From: squid-users [mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Eliezer Croitoru > Sent: Sunday, February 26, 2017 8:51 PM > To: 'Test User' <tuser6485@xxxxxxxxx> > Cc: squid-users@xxxxxxxxxxxxxxxxxxxxx > Subject: Re: SSL-Bump: NAT/TPROXY lookup failed to locate original IPs > > Hey Michael, > > The details you attached explained pretty well the cause for the issues you have described. > What you will need to do in order to make this setup to work can be done in more then one way. > For a sysadmin the simplest way is to create a VPN or some kind of a tunnel between the AWS instance to the local router. > I am almost sure that you can use haproxy to do a local tproxy or interception that will forward the traffic to the remote squid with the PROXY protocol keeping original source and original destination visible to the remote squid. > > The choice will depend on both: > - your skills and will to dig some time about couple subjects > - The availability of static IP addresses(both local and AWS). > - The OS on both sides > > I believe that the next haproxy settings can be used as a compromise to a tunnel: > http://ngtech.co.il/paste/1605/ > And some tproxy route and iptables rules .. > With a squid.conf which will be similar to: > acl frontend src 100.0.0.1 > proxy_protocol_access allow frontend > http_port 3127 > http_port 3128 require-proxy-header ... ssl-bump settings > ##END of example > > However I do still believe that the more secure way would be to use some kind of vpn tunnel like OpenVPN between the local router to the remote AWS instance. > > All The Bests, > Eliezer > > ---- > Eliezer Croitoru > Linux System Administrator > Mobile: +972-5-28704261 > Email: eliezer@xxxxxxxxxxxx > > > -----Original Message----- > From: Test User [mailto:tuser6485@xxxxxxxxx] > Sent: Sunday, February 26, 2017 8:38 AM > To: Eliezer Croitoru <eliezer@xxxxxxxxxxxx> > Cc: squid-users@xxxxxxxxxxxxxxxxxxxxx > Subject: Re: SSL-Bump: NAT/TPROXY lookup failed to locate original IPs > > On Sun, Feb 26, 2017 at 10:40 AM, Eliezer Croitoru <eliezer@xxxxxxxxxxxx> wrote: >> Hey Michael, >> >> You will need to clear out couple things for us. >> First we will need one of the next ouputs or both: >> iptables-save >> iptables -L -nv >> >> And then clear out where is this proxy sittings and the network structure. >> It's not clear if the squid box is the router or a machine somewhere on AWS. >> If you wish to pass traffic from a local router to a one on AWS you will need to create a tunnel like using OpenVPN or a similar solution and to use some routing rules to pass the traffic from the local LAN to AWS without removing the original destination address. >> >> When more details on the setup will be available it will be much simpler to understand what is the root for some of the issues you are having. >> >> All The Bests, >> Eliezer >> >> ---- >> Eliezer Croitoru >> Linux System Administrator >> Mobile: +972-5-28704261 >> Email: eliezer@xxxxxxxxxxxx >> >> >> -----Original Message----- >> From: squid-users [mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Test User >> Sent: Friday, February 24, 2017 8:52 AM >> To: squid-users@xxxxxxxxxxxxxxxxxxxxx >> Subject: SSL-Bump: NAT/TPROXY lookup failed to locate original IPs >> >> Hi, >> Sorry I am asking this question again. I am trying to setup HTTPS >> proxy using ssl-bump. I have followed >> steps mentioned in: >> http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit >> >> Following are Squid setup details: >> >> Squid Cache: Version 3.5.12 >> Service Name: squid >> Ubuntu linux >> >> configure options: '--build=x86_64-linux-gnu' '--prefix=/usr' >> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' >> '--infodir=${prefix}/share/info' '--sysconfdir=/etc' >> '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' >> '--srcdir=.' '--disable-maintainer-mode' >> '--disable-dependency-tracking' '--disable-silent-rules' >> 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat >> -Werror=format-security -Wl,-Bsymbolic-functions -fPIE -pie >> -Wl,-z,relro -Wl,-z,now' '--datadir=/usr/share/squid' >> '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' >> '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' >> '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' >> '--enable-removal-policies=lru,heap' '--enable-delay-pools' >> '--enable-cache-digests' '--enable-icap-client' >> '--enable-follow-x-forwarded-for' >> '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' >> '--enable-auth-digest=file,LDAP' >> '--enable-auth-negotiate=kerberos,wrapper' >> '--enable-auth-ntlm=fake,smb_lm' >> '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group' >> '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' >> '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--with-openssl' >> '--enable-ssl-crtd' '--disable-translation' >> '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' >> '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536' >> '--with-large-files' '--with-default-user=proxy' >> '--enable-build-info=Ubuntu linux' '--enable-linux-netfilter' >> 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE >> -fstack-protector-strong -Wformat -Werror=format-security -Wall' >> 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' >> 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE >> -fstack-protector-strong -Wformat -Werror=format-security' >> >> >> Following is my squid.conf file: >> >> acl SSL_ports port 443 >> acl Safe_ports port 80 # http >> acl Safe_ports port 21 # ftp >> acl Safe_ports port 443 # https >> acl Safe_ports port 70 # gopher >> acl Safe_ports port 210 # wais >> acl Safe_ports port 1025-65535 # unregistered ports >> acl Safe_ports port 280 # http-mgmt >> acl Safe_ports port 488 # gss-http >> acl Safe_ports port 591 # filemaker >> acl Safe_ports port 777 # multiling http >> acl CONNECT method CONNECT >> acl step1 at_step SslBump1 >> http_access deny !Safe_ports >> http_access deny CONNECT !SSL_ports >> http_access allow localhost manager >> http_access deny manager >> http_access allow localhost >> http_access allow all >> http_port 3128 ssl-bump \ >> cert=/etc/squid/ssl_cert/squidCA.pem \ >> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB >> https_port 3129 intercept ssl-bump generate-host-certificates=on \ >> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/squidCA.pem \ >> dhparams=/etc/squid/ssl_cert/dhparam.pem >> sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_DH_USE >> sslproxy_cipher >> EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS >> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/spool/squid3_ssldb -M 4MB >> debug_options ALL,1 3,5 4,5 11,5 17,5 23,5 46,5 78,5 rotate=1 >> coredump_dir /var/spool/squid >> refresh_pattern ^ftp: 1440 20% 10080 >> refresh_pattern ^gopher: 1440 0% 1440 >> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 >> refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880 >> refresh_pattern . 0 20% 4320 >> >> >> I get no errors while starting Squid. Following are the logs when Squid starts: >> >> 2017/02/23 09:59:53 kid1| Set Current Directory to /var/spool/squid >> 2017/02/23 09:59:53 kid1| Starting Squid Cache version 3.5.12 for >> x86_64-pc-linux-gnu... >> 2017/02/23 09:59:53 kid1| Service Name: squid >> 2017/02/23 09:59:53 kid1| Process ID 26236 >> 2017/02/23 09:59:53 kid1| Process Roles: worker >> 2017/02/23 09:59:53 kid1| With 65535 file descriptors available >> 2017/02/23 09:59:53 kid1| Initializing IP Cache... >> 2017/02/23 09:59:53.756 kid1| 78,2| dns_internal.cc(1525) dnsInit: >> idnsInit: attempt open DNS socket to: [::] >> 2017/02/23 09:59:53.756 kid1| 78,2| dns_internal.cc(1534) dnsInit: >> idnsInit: attempt open DNS socket to: 0.0.0.0 >> 2017/02/23 09:59:53.756 kid1| DNS Socket created at [::], FD 6 >> 2017/02/23 09:59:53.756 kid1| DNS Socket created at 0.0.0.0, FD 7 >> 2017/02/23 09:59:53.756 kid1| Adding nameserver 172.31.0.2 from /etc/resolv.conf >> 2017/02/23 09:59:53.756 kid1| 78,3| dns_internal.cc(321) >> idnsAddNameserver: idnsAddNameserver: Added nameserver #0 >> (172.31.0.2:53) >> 2017/02/23 09:59:53.756 kid1| Adding domain >> ap-south-1.compute.internal from /etc/resolv.conf >> 2017/02/23 09:59:53.756 kid1| 78,3| dns_internal.cc(350) >> idnsAddPathComponent: idnsAddPathComponent: Added domain #0: >> ap-south-1.compute.internal >> 2017/02/23 09:59:53.756 kid1| helperOpenServers: Starting 5/32 >> 'ssl_crtd' processes >> 2017/02/23 09:59:53.775 kid1| 46,2| Format.cc(64) parse: got >> definition '%>a/%>A %un %>rm myip=%la myport=%lp' >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for >> possible 1C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for >> possible 1C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,2| Format.cc(64) parse: got >> definition '%>a/%>A %un %>rm myip=%la myport=%lp' >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for >> possible 1C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for >> possible 1C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for >> possible Misc token >> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for >> possible 2C token >> 2017/02/23 09:59:53.775 kid1| Logfile: opening log >> daemon:/var/log/squid/access.log >> 2017/02/23 09:59:53.775 kid1| Logfile Daemon: opening log >> /var/log/squid/access.log >> 2017/02/23 09:59:53.779 kid1| 23,5| url.cc(43) urlInitialize: >> urlInitialize: Initializing... >> 2017/02/23 09:59:53.779 kid1| Local cache digest enabled; >> rebuild/rewrite every 3600/3600 sec >> 2017/02/23 09:59:53.779 kid1| Store logging disabled >> 2017/02/23 09:59:53.779 kid1| Swap maxSize 0 + 262144 KB, estimated >> 20164 objects >> 2017/02/23 09:59:53.779 kid1| Target number of buckets: 1008 >> 2017/02/23 09:59:53.779 kid1| Using 8192 Store buckets >> 2017/02/23 09:59:53.779 kid1| Max Mem size: 262144 KB >> 2017/02/23 09:59:53.779 kid1| Max Swap size: 0 KB >> 2017/02/23 09:59:53.779 kid1| Using Least Load store dir selection >> 2017/02/23 09:59:53.779 kid1| Set Current Directory to /var/spool/squid >> 2017/02/23 09:59:53.785 kid1| 23,3| url.cc(357) urlParse: urlParse: >> Split URL 'http://ip-172-31-25-235:3128/squid-internal-static/icons/silk/image.png' >> into proto='http', host='ip-172-31-25-235', port='3128', >> path='/squid-internal-static/icons/silk/image.png' >> 2017/02/23 09:59:53.785 kid1| 23,3| url.cc(357) urlParse: urlParse: >> Split URL 'http://ip-172-31-25-235:3128/squid-internal-static/icons/silk/page_white_text.png' >> into proto='http', host='ip-172-31-25-235', port='3128', >> path='/squid-internal-static/icons/silk/page_white_text.png' >> >> ****several urlParse logs like above. Removing them to shorten the >> email. Further logs below...**** >> >> 2017/02/23 09:59:53.815 kid1| Finished loading MIME types and icons. >> 2017/02/23 09:59:53.815 kid1| HTCP Disabled. >> 2017/02/23 09:59:53.815 kid1| Pinger socket opened on FD 25 >> 2017/02/23 09:59:53.815 kid1| Squid plugin modules loaded: 0 >> 2017/02/23 09:59:53.815 kid1| Adaptation support is off. >> 2017/02/23 09:59:53.815 kid1| Accepting SSL bumped HTTP Socket >> connections at local=[::]:3128 remote=[::] FD 22 flags=9 >> 2017/02/23 09:59:53.815 kid1| Accepting NAT intercepted SSL bumped >> HTTPS Socket connections at local=[::]:3129 remote=[::] FD 23 flags=41 >> 2017/02/23 09:59:53| pinger: Initialising ICMP pinger ... >> 2017/02/23 09:59:53| pinger: ICMP socket opened. >> 2017/02/23 09:59:53| pinger: ICMPv6 socket opened >> 2017/02/23 09:59:54 kid1| storeLateRelease: released 0 objects >> >> >> >> I tested this setup by providing proxy details to Firefox. Firefox was >> able to show HTTP websites but when I tried to open an HTTPS website I >> got following error: >> >> 2017/02/23 11:00:50 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on >> local=172.31.25.235:3129 remote=182.72.78.122:50655 FD 7 flags=33: >> (92) Protocol not available >> 2017/02/23 11:00:50 kid1| ERROR: NAT/TPROXY lookup failed to locate >> original IPs on local=172.31.25.235:3129 remote=182.72.78.122:50655 FD >> 7 flags=33 >> 2017/02/23 11:00:50 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on >> local=172.31.25.235:3129 remote=182.72.78.122:50656 FD 7 flags=33: >> (92) Protocol not available >> 2017/02/23 11:00:50 kid1| ERROR: NAT/TPROXY lookup failed to locate >> original IPs on local=172.31.25.235:3129 remote=182.72.78.122:50656 FD >> 7 flags=33 >> 2017/02/23 11:00:50 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on >> local=172.31.25.235:3129 remote=182.72.78.122:50657 FD 7 flags=33: >> (92) Protocol not available >> 2017/02/23 11:00:50 kid1| ERROR: NAT/TPROXY lookup failed to locate >> original IPs on local=172.31.25.235:3129 remote=182.72.78.122:50657 FD >> 7 flags=33 >> >> I googled this error and found this mail thread which had similar problems: >> http://squid-web-proxy-cache.1019090.n4.nabble.com/NAT-TPROXY-lookup-failed-to-locate-original-IPs-td4675464.html >> >> I found this link from the above thread. I modified the steps for >> HTTPS from the below link: >> http://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxDnat >> >> Now my sysctl.conf is: >> >> net.ipv4.conf.all.rp_filter=0 >> net.ipv4.ip_forward = 1 >> net.ipv4.conf.default.rp_filter = 0 >> net.ipv4.conf.default.accept_source_route = 0 >> >> My iptables -t nat -L result: >> >> Chain PREROUTING (policy ACCEPT) >> target prot opt source destination >> ACCEPT tcp -- ec2-35-154-101-8.ap-south-1.compute.amazonaws.com >> anywhere tcp dpt:https >> DNAT tcp -- anywhere anywhere tcp >> dpt:https to:35.154.101.8:3129 >> >> Chain INPUT (policy ACCEPT) >> target prot opt source destination >> >> Chain OUTPUT (policy ACCEPT) >> target prot opt source destination >> >> Chain POSTROUTING (policy ACCEPT) >> target prot opt source destination >> MASQUERADE all -- anywhere anywhere >> >> >> Once this was done, I tried to hit HTTPS website from Firefox and now >> I get connection timeout error. Nothing shows in syslog, access.log or >> cache.log. Could you please help me resolve this. >> >> Thanks, >> Michael >> _______________________________________________ >> squid-users mailing list >> squid-users@xxxxxxxxxxxxxxxxxxxxx >> http://lists.squid-cache.org/listinfo/squid-users >> > > > Thanks for replying Eliezer. Following are the outputs you asked: > > 1. iptables-save: > > # Generated by iptables-save v1.6.0 on Sun Feb 26 06:28:46 2017 > *filter > :INPUT ACCEPT [171:12090] > :FORWARD ACCEPT [0:0] > :OUTPUT ACCEPT [106:15187] > COMMIT > # Completed on Sun Feb 26 06:28:46 2017 > # Generated by iptables-save v1.6.0 on Sun Feb 26 06:28:46 2017 > *mangle > :PREROUTING ACCEPT [89003:74850371] > :INPUT ACCEPT [88973:74849159] > :FORWARD ACCEPT [30:1212] > :OUTPUT ACCEPT [76710:51478183] > :POSTROUTING ACCEPT [76740:51479395] > -A PREROUTING -p tcp -m tcp --dport 3129 -j DROP > COMMIT > # Completed on Sun Feb 26 06:28:46 2017 > # Generated by iptables-save v1.6.0 on Sun Feb 26 06:28:46 2017 > *nat > :PREROUTING ACCEPT [7766:436942] > :INPUT ACCEPT [7766:436942] > :OUTPUT ACCEPT [952:102330] > :POSTROUTING ACCEPT [0:0] > -A PREROUTING -s 35.154.101.8/32 -p tcp -m tcp --dport 443 -j ACCEPT > -A PREROUTING -p tcp -m tcp --dport 443 -j DNAT --to-destination > 35.154.101.8:3129 > -A POSTROUTING -j MASQUERADE > COMMIT > # Completed on Sun Feb 26 06:28:46 2017 > > 2. Also pasting sudo iptables -L -nv: > > Chain INPUT (policy ACCEPT 216 packets, 16058 bytes) > pkts bytes target prot opt in out source > destination > > Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) > pkts bytes target prot opt in out source > destination > > Chain OUTPUT (policy ACCEPT 161 packets, 24629 bytes) > pkts bytes target prot opt in out source destination > > > >> And then clear out where is this proxy sittings and the network structure. >> It's not clear if the squid box is the router or a machine somewhere on AWS. > > [Michael] This proxy is installed on an AWS instance. > >> If you wish to pass traffic from a local router to a one on AWS you will need to create a tunnel like using OpenVPN or a similar solution and to use some routing rules to pass the traffic from the local LAN to AWS without removing the original destination address. >> > > [Michael] Does this mean, to make ssl-bump work, I will have to setup > a VPN server and configure the VPN clients to use this proxy via VPN > server? > > > Thanks, > Michael. > > _______________________________________________ > squid-users mailing list > squid-users@xxxxxxxxxxxxxxxxxxxxx > http://lists.squid-cache.org/listinfo/squid-users > Thanks for replying Eliezer. Your advice is much appreciated. > The details you attached explained pretty well the cause for the issues you have described. > What you will need to do in order to make this setup to work can be done in more then one way. > For a sysadmin the simplest way is to create a VPN or some kind of a tunnel between the AWS instance to the local router. > I am almost sure that you can use haproxy to do a local tproxy or interception that will forward the traffic to the remote squid with the PROXY protocol keeping original source and original destination visible to the remote squid. > > The choice will depend on both: > - your skills and will to dig some time about couple subjects > - The availability of static IP addresses(both local and AWS). > - The OS on both sides [Michael] Actually, my original setup involves a VPN server. I wasn't using it because I wanted to setup ssl-bump with simplest possible settings. My actual setup involves: 1. strongSwan IPSec VPN server 2. Squid Proxy server 3. Clients will be IPSec VPN clients. I can specify the IP address and port of HTTPS Proxy server in IPSec VPN client itself. In the above setup described, will I have to do something extra to make ssl-bump work? Thanks, Michael. _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
