Hi all, I am experiencing somewhat of a strange error with squid using ssl-bump. I think I am running a somewhat typical set up in which we run a squid proxy server fleet that is used by our other servers and we use "ssl_bump" to man-in-the-middle all our traffic. The problem is that occasionally requests to "https://pypi.org"; will time out. Even weirder, we are running in Google Compute Platform and the same squid container will error out depending on different regions. This bug has been confounding us for months now and any help would be greatly appreciated. We have tried turning on verbose debugging and I think I have found the logs in which squid encounters an error with the request: 2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(88) prepareSocket: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1, this=0x1d3b788 2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(94) prepareSocket: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 2020/05/21 19:27:28.567| 9,5| AsyncCall.cc(26) AsyncCall: The AsyncCall Security::PeerConnector::commCloseHandler constructed, this=0x1d2c990 [call2471] 2020/05/21 19:27:28.567| 5,5| comm.cc(985) comm_add_close_handler: comm_add_close_handler: FD 14, AsyncCall=0x1d2c990*1 2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(107) initialize: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1, ctx=0x176ea70 2020/05/21 19:27:28.567| 83,5| Session.cc(103) NewSessionObject: SSL_new session=0x1d37d50 2020/05/21 19:27:28.567| 83,5| bio.cc(612) squid_bio_ctrl: 0x1d1a380 104(6001, 0x7ffe0c004ee4) 2020/05/21 19:27:28.567| 83,5| Session.cc(161) CreateSession: link FD 14 to TLS session=0x1d37d50 2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(123) initialize: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1, session=0x1d37d50 2020/05/21 19:27:28.567| 14,3| Address.cc(382) lookupHostIP: Given Non-IP 'pypi.org': Name or service not known 2020/05/21 19:27:28.567| 83,5| BlindPeerConnector.cc(60) initialize: success 2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(188) negotiate: SSL_connect session=0x1d37d50 2020/05/21 19:27:28.567| 83,5| bio.cc(612) squid_bio_ctrl: 0x1d1a380 6(0, 0x1d16bf0) 2020/05/21 19:27:28.568| 83,5| bio.cc(113) write: FD 14 wrote 314 <= 314 2020/05/21 19:27:28.568| 83,5| bio.cc(612) squid_bio_ctrl: 0x1d1a380 11(0, 0) 2020/05/21 19:27:28.568| 83,5| bio.cc(136) read: FD 14 read -1 <= 65535 2020/05/21 19:27:28.568| 83,5| bio.cc(141) read: error: 11 ignored: 1 2020/05/21 19:27:28.568| 83,5| PeerConnector.cc(462) noteWantRead: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 2020/05/21 19:27:28.568| 5,3| comm.cc(559) commSetConnTimeout: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 timeout 60 2020/05/21 19:27:28.568| 5,5| ModEpoll.cc(117) SetSelect: FD 14, type=1, handler=1, client_data=0x1d2ca60, timeout=0 2020/05/21 19:27:28.568| 93,5| AsyncJob.cc(154) callEnd: Security::BlindPeerConnector status out: [ FD 14 job74] 2020/05/21 19:27:28.568| 93,5| AsyncCallQueue.cc(57) fireNext: leaving AsyncJob::start() 2020/05/21 19:27:28.580| 83,5| PeerConnector.cc(188) negotiate: SSL_connect session=0x1d37d50 2020/05/21 19:27:28.580| 83,5| bio.cc(136) read: FD 14 read 4143 <= 65535 2020/05/21 19:27:28.580| 83,5| Handshake.cc(405) parseExtensions: first unsupported extension: 43 2020/05/21 19:27:28.580| 24,5| BinaryTokenizer.cc(47) want: 1 more bytes for TLSPlaintext.type occupying 1 bytes @4143 in 0x1b94840; 2020/05/21 19:27:28.580| 83,5| Handshake.cc(533) parseHello: need more data 2020/05/21 19:27:28.580| 83,5| PeerConnector.cc(462) noteWantRead: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 2020/05/21 19:27:28.580| 5,3| comm.cc(559) commSetConnTimeout: local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 timeout 60 2020/05/21 19:27:28.580| 5,5| ModEpoll.cc(117) SetSelect: FD 14, type=1, handler=1, client_data=0x1d18730, timeout=0 2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(26) AsyncCall: The AsyncCall logfileFlush constructed, this=0x1d018e0 [call2473] 2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(93) ScheduleCall: event.cc(241) will call logfileFlush(0x1895498*?) [call2473] 2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(55) fireNext: entering logfileFlush(0x1895498*?) 2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(38) make: make call logfileFlush [call2473] 2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(57) fireNext: leaving logfileFlush(0x1895498*?) 2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(26) AsyncCall: The AsyncCall MaintainSwapSpace constructed, this=0x1d018e0 [call2474] 2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(93) ScheduleCall: event.cc(241) will call MaintainSwapSpace() [call2474] 2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(55) fireNext: entering MaintainSwapSpace() 2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(38) make: make call MaintainSwapSpace [call2474] 2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(57) fireNext: leaving MaintainSwapSpace() 2020/05/21 19:27:29.651| 41,5| AsyncCall.cc(26) AsyncCall: The AsyncCall memPoolCleanIdlePools constructed, this=0x1d018e0 [call2475] We are running squid 4.9 with the following configs: # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN) acl localnet src 10.0.0.0/8 # RFC 1918 local private network (LAN) acl localnet src 100.64.0.0/10 # RFC 6598 shared address space (CGN) acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly plugged) machines acl localnet src 172.16.0.0/12 # RFC 1918 local private network (LAN) acl localnet src 192.168.0.0/16 # RFC 1918 local private network (LAN) acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl google_bq_storage ssl::server_name bigquerystorage.googleapis.com http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager ### Custom rules follows: ### # Logging rules cache_log /var/log/squid/cache.log access_log daemon:/var/log/squid/access.log squid # Intercept all egress and man-the-middle with our own certificate ssl_bump splice google_bq_storage ssl_bump bump all http_port 3128 ssl-bump tls-cert=/etc/squid/ssl_cert/our_cert.pem tls-key=/etc/squid/ssl_cert/our_cert.pem options=SINGLE_DH_USE,SINGLE_ECDH_USE sslcrtd_program /nix/store/3pwrqm8fjf6bljp3rk50bk5n3ziwaz0a-squid-4.9/libexec/security_file_certgen -s /nix/store/3pwrqm8fjf6bljp3rk50bk5n3ziwaz0a-squid-4.9/var/cache/squid/ssl_db -M 4MB # Additional certificate authorities to trust. Used when squid establishes TLS with the target site. tls_outgoing_options cafile=/etc/squid/ssl_cert/additional_root_certs.crt cache deny all ### Custom rules end: ### coredump_dir /squid/var/cache/squid refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 -- Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
