Thank you Alex, Indeed something is listening on this port, but it looks to be Squid: root@A2-3:/# nc -6 -l 3128 nc: Address already in use root@A2-3:/# lsof -i:3128 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME squid 3480423 proxy 25u IPv4 283726201 0t0 TCP A2-3:3128 (LISTEN) but the socket is IPV4 only on the problem host: root@A2-3:/# lsof -a -i4 -i6 -itcp | grep 3128 squid 3480423 proxy 25u IPv4 283726201 0t0 TCP A2-3:3128 (LISTEN) compared to a 'healthy' server: root@A2-2:~# lsof -a -i4 -i6 -itcp | grep 3128 squid 997651 proxy 12u IPv6 254219302 0t0 TCP A2-2:3128->x.x.x.x:46816 (ESTABLISHED) squid 997651 proxy 25u IPv6 241163587 0t0 TCP *:3128 (LISTEN) As I know a IPV6 socket accepts both v4 and v6 connections but a V4 socket only V4 connections, and this looks to be the symptom. This is what I found in the cache.log: 2024/03/04 16:09:28 kid1| With 1000000 file descriptors available 2024/03/04 16:09:28 kid1| Initializing IP Cache... 2024/03/04 16:09:28 kid1| DNS IPv6 socket created at [::], FD 9 2024/03/04 16:09:28 kid1| DNS IPv4 socket created at 0.0.0.0, FD 10 so it looks like it creates the IPv6 socket but it's not working somehow: root@A2-3:/# telnet ::1 3128 Trying ::1... telnet: Unable to connect to remote host: Connection refused Unfortunately nothing else relevant to me in the cache.log, I enabled debugging, to what email can I send the archive for you to look at it, please? Thank you, Dragos Sent with Proton Mail secure email. On Monday, March 4th, 2024 at 9:43 PM, Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> wrote: > On 2024-03-04 14:03, Dragos Pacher wrote: > > > POC running well on 3 servers but on the 4th I get no IPv6 > > sockets: > > ubuntu@A2-3:/$ sudo netstat -patun | grep squid | grep tcp > > tcp 0 0 10.10.0.16:3128 0.0.0.0:* > > LISTEN 2891391/(squid-1) > > > Are there any other processes listening on IPv6 addresses on this > problematic host? > > Does something like "nc -6 -l 3128" listen on an IPv6 address on this > problematic host? > > If possible, please also check cache.log for messages mentioning IPv6 > and "BCP 177"; I know you shared syslog output, but I am a bit worried > that syslog might be missing some relevant early debugging messages. > > > If nothing helps, consider sharing a pointer to compressed Squid startup > cache.log after adding "debug_options ALL,2 50,3" to your squid.conf. We > do not need to see any transactions, just Squid startup steps. Still, > this log may contain some sensitive details, so share privately if needed. > > > Thank you, > > Alex. > > > > and on the other 3 I have IPv6: > > ubuntu@A2-2:/$ sudo netstat -patun | grep squid | grep tcp > > tcp 0 0 x.x.x.x:52386 x.x.x.x:443 ESTABLISHED > > 997651/(squid-1) > > tcp6 0 0 :::3128 :::* > > LISTEN 997651/(squid-1) > > tcp6 0 0 10.10.0.12:3128 10.20.0.1:39428 > > ESTABLISHED 997651/(squid-1) > > > > > > > > This creates a problem for us since the apps I monitor are not starting > > since their start routine is IPV6 only and then they switch to > > IPv4/IPV6, but the start is IPV6 alone. > > > > Therefore my questions are as follows: > > > > 1. How can I make it listen on both IPV6/IPV4 like on the other servers? > > 2. Any configuration improvement suggestions? > > > > Please find all details here: > > So far I did a POC on 4 servers, here is the full config, nothing > > sophisticated since this is where my Squid knowledge took me so far. > > Running Squid 6.7 with some basic options > > on Ubuntu 22.04 kernel 5.15.0-89-generic x86_64 > > squid -v > > Squid Cache: Version 6.7 > > Service Name: squid > > This binary uses OpenSSL 3.0.2 15 Mar 2022. configure options: > > '--prefix=/usr' '--localstatedir=/var' '--libexecdir=/lib/squid' > > '--datadir=/share/squid' '--sysconfdir=/etc/squid' > > '--with-default-user=proxy' '--with-logdir=/var/log/squid' > > '--enable-ssl-crtd' '--with-openssl' > > > > and here is the syslog of Squid start: > > Mar 4 16:09:28 A2-3 systemd[1]: Starting Squid Web Proxy Server... > > Mar 4 16:09:28 A2-3 squid[3094662]: 2024/03/04 16:09:28| Processing > > Configuration File: /etc/squid/squid.conf (depth 0) > > Mar 4 16:09:28 A2-3 squid[3094662]: 2024/03/04 16:09:28| WARNING: empty > > ACL: acl broken_sites ssl::server_name "/etc/squid/ssl_broken_sites.txt" > > Mar 4 16:09:28 A2-3 squid[3094662]: 2024/03/04 16:09:28| WARNING: The > > "Hs" formatting code is deprecated. Use the ">Hs" instead. > > Mar 4 16:09:28 A2-3 squid[3094662]: 2024/03/04 16:09:28| Created PID > > file (/var/run/squid.pid) > > Mar 4 16:09:28 A2-3 squid[3094662]: Squid Parent: will start 1 kids > > Mar 4 16:09:28 A2-3 squid[3094662]: Squid Parent: (squid-1) process > > 3094665 started > > Mar 4 16:09:28 A2-3 squid[3094665]: 2024/03/04 16:09:28 kid1| > > Processing Configuration File: /etc/squid/squid.conf (depth 0) > > Mar 4 16:09:28 A2-3 squid[3094665]: 2024/03/04 16:09:28 kid1| WARNING: > > empty ACL: acl broken_sites ssl::server_name > > "/etc/squid/ssl_broken_sites.txt" > > Mar 4 16:09:28 A2-3 squid[3094665]: 2024/03/04 16:09:28 kid1| WARNING: > > The "Hs" formatting code is deprecated. Use the ">Hs" instead. > > Mar 4 16:09:28 A2-3 squid[3094665]: 2024/03/04 16:09:28 kid1| Set > > Current Directory to /var/cache/squid > > Mar 4 16:09:28 A2-3 squid[3094665]: 2024/03/04 16:09:28 kid1| Creating > > missing swap directories > > Mar 4 16:09:28 A2-3 squid[3094665]: 2024/03/04 16:09:28 kid1| No > > cache_dir stores are configured. > > Mar 4 16:09:28 A2-3 squid[3094662]: Squid Parent: squid-1 process > > 3094665 exited with status 0 > > Mar 4 16:09:28 A2-3 squid[3094662]: 2024/03/04 16:09:28| Removing PID > > file (/var/run/squid.pid) > > Mar 4 16:09:28 A2-3 squid[3094666]: Processing Configuration File: > > /etc/squid/squid.conf (depth 0) > > Mar 4 16:09:28 A2-3 squid[3094666]: WARNING: empty ACL: acl > > broken_sites ssl::server_name "/etc/squid/ssl_broken_sites.txt" > > Mar 4 16:09:28 A2-3 squid[3094666]: WARNING: The "Hs" formatting code > > is deprecated. Use the ">Hs" instead. > > Mar 4 16:09:28 A2-3 squid[3094666]: Created PID file (/var/run/squid.pid) > > Mar 4 16:09:28 A2-3 squid[3094666]: Squid Parent: will start 1 kids > > Mar 4 16:09:28 A2-3 squid[3094666]: Squid Parent: (squid-1) process > > 3094668 started > > Mar 4 16:09:28 A2-3 squid[3094668]: Processing Configuration File: > > /etc/squid/squid.conf (depth 0) > > Mar 4 16:09:28 A2-3 squid[3094668]: WARNING: empty ACL: acl > > broken_sites ssl::server_name "/etc/squid/ssl_broken_sites.txt" > > Mar 4 16:09:28 A2-3 squid[3094668]: WARNING: The "Hs" formatting code > > is deprecated. Use the ">Hs" instead. > > Mar 4 16:09:28 A2-3 squid[3094668]: Set Current Directory to > > /var/cache/squid > > Mar 4 16:09:28 A2-3 squid[3094668]: Starting Squid Cache version 6.7 > > for x86_64-pc-linux-gnu... > > Mar 4 16:09:28 A2-3 squid[3094668]: Service Name: squid > > Mar 4 16:09:28 A2-3 squid[3094668]: Process ID 3094668 > > Mar 4 16:09:28 A2-3 squid[3094668]: Process Roles: worker > > Mar 4 16:09:28 A2-3 squid[3094668]: With 1000000 file descriptors available > > Mar 4 16:09:28 A2-3 squid[3094668]: Initializing IP Cache... > > Mar 4 16:09:28 A2-3 squid[3094668]: DNS IPv6 socket created at [::], FD 9 > > Mar 4 16:09:28 A2-3 squid[3094668]: DNS IPv4 socket created at 0.0.0.0, > > FD 10 > > Mar 4 16:09:28 A2-3 squid[3094668]: Adding nameserver 127.0.0.53 from > > /etc/resolv.conf > > Mar 4 16:09:28 A2-3 squid[3094668]: Adding domain . from /etc/resolv.conf > > Mar 4 16:09:28 A2-3 squid[3094668]: helperOpenServers: Starting 5/5 > > 'security_file_certgen' processes > > Mar 4 16:09:28 A2-3 squid[3094668]: Logfile: opening log > > stdio:/var/log/squid/success.log > > Mar 4 16:09:28 A2-3 squid[3094668]: Logfile: opening log > > stdio:/var/log/squid/failure.log > > Mar 4 16:09:28 A2-3 squid[3094668]: Logfile: opening log > > daemon:/var/log/squid/access.log > > Mar 4 16:09:28 A2-3 squid[3094668]: Logfile Daemon: opening log > > /var/log/squid/access.log > > Mar 4 16:09:28 A2-3 squid[3094668]: Store logging disabled > > Mar 4 16:09:28 A2-3 squid[3094668]: Swap maxSize 0 + 262144 KB, > > estimated 20164 objects > > Mar 4 16:09:28 A2-3 squid[3094668]: Target number of buckets: 1008 > > Mar 4 16:09:28 A2-3 squid[3094668]: Using 8192 Store buckets > > Mar 4 16:09:28 A2-3 squid[3094668]: Max Mem size: 262144 KB > > Mar 4 16:09:28 A2-3 squid[3094668]: Max Swap size: 0 KB > > Mar 4 16:09:28 A2-3 squid[3094668]: Using Least Load store dir selection > > Mar 4 16:09:28 A2-3 squid[3094668]: Set Current Directory to > > /var/cache/squid > > Mar 4 16:09:28 A2-3 squid[3094668]: Finished loading MIME types and icons. > > Mar 4 16:09:28 A2-3 squid[3094668]: HTCP Disabled. > > Mar 4 16:09:28 A2-3 squid[3094668]: Squid plugin modules loaded: 0 > > Mar 4 16:09:28 A2-3 squid[3094668]: Adaptation support is off. > > Mar 4 16:09:28 A2-3 squid[3094668]: Accepting SSL bumped HTTP Socket > > connections at conn13 local=10.10.0.16:3128 remote=[::] FD 25 > > flags=9#012 listening port: 10.10.0.16:3128 > > Mar 4 16:09:28 A2-3 systemd[1]: Started Squid Web Proxy Server. > > Mar 4 16:09:29 A2-3 squid[3094668]: storeLateRelease: released 0 objects > > > > -- full config -- > > acl SSL_ports port 443 > > acl SSL_ports port 443 > > http_access allow localhost > > http_access allow localnet > > http_access allow all > > > > acl step1 at_step SslBump1 > > acl step2 at_step SslBump2 > > acl step3 at_step SslBump3 > > > > acl broken_sites ssl::server_name "/etc/squid/ssl_broken_sites.txt" > > http_upgrade_request_protocols websocket allow all > > > > ssl_bump peek step1 all > > ssl_bump splice broken_sites > > ssl_bump stare step2 all > > ssl_bump bump step3 all > > > > acl CONNECT method CONNECT > > acl success_hier hier_code HIER_DIRECT > > acl failure_hier hier_code HIER_NONE > > acl failure all-of CONNECT failure_hier > > acl failure all-of !CONNECT failure_codes > > acl success all-of CONNECT success_hier > > acl success all-of !CONNECT success_codes > > > > access_log stdio:/var/log/squid/success.log logformat=squid success > > access_log stdio:/var/log/squid/failure.log logformat=squid failure > > > > cache deny all > > > > http_port [::]:3128 ssl-bump generate-host-certificates=on > > dynamic_cert_mem_cache_size=8MB tls-cert=/etc/squid/myCA.pem > > tls-key=/etc/squid/myCA1.pem > > strip_query_terms off > > > > logformat timereadable %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt > > access_log daemon:/var/log/squid/access.log timereadable > > > > coredump_dir /var/cache/squid > > refresh_pattern ^ftp: 1440 20% 10080 > > refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 > > refresh_pattern . 0 20% 4320 > > sslcrtd_program /usr/lib/squid/security_file_certgen -s > > /var/lib/squid/ssl_db -M 16MB > > sslcrtd_children 5 > > ssl_bump server-first all > > sslproxy_cert_error allow all > > -- end of config > > > > Thank you, > > > > Dragos > > > > Sent with Proton Mail https://proton.me/ secure email. > > > > _______________________________________________ > > squid-users mailing list > > squid-users@xxxxxxxxxxxxxxxxxxxxx > > https://lists.squid-cache.org/listinfo/squid-users > > > _______________________________________________ > squid-users mailing list > squid-users@xxxxxxxxxxxxxxxxxxxxx > https://lists.squid-cache.org/listinfo/squid-users _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx https://lists.squid-cache.org/listinfo/squid-users
