When a TCP client does not receive a response from a service, the client can opt to time out and exit. If the connection is passed through an SSH tunnel, however, certain circumstances can make the SSH tunnel hang indefinitely. This affects both remote port forwarding (-R) and local (-L). This report is for current openssh-portable git running on Linux. For released versions, at least OpenSSH 8.9p1 is affected as well, though I did not test other versions. --- Remote Forwarding --- To reproduce, first start up a TCP service which will accept connections but fail to respond thereafter. One way to do this is by stopping netcat shortly after startup (options are for the OpenBSD version of netcat). $ nc -k -l 127.0.0.1 9999 > /dev/null & sleep 1 ; kill -STOP %% Next use a different terminal to start an SSH service. Debug options are not required but are helpful for diagnosis. This example uses port 2222 in order to avoid conflict with the system SSH service. The service can be run on the same local host or on a remote host. $ sudo /usr/sbin/sshd -d -d -f /dev/null -o Port=2222 -o \ HostKey=/etc/ssh/ssh_host_ed25519_key Next use a different terminal to start an SSH client which runs a TCP client over a forwarded connection. This example uses wget, but any TCP client that can be configured to time out should behave the same. $ ssh localhost -p 2222 -R 8888:127.0.0.1:9999 -v -v wget \ --timeout=1 --tries=1 http://127.0.0.1:8888 The observed results are that the TCP client (wget) exits, but the SSH client hangs until either manually killed or the TCP service (netcat) is resumed. In detail, the sequence of events is as follows: 1. The SSH client connects to the server; the client and server set up channels as usual, including one for the port-forwarding. The SSH client starts a TCP client on the SSH server. 2. The TCP client connects to the SSH server's listening socket, and the SSH client connects to the TCP service's listening socket. The 3-way handshakes complete, but when the TCP client sends data to the service, the service never responds. 3. The TCP client times out, closes its socket for the connection to the SSH server, and exits. The SSH server sends the client an EOF on the forwarded channel, but does not close its own socket for the connection to the now-exited TCP client; this socket remains in CLOSE_WAIT. 4. The SSH client receives the EOF and drains the channel output, but continues to wait for data on the channel input. The SSH server won't close the channel until the client does, and the client won't close the channel until it receives data (or an error) from the channel. --- Local Forwarding --- The situation for local forwarding is similar, but requires different steps to reproduce. First start a TCP service and an SSH service as described above. Then use a new terminal to start an SSH client: $ ssh localhost -p 2222 -L 8888:127.0.0.1:9999 -v -v Use a new terminal to run a TCP client over the forwarded connection. $ wget --timeout=1 --tries=1 http://127.0.0.1:8888 Lastly, exit from the SSH client's interactive shell. The observed results are then the same hang as for remote forwarding. I will send a patch shortly that fixes the issue for me, though I do not know if my fix is correct. Thanks, Corey _______________________________________________ openssh-unix-dev mailing list openssh-unix-dev@xxxxxxxxxxx https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
