Hi! I have run two Chrome browsers each behind their own Linux NAT. I have then tried to instruct them to establish a P2P connection with each other using WebRTC. WebRTC is a standard framework that allows P2P communication between browsers and which is available from JavaScript. It turned out that the two browsers were not able to establish a P2P connection. I have looked further into this (see below for details) and it seems to me like the Linux NAT behaves in a way that is far from optimal for supporting nat hole punching and a way that is probably not supported by standard techniques used by WebRTC and other P2P protocols. Can this really be true or am I missing something? I have tried to Google the problem but I did not find any discussions of this nor details on exactly how NATing works in Linux. In the rest of this email I give more details on an experiment I did to confirm the problem. Note that the setup in this experiment is slightly simpler than the one sketched above. Suppose you have four hosts SRV1, SRV2, NAT and CLIENT where: SRV1, SRV2 and NAT are on the same public network NAT acts as a Linux natting router for CLI giving CLI access to the public network. Suppose CLI allocates a UDP port X and sends a packet from X to SRV1 and then to SRV2. This works fine. NAT will preserve the port X when it forwards the packet to SRV1 and SRV2 and further SRV1 and SRV2 can reply back and the packet will get back to CLI. Suppose now CLI allocates a UDP port Y and sends a packet from Y but this time only to SRV1. Suppose now that SRV2 sends a packet to port Y on NAT. This packet is not forwarded back to CLI by NAT which I think it should, but it is no big deal. What nat hole punching would normally do is now to let CLI send a packet from Y to SRV2. However, when this happens, NAT allocates a new port Z and uses that port when forwaring the packet to SRV2. And it is not possible for SRV2 to send packets to CLI on port Y - SRV2 will have to use port Z. So it seems like that the fact that SRV2 sends a packet to CLI (which is discarded) before CLI sends a packet to SRV2, has the side effect that the packet from CLI to SRV2 is NATTed using a different port - which is very bad for nat hole punching. I have created a script that illustrates the problem. Below is both commented output of the script followed by the script itself. In this script: NAT has address 10.0.0.1 on the public network and 192.168.1.1 on the private network SRV1 has address 10.0.0.2 SRV2 has address 10.0.0.3 CLI has address 192.168.1.2 Here is commented output of the sript. First, we look at the case X=5001 which is where things work fine. # CLI sends a packet to SRV1: 19:01:54.097299 IP 192.168.1.2.5001 > 10.0.0.2.6001: UDP, length 0 19:01:54.097337 IP 10.0.0.1.5001 > 10.0.0.2.6001: UDP, length 0 # SRV1 replies back and the packets gets back to CLI: 19:01:54.641166 IP 10.0.0.2.6001 > 10.0.0.1.5001: UDP, length 0 19:01:54.641186 IP 10.0.0.2.6001 > 192.168.1.2.5001: UDP, length 0 # CLI next sends a pakcet to SRV2 from the same port: 19:01:55.181445 IP 192.168.1.2.5001 > 10.0.0.3.6001: UDP, length 0 19:01:55.181513 IP 10.0.0.1.5001 > 10.0.0.3.6001: UDP, length 0 # SRV2 replies back and the packets gets to CLI: 19:01:55.729383 IP 10.0.0.3.6001 > 10.0.0.1.5001: UDP, length 0 19:01:55.729403 IP 10.0.0.3.6001 > 192.168.1.2.5001: UDP, length 0 Next, we look at the case Y=5002 which does not work # CLI sends a packet to SRV1 19:01:56.768882 IP 192.168.1.2.5002 > 10.0.0.2.6002: UDP, length 0 19:01:56.768907 IP 10.0.0.1.5002 > 10.0.0.2.6002: UDP, length 0 # SRV1 replies back and the packets gets back to CLI: 19:01:57.302381 IP 10.0.0.2.6002 > 10.0.0.1.5002: UDP, length 0 19:01:57.302399 IP 10.0.0.2.6002 > 192.168.1.2.5002: UDP, length 0 # SRV2 sends a packet to CLI on Y - this is NOT forwared to CLI 19:01:57.833242 IP 10.0.0.3.6002 > 10.0.0.1.5002: UDP, length 0 # CLI sends a packet to SRV2 from Y. This packet is forwarded with port Z=1024 19:01:58.364815 IP 192.168.1.2.5002 > 10.0.0.3.6002: UDP, length 0 19:01:58.364834 IP 10.0.0.1.1024 > 10.0.0.3.6002: UDP, length 0 # SRV2 tries again to send a packet to CLI on Y - this is still not forwarded to CLI: 19:01:58.897158 IP 10.0.0.3.6002 > 10.0.0.1.5002: UDP, length 0 And here is the script: #!/bin/bash # Root bridge - hosts on this have 10.0.0.0/24 addresses ip netns add nsroot ip netns exec nsroot ip link add bridge type bridge ip netns exec nsroot ip link set dev bridge up # NAT host - has address on 10.0.0.1 on the root bridge ip netns add nsnath ip link add nstmp1 type veth peer name nstmp2 ip link set nstmp1 netns nsnath name veth0 ip link set nstmp2 netns nsroot name veth0 ip netns exec nsroot ip link set dev veth0 up ip netns exec nsroot ip link set veth0 master bridge ip netns exec nsnath ip addr add 10.0.0.1/24 dev veth0 ip netns exec nsnath ip link set dev lo up ip netns exec nsnath ip link set dev veth0 up # NAT bridge - hosts on this havd 192.168.1.0/24 addresses ip netns add nsnatb ip netns exec nsnatb ip link add bridge type bridge ip netns exec nsnatb ip link set dev bridge up # Connect NAT host to NAT bridge with address 192.168.1.1 ip link add nstmp1 type veth peer name nstmp2 ip link set nstmp1 netns nsnath name veth1 ip link set nstmp2 netns nsnatb name veth0 ip netns exec nsnatb ip link set dev veth0 up ip netns exec nsnatb ip link set veth0 master bridge ip netns exec nsnath ip addr add 192.168.1.1/24 dev veth1 ip netns exec nsnath ip link set dev lo up ip netns exec nsnath ip link set dev veth1 up ip netns exec nsnath route add default gw 192.168.1.1 dev veth1 # Set of the NAT host to forward ip netns exec nsnath modprobe iptable_nat ip netns exec nsnath iptables -t nat -A POSTROUTING -o veth0 -j SNAT --to-source 10.0.0.1 ip netns exec nsnath echo 1 > /proc/sys/net/ipv4/ip_forward # CLI host - this is behind the nat with address 192.168.1.2 ip netns add nscli ip link add nstmp1 type veth peer name nstmp2 ip link set nstmp1 netns nscli name veth0 ip link set nstmp2 netns nsnatb name veth1 ip netns exec nsnatb ip link set dev veth1 up ip netns exec nsnatb ip link set veth1 master bridge ip netns exec nscli ip addr add 192.168.1.2/24 dev veth0 ip netns exec nscli ip link set dev lo up ip netns exec nscli ip link set dev veth0 up ip netns exec nscli route add default gw 192.168.1.1 dev veth0 # SRV1 and SRV2 - this is just one host with the two addresses 10.0.0.2 and 10.0.0.3 ip netns add nssrv ip link add nstmp1 type veth peer name nstmp2 ip link set nstmp1 netns nssrv name veth0 ip link set nstmp2 netns nsroot name veth1 ip netns exec nsroot ip link set dev veth1 up ip netns exec nsroot ip link set veth1 master bridge ip netns exec nssrv ip addr add 10.0.0.2/24 dev veth0 ip netns exec nssrv ip addr add 10.0.0.3/24 dev veth0 ip netns exec nssrv ip link set dev lo up ip netns exec nssrv ip link set dev veth0 up # Start tcpdump ip netns exec nsnath tcpdump -n -l -i veth0 udp > external 2>&1 & ip netns exec nsnath tcpdump -n -l -i veth1 udp > internal 2>&1 & # Run commands to send UDP packets sleep 1 ip netns exec nscli sendip -p ipv4 -p udp -us 5001 -ud 6001 -is 192.168.1.2 10.0.0.2 sleep 0.5 ip netns exec nssrv sendip -p ipv4 -p udp -us 6001 -ud 5001 -is 10.0.0.2 10.0.0.1 sleep 0.5 ip netns exec nscli sendip -p ipv4 -p udp -us 5001 -ud 6001 -is 192.168.1.2 10.0.0.3 sleep 0.5 ip netns exec nssrv sendip -p ipv4 -p udp -us 6001 -ud 5001 -is 10.0.0.3 10.0.0.1 sleep 1 ip netns exec nscli sendip -p ipv4 -p udp -us 5002 -ud 6002 -is 192.168.1.2 10.0.0.2 sleep 0.5 ip netns exec nssrv sendip -p ipv4 -p udp -us 6002 -ud 5002 -is 10.0.0.2 10.0.0.1 sleep 0.5 ip netns exec nssrv sendip -p ipv4 -p udp -us 6002 -ud 5002 -is 10.0.0.3 10.0.0.1 sleep 0.5 ip netns exec nscli sendip -p ipv4 -p udp -us 5002 -ud 6002 -is 192.168.1.2 10.0.0.3 sleep 0.5 ip netns exec nssrv sendip -p ipv4 -p udp -us 6002 -ud 5002 -is 10.0.0.3 10.0.0.1 sleep 1 # Kill everything again killall tcpdump ip netns delete nssrv ip netns delete nscli ip netns delete nsnatb ip netns delete nsnath ip netns delete nsroot -- To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
