IPsec peer with SNAT not working

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello list,

I have an issue with conntrack on kernel 3.10.33-1-lts on Arch Linux.
There are peers connected to  a box on the network, that runs the strongSwan IPsec daemon, which handles the establishment of the IPsec tunnels and
then gives them IPs from the from me reserved range to them.
I use the 192.168.178.0/24 address space in the LAN and 172.16.19.0/24 for the IPsec peers.
The box that the peers connect to, does SNAT to hide the unknown IP addresses from the LAN hosts, so they can communicate with the IPsec peers.

I can not do without SNAT in this situation. Please do not recommend getting the NAT out. That is not an option.
Neither is connecting the peers to the LAN on layer two.

When the peers send out an TCP SYN packet to hosts on the LAN, they get SNATed correctly, but the response does not get translated.
The response just gets dropped after *mangle prerouting.
The rules I used to find that out are attached to the email.

Short info to the rules:
I use connmark to mark connections from the IPsec peers with hexadecimal 0xAA (decimal 170).
I then use -j LOG to get the packets that run through *mangle prerouting and see the mark contnrack sets on them.
The SYN packets have them, the response does not.
A log snippet showing that is attached to the email.

Also, the output of "conntrack -L -s 172.16.19.1 -d 192.168.178.43" is attached to the email.

The packet capture and the log files show the same traffic that is an IPsec peer trying to connect to 192.168.178.43 tcp port 8080.

IPs:
192.168.178.48 LAN side IP of the IPsec responder (Where the tunnel terminates and the SNAT is on)
192.168.178.43 LAN IP of the box I tried to connect to.

IP ranges:
192.168.178.0/24 for LAN
172.16.19.0/24 IPsec peers
172.16.20.0/24 IPsec peers

It would be very nice if anyone could tell me what could be the cause of this.


Regards

Noel Kuntze

Attachements:

The iptables rules I used to find out, that the packets don't get tracked: debug.ipt
A packet capture from a box on the lan with an IPsec peer trying to connect to it on port 8080 tcp: capture.pcapng
A log snippet from the box the SNAT is working on and the IPsec tunnels terminate: snippet.log
Signature files for them.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTLhldAAoJEDg5KY9j7GZYIxkP/2mwliDE2hzbTV94G4TX+McI
/l4mGaTC7EutQQ7xzM/h80yZFzZO9ucfRJoltTalAX/EDXx9oGryAzgN1bzn6W61
UV89Gzbr3MlN0Qjz4X1/9PQAPcnYoze5wI9P3N1f+FpN0gPEAiBPWYNTtG1Sweg3
VwWGqjwdk+nsvB3QN7TV8RSADraTcmHb7/ENmyxKCxuf+j5QHOFhLjb9KfBl01Qh
LqEOGNPLk7Kwxaq4egp47wLqEtUELeKHKl7kR/WTo+aIh+F23YRhc71ld60x+PGJ
QJLOUJeAwl+HN6I+fTR7oD8dKsHJvmBpk0yOX4gRe1mOSRzwC+NVh1mrHJbmJQ0o
YtpAF4jADhtvmldLqyCCEQ4OWFK/ksiC4ktBUMbUeJqoNIID+ogwikLz8u/FiB5T
cXEaPWVDhPV++SYBqD6daHODbeuUswzP0wAd1R+PkYdzkDUi4tMzGKbEj3fkhHzD
lR61RKYInbwiDuBJWAKgoskUtojJS9k3otwDzk8xNadyQz1vnAGAo0x8cSzQvest
DPmRT2suu5uhB1U1A+7nSPFdlGjLNB/0OUsNQIXMjcTFwIshms4yQEqK9Q6IHR5W
Z5Q0qYyXf7tVacNbdJV53+w2oQRh2m25EprPwklvijr4OSxhzwhpji37Fih3RePC
Y3tO7PvIZ8mh7reyLdmX
=lkdZ
-----END PGP SIGNATURE-----

Attachment: capture.pcapng
Description: Binary data

conntrack -L -s 172.16.19.1 -d 192.168.178.43
tcp      6 58 SYN_RECV src=172.16.19.1 dst=192.168.178.43 sport=57532 dport=8080 src=192.168.178.43 dst=192.168.178.48 sport=8080 dport=4127 mark=170 use=1
tcp      6 37 SYN_RECV src=172.16.19.1 dst=192.168.178.43 sport=57531 dport=8080 src=192.168.178.43 dst=192.168.178.48 sport=8080 dport=4126 mark=170 use=1
conntrack v1.4.2 (conntrack-tools): 2 flow entries have been shown.


Mär 22 22:43:25 vms.thermi kernel: TRACE: raw:PREROUTING:policy:3 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=6
4 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:PREROUTING:rule:2 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=
64 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:PREROUTING:rule:5 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=
64 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: PREROUTING IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=18047 DF PROTO=
TCP SPT=57561 DPT=8080 WINDOW=13600 RES=0x00 SYN URGP=0 MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:PREROUTING:policy:7 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TT
L=64 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: nat:PREROUTING:policy:1 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=6
4 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:FORWARD:rule:1 IN=br0 OUT=br0 MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=
63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: FORWARD IN=br0 OUT=br0 MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18047 DF PROTO=
TCP SPT=57561 DPT=8080 WINDOW=13600 RES=0x00 SYN URGP=0 MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:FORWARD:policy:5 IN=br0 OUT=br0 MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TT
L=63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: filter:FORWARD:policy:1 IN=br0 OUT=br0 MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TT
L=63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:POSTROUTING:rule:2 IN= OUT=br0 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 S
EQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: POSTROUTING IN= OUT=br0 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 WINDOW=13600 RES=0x00 
SYN URGP=0 MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:POSTROUTING:policy:3 IN= OUT=br0 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080
 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: nat:POSTROUTING:rule:1 IN= OUT=br0 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18047 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=
3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A00908F090000000001030301) MARK=0xaa 
Mär 22 22:43:25 vms.thermi kernel: TRACE: raw:PREROUTING:policy:3 IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00 
PREC=0x00 TTL=64 ID=34223 DF PROTO=TCP SPT=8080 DPT=4096 SEQ=0 ACK=3919038839 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:PREROUTING:rule:6 IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00
 PREC=0x00 TTL=64 ID=34223 DF PROTO=TCP SPT=8080 DPT=4096 SEQ=0 ACK=3919038839 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:25 vms.thermi kernel: PREROUTING IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=
34223 DF PROTO=TCP SPT=8080 DPT=4096 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:25 vms.thermi kernel: TRACE: mangle:PREROUTING:policy:7 IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x
00 PREC=0x00 TTL=64 ID=34223 DF PROTO=TCP SPT=8080 DPT=4096 SEQ=0 ACK=3919038839 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:28 vms.thermi kernel: TRACE: raw:PREROUTING:policy:3 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=6
4 ID=18048 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A0090908A0000000001030301) 
Mär 22 22:43:28 vms.thermi kernel: TRACE: mangle:PREROUTING:rule:2 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=
64 ID=18048 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A0090908A0000000001030301) 
Mär 22 22:43:28 vms.thermi kernel: TRACE: mangle:PREROUTING:rule:5 IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=
64 ID=18048 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A0090908A0000000001030301) MARK=0xaa 
Mär 22 22:43:28 vms.thermi kernel: PREROUTING IN=br0 OUT= MAC=38:ea:a7:a4:a1:a1:00:1b:11:f0:5f:62:08:00 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=18048 DF PROTO=
TCP SPT=57561 DPT=8080 WINDOW=13600 RES=0x00 SYN URGP=0 MARK=0xaa 
Mär 22 22:43:28 vms.thermi kernel: TRACE: mangle:POSTROUTING:policy:3 IN= OUT=br0 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18048 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A0090908A0000000001030301) MARK=0xaa 
Mär 22 22:43:28 vms.thermi kernel: TRACE: nat:POSTROUTING:rule:1 IN= OUT=br0 SRC=172.16.19.1 DST=192.168.178.43 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=18048 DF PROTO=TCP SPT=57561 DPT=8080 SEQ=3919038838 ACK=0 WINDOW=13600 RES=0x00 SYN URGP=0 OPT (020405500402080A0090908A0000000001030301) MARK=0xaa 
Mär 22 22:43:28 vms.thermi kernel: TRACE: raw:PREROUTING:policy:3 IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=34224 DF PROTO=TCP SPT=8080 DPT=4096 SEQ=0 ACK=3919038839 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:28 vms.thermi kernel: TRACE: mangle:PREROUTING:rule:6 IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=34224 DF PROTO=TCP SPT=8080 DPT=4096 SEQ=0 ACK=3919038839 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:28 vms.thermi kernel: PREROUTING IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=34224 DF PROTO=TCP SPT=8080 DPT=4096 WINDOW=0 RES=0x00 ACK RST URGP=0 
Mär 22 22:43:28 vms.thermi kernel: TRACE: mangle:PREROUTING:policy:7 IN=br0 OUT= PHYSIN=eth0 MAC=38:ea:a7:a4:a1:a1:00:23:54:45:e6:ac:08:00 SRC=192.168.178.43 DST=192.168.178.48 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=34224 DF PROTO=TCP SPT=8080 DPT=4096 SEQ=0 ACK=3919038839 WINDOW=0 RES=0x00 ACK RST URGP=0 


# Generated by iptables-save v1.4.21 on Wed Mar 19 01:30:40 2014
*filter
:INPUT ACCEPT [189057:198102469]
:FORWARD ACCEPT [2122:135954]
:OUTPUT ACCEPT [109534:36441281]
COMMIT
# Completed on Wed Mar 19 01:30:40 2014
# Generated by iptables-save v1.4.21 on Wed Mar 19 01:30:40 2014
*nat
:PREROUTING ACCEPT [1053:155096]
:INPUT ACCEPT [227:30106]
:OUTPUT ACCEPT [229:17502]
:POSTROUTING ACCEPT [229:17502]
-A POSTROUTING -p tcp -m connmark --mark 0xaa -j SNAT --to-source 192.168.178.48:4096-32000
-A POSTROUTING -p udp -m connmark --mark 0xaa -j SNAT --to-source 192.168.178.48:4096-32000
-A POSTROUTING -p tcp -m connmark --mark 0xba -j SNAT --to-source 192.168.178.48:4096-32000
-A POSTROUTING -p udp -m connmark --mark 0xba -j SNAT --to-source 192.168.178.48:4096-32000
COMMIT
# Completed on Wed Mar 19 01:30:40 2014
# Generated by iptables-save v1.4.21 on Wed Mar 19 01:30:40 2014
*mangle
:PREROUTING ACCEPT [158815:193064478]
:INPUT ACCEPT [157239:192917818]
:FORWARD ACCEPT [536:34792]
:OUTPUT ACCEPT [87470:19044799]
:POSTROUTING ACCEPT [88021:19080291]
-A PREROUTING -s 172.16.20.0/24 -m policy --dir in --pol ipsec -j CONNMARK --set-xmark 0xba/0xffffffff
-A PREROUTING -s 172.16.19.0/24 -m policy --dir in --pol ipsec -j CONNMARK --set-xmark 0xaa/0xffffffff
-A PREROUTING -d 172.16.20.0/24 -m policy --dir in --pol ipsec -j CONNMARK --set-xmark 0xbb/0xffffffff
-A PREROUTING -d 172.16.19.0/24 -m policy --dir in --pol ipsec -j CONNMARK --set-xmark 0xab/0xffffffff
-A PREROUTING -p tcp -m tcp --dport 8080 -j LOG --log-prefix "PREROUTING "
-A PREROUTING -p tcp -m tcp --sport 8080 -j LOG --log-prefix "PREROUTING "
-A INPUT -p tcp -m tcp --sport 8080 -j LOG --log-prefix "INPUT "
-A FORWARD -p tcp -m tcp --dport 8080 -j LOG --log-prefix "FORWARD "
-A FORWARD -p tcp -m tcp --sport 8080 -j LOG --log-prefix "FORWARD "
-A FORWARD -d 172.16.19.0/24 -m policy --dir in --pol ipsec -j CONNMARK --set-xmark 0xab/0xffffffff
-A FORWARD -d 172.16.20.0/24 -m policy --dir in --pol ipsec -j CONNMARK --set-xmark 0xbb/0xffffffff
-A POSTROUTING -p tcp -m tcp --sport 8080 -j LOG --log-prefix "POSTROUTING "
-A POSTROUTING -p tcp -m tcp --dport 8080 -j LOG --log-prefix "POSTROUTING "
COMMIT
# Completed on Wed Mar 19 01:30:40 2014
# Generated by iptables-save v1.4.21 on Wed Mar 19 01:30:40 2014
*raw
:PREROUTING ACCEPT [193787:198508440]
:OUTPUT ACCEPT [109536:36441449]
-A PREROUTING -s 172.16.19.1/24 -d 192.168.178.43 -p tcp --dport 8080 -j TRACE
-A PREROUTING -s 192.168.178.43 -p tcp --sport 8080 -j TRACE
COMMIT
# Completed on Wed Mar 19 01:30:40 2014

Attachment: capture.pcapng.sig
Description: PGP signature

Attachment: conntrack.log.sig
Description: PGP signature

Attachment: snippet.log.sig
Description: PGP signature

Attachment: debug.ipt.sig
Description: PGP signature

[Index of Archives]     [Linux Netfilter Development]     [Linux Kernel Networking Development]     [Netem]     [Berkeley Packet Filter]     [Linux Kernel Development]     [Advanced Routing & Traffice Control]     [Bugtraq]

  Powered by Linux