[LARTC] ICMP redirect and DNAT problem

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I've tried to trace down this problem as much as I can, but couldn't
really solve it, so I send this long description of what I think is
happening. Hopfully, someone can help.

Since recently, sending ICQ messages through the server just fails
most of the time. The only alternative I know of is to allow direct
connections, which do work. So I assigned a small port range to each
client on the local net and DNAT incoming connections for them. This
seems to work fine. The problem is with clients which are actually on
the same local net. The initical connection takes a long time, but
then, it succeeds and works very quickly. A closer look showed that
the firewall returns the packet with an ICMP type 5 redirect message,
but the kernel continues accepting and sending ICMP type 5 messages
after setting those to 0 in the /proc directory of the 3 computers.

To find out why this happens, I've traced everything using tcpdump,
logging all packets on the firewall's netfilter script and the
logfiles from the ICQ clients.

As far as I understood ICMP redirects, it works like this. Let's
assume machine "boy" tries to contact machine "girl", crossing several
routers. On "boy", the kernel's routing tables tell to send the packet
to "router1". That thinks, that the correct way to reach "girl" is
sending the packet to "cop", which could reach "girl" via "router3":

	boy -> router1 -> cop -> router3 -> girl

But cop finds out, that router1 also could reach girl, using router3
as a gateway. So cop sends router1 an ICMP redirect message, and
router1 sends the same packet again to router3. That knows that girl
can be reached imediately and forwards the packet. Both, router1 and
router3, will record this route in the routing cache, so the next
packet would never cross cop. In this configuration, no mangling of
the packet by cop would be necessary. That there is absolutely no need
to change the original packet in any way, specially not the source or
destination address.

Now in my special situation, router1 and router3 are only the routing
tables of "boy" and "girl" respectively, there is no additional
hardware:

	boy -> cop -> girl

Initially, "boy" sends a packet which goes to cop. The source address
is "boy", but the destination address is the external IP of cop
(that's the information boy's ICQ has about girl). At cop this
destination address (DNAT) is changed. As this is done in a PREROUTING
chain, the routing system isn't aware of this. It will just see a
packet going through a route which might be one hop shorter. So it
sends an ICMP redirect to boy. This message was logged by my netfilter
script:

Nov 2 21:56:12 cop kernel: IN= OUT=eth1 SRC=cop DST=boy LEN=88
TOS=0x00 PREC=0xC0 TTL=255 ID=5757 PROTO=ICMP TYPE=5 CODE=1
GATEWAY=girl [SRC=boy DST=girl LEN=60 TOS =0x00 PREC=0x00 TTL=64
ID=1404 DF PROTO=TCP SPT=1119 DPT=40150 WINDOW=5840 RES=0 x00 SYN
URGP=0 ]

To make reading easier, I've changed the addresses by names. "cop" is
here my firewall. eth1 of course is the internal interface (because
this ICMP packet originates on the firewall and goes back to boy,
which is on the local network). It is interesting, that "girl" isn't
mentioned as the new destination, but as the better *gateway* and that
the destination address given by boy (the external IP) is already
mangled). Within brackets, we get information about the packet which
should be retransmitted: the source continues being "boy", but now
it's "girl", the DNAT'ted address, rather than what "boy" used
initially. We also see the SYNC flag set, so this is the first packet
of a connection to be opened.

This request is honored by "boy's" kernel (and not by the ICQ client):
It will send the same packet (a SYNC) with the source address of "boy"
and the destination address to "girl" rathern than the external IP,
before DNAT on cop. Actually, it doesn't need the gateway address
which cop informed of, as it knows by itself that "girl" can be
reached directly, but probably it will use that anyway.

When this packet reaches "girl", her ICQ client, which is listening on
that port, will accept this packet, as it *is* for her, and it comes
on the right port. There is no reason why she shouldn't accept it. So
she replies with a SYNC-ACK packet. The source address is of course
"girl", and the destination, "boy". The local routing table tells,
that "boy" can be reached directly, so "cop" wouldn't be involved.

Finally, boy's routing table receives the SYN-ACK. The source address
however is now from "girl" and not the external IP address where the
SYNC packet was sent to. The kernel sends a RST packet as it has
nobody waiting for an answer from that address, canceling the request,
and the ICQ client continues waiting for a SYNC-ACK from the source
where it sent the SYNC to. As it's the kernel who sends this RST, the
ICQ client never receives any answer.

After a total of 6 attempts, starting with a timeout of 3 seconds and
doubling that each time, i.e., after exactly 3 minutes and 9 seconds,
the ICQ client gives up for not receiving any response, which is
perfectly true. But as it has the masqueraded address of "girl" as
well as the "real" address, for some (this?) reason it tries a 7th
time using that, which succeeds.

First of all, I would like to ask if this explanation is correct. The
second question is why I can't I disable ICMP redirects to be sent by
cop, and the third: What would be the solution to this problem? I know
that it will work after 3 minutes and 9 seconds, but most user's wont
wait that long. Also, the behaviour above is for licq; I didn't test
wether windows or mac implementations would do the same or give up
earlier.

Thanks in advance.

--
Christoph Simon
datageo@xxxxxxxxxxxx
---
^X^C
q
quit
:q
^C
end
x
exit
ZZ
^D
?
help
NO CARRIER
.



[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux