Julian, I want to thank you for your help on this one. After pouring over books and every google reference I could find, I think I finally understand what ICMP_REDIRECT is all about. I'm still not clear about RP_FILTER but getting better. I gathered some more data on the problem. (The problem was two routers connected in parallel to the inside and outside networks. The Linux router is everyone's default gateway and has a route for one subnet to the other router. The other router then VPNs it across the Internet to a remote branch office.) The deal with ICMP_REDIRECT is, if you have more than one router and one router gets a packet bound for somewhere, and that router knows there is a better first hop, that router will send an ICMP_REDIRECT message back to the original sender and the original sender is supposed to update its routing tables. That is what's supposed to happen. The Linux router is running Red Hat Linux 7.1, which is kernel 2.4-2. I found out that ICMP Echo Reply packets weren't being handled right and I took a bunch of tcpdump output showing details. Based on Julian's advice with ICMP_REDIRECT, I got back to the customer site a few days ago and took advantage of an internal DNS server named igor. If I did telnet igor 53 from across the VPN and then sent some illegal junk to that DNS server, then I could ping igor across the network. And sure enough, ICMP_REDIRECT set up the route in igor. But if I removed the route in igor and tried to ping igor, igor would reply, but the original "pinger" never saw it and gor would never get an updated route from the Linux router. tcpdump on the Linux router showed me that igor was replying, but the router never forwarded the reply anywhere. The Linux router did handle my TCP port 53 stuff, but not ICMP. So the problem had to be some kind of kernel bug around ICMP redirects. I got some more time to work the problem today and finally reproduced the problem at my place. Then I setup a system with Red Hat 7.2, which uses kernel 2.4-7. Still old, but not as old! I set this sytem up as a router in parallel with my other Linux router and ran the same tests again. This time, everything did what it was supposed to do. So I think the whole thing was a bug with the 2.4-2 kernel. So for people running the 2.4-2 kernel with routing turned on and in parallel with another router, one workaround is to send any kind of TCP packet first to the other side, which will make ICMP_REDIRECT do its thing. A better workaround - as Julian pointed out: echo 1 > /proc/sys/net/ipv4/conf/eth0/send_redirects echo 1 > /proc/sys/net/ipv4/conf/eth1/send_redirects An even better fix - upgrade from Red Hat 7.1 to 7.2. And maybe the best fix is to just bite the bullet and upgrade to the latest stable kernel. That's the next challenge I'm going to work on! thanks - Greg -----Original Message----- From: Julian Anastasov [mailto:ja@ssi.bg] Sent: Thursday, February 28, 2002 4:49 PM To: Greg Scott Cc: LARTC@mailman.ds9a.nl; Patsy Rossow (E-mail) Subject: RE: [LARTC] Re: Routing that doesn't route Hello, On Thu, 28 Feb 2002, Greg Scott wrote: > > What is the value in /proc/sys/net/ipv4/conf/eth1/send_redirects ? > > [root@csfampls-fw /]# more /proc/sys/net/ipv4/conf/eth1/send_redirects > 1 This is one of the values you can try to alter. Try to set both all/send_redirects and eth1/send_redirects to 0. > > On each Linux try to check with > > ip route get from <SADDR> to <DADDR> iif <IN_DEVICE> > > [root@csfampls-fw /]# > [root@csfampls-fw /]# /sbin/ip route get from 172.16.16.2 to 172.16.0.252 > iif eth1 > 172.16.0.252 from 172.16.16.2 via 172.16.16.3 dev eth1 src 172.16.16.1 > cache <src-direct,redirect> mtu 1500 iif eth1 OK, traffic is not dropped from rp_filter. But this box should send ICMP redirects (note the "redirect" flag), i.e. it redirects traffic from 16.2 to 0.252 through 16.3. If you try with {all,eth1}/send_redirects=0 the traffic will be silently accepted and then forwarded to 16.3. > [root@csfampls-fw /]# > [root@csfampls-fw /]# /sbin/ip route show > aaa.bbb.228.32/27 dev eth0 proto kernel scope link src aaa.bbb.228.33 > ccc.ddd.200.64/27 via aaa.bbb.228.34 dev eth0 > 172.16.16.0/20 dev eth1 proto kernel scope link src 172.16.16.1 you are trying alternative routes which work only by using my patches, the following can't work in plain kernel: > 172.16.0.0/20 via 172.16.16.3 dev eth1 > 172.16.0.0/20 via 172.16.16.151 dev eth1 Note also that the above routes should have a preferred source IP. Avoid using the "route" tool in advanced routing setups. Try with "ip": ip route add 172.16.0.0/20 via 172.16.16.3 dev eth1 src 172.16.16.1 > 127.0.0.0/8 dev lo scope link same here, make sure all your routes have right preferred source IP: > default via aaa.bbb.228.62 dev eth0 In the other case you risk the kernel to select wrong IP address for your originating or masqueraded traffic. > [root@csfampls-fw /]# > > > One of the problems > > could be the conf/DEV/rp_filter settings but I don't see why it should > > drop the packets. > > How do I look at these and what are they? /proc/sys/net/ipv4/conf/{all,eth1}/rp_filter but it seems it's not a problem. Of course, try to set all them to 0 for the test. > Every setting I can think of so far looks good. What am I > missing???? Or is there a bug? Note, this is happening in two completely > unrelated places. But it could be that I set up both places with the same > mistake. If clearing all send_redirects and rp_filter flags to 0 and using correct preferred source IP addresses does not help then you hit a kernel bug. Try with recent kernel. > thanks > > - Greg Scott Regards -- Julian Anastasov <ja@ssi.bg> _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
