Hello, On Sun, 28 Oct 2001, Christoph Simon wrote: > > OTOH, every usage of multipart routes should take care whether > > the different gateways allow traffic with the used source addresses > > to pass through all paths. The key in using multipath routes is teaching > > the NAT code to correctly use the right source addresses when sending > > to each path from the multipath routes, especially when they provide > > you with distinct public IP ranges. If someone thinks that this should > > work by magic, without coding such feature, he is in wrong direction. > > When I saw David's message, I didn't even think about this (as in my > case it brakes much earlier). But as AIM is usually very low traffic, > I guess my suggestion to direct it only to one interface was not that > wrong. Unless this is a major AIM gateway, I don't think any coding > (and getting incompatible with the rest of linux kernels) would not be > worth the efford. Agreed. I don't know the David's setup requirements, whether the multipath route is suitable (see below why). Multipath is suitable for any NAT setup, of course, if NAT can use it (after patching). > > Chris, did you asked the Netfilter gurus for such features? [I > > hope you received my email from October 20] > > No, unfortunately, I didn't receive any answer. This is odd, as I'm > using email quite frequently, also receiving answers, which makes my > think that my return address should be correct. But knowing my ISP and > Mr. Murphies Law, anything specially interesting has a great > probability to fail. I sent it privately again. Did you received it? > > > if I did understand it well, the usage of the `equalize' argument to > > > ip makes the selection of a particular interface packet based, while > > > the omission should make it session base. I have tried all > > > > I'm not sure the equalize patch is into the mainstream kernel. > > I'm using only standard kernels, currently 2.4.9 and 2.4.12-acX. As > I've tried to use equalize and didn't get an error message, so it > should be working. But I have done hundrets of tests, so it is easy > that I oversaw or don't recall something. No, IIRC, it is available only in user space and as a separate patch. From ip-cref: "only works if the kernel is patched" but for multipath route to universe this shouldn't make any sense. Equalize is useful for routes with small number of targets (I never used it), i.e. for directly connected networks. > > IMO, Netfilter should be teached to use destination cache and > > multipath routes. I already have it completely working for Linux 2.2 > > masquerade (5 patches) but for 2.4 I'm at 60% (3 of 5 patches): > > I'm forced to use 2.4 kernels for not networking, hardware dependant > reasons. So it's a pitty that the 2.4 patch isn't finished yet as i'd > loved to try it, but if you want, I would be gladly willing to help > testing. What are the chances that this will be introduced to the main > stream kernel sources? When I finish it I will notify you, I'll be happy if you test it :) About the mainstream kernels, for now I'm not sure. I have so much pending stuff for the kernel but the kernel is still in "stable" state. And may be some of the stuff does not sound very good to the net gurus :) > I do have a problem understanding this: Let me take a concrete > situation as an example: I have a firewall with three interfaces, one > to the LAN and two to the DSL modems. There is a server behind running > squid. All clients on the LAN are configured to use squid for Internet > access; no forwarding is done beside for the server itself. If there > is a destination cache, all connections to the same server would go > through the same interface. Is this right? There ares some very The destination cache usage handles the right routing once the connection has source address bound. Even if the multipart route mislead the socket or the conntracking they can reroute the packet to the right gateway according to the bound source address. Of course, this can sound ugly for some gurus but rerouting is needed, you can see it for the sockets. As result, in some cases, the first input route does not select the right gateway (you know, bad design, the routing is not managed from the conntracking). The main problem is that the routing occurs before we see what is the bound source address for the NAT-ed connection. > important sites which are visited (according to squid reports) by up > to 80% of clients and time (not every day and not always the same, but > it happens frequently). Those sites also use load balancing (the same > address resolves to many IPs). If I understood right, one session is > the set of packets which are exchanged between a connect and a close > statement. From the HTTP point of view (including cookies and IP Yes, your NAT always selects one gateway, may be? > tracking) however, a session can be from the first access until the > user explicitly closing of the session, or a timeout of inactivity > happens. I have observed that within an HTTP session to that site, the > destination IP does change (but not within a session in the sense of > connect-close). In this very situation, how can the destination cache > tell that the following connect (i.e., a new tcp session) is part of > an already existing HTTP session or that the proxy is doing this in > name of a different user or workstation on the LAN? Unless I missed Once tha path (src->dst) is cached it is used from all connections that hit this cache entry (same src and dst). About the HTTP sessions selecting different source address, this is really a problem where connection persistence is required into the remote servers. But may be this is a task for the proxy server always to use specific source address when serving a client. This is only a beta idea :) The destination cache entry has only one purpose: to reroute the packet to its gateway according to the bound source address (the masquerade address). Of course, the best implementation can be discussed but this is what I see for the current netfilter infrastructure. Such details should be discussed in netfilter mail list. I know, that there are solutions using fwmarking and they work but you can't use the fwmarks for other purposes. In any case, when multipart routes are used you should be ready to send packets with specific source IPs through any path from the route. But you can see in dgd-usage.txt file in my page that the multipath is used in different way: to select different gateways for the masqueraded traffic in loadbalancing way but after that point alternative routes are used because in the example each gateway can pass only traffic from one public IP rangeand only the alternative routes can use different prefsrc. The number of ISPs that provide you with two or more lines covering same IP range is small. Most of the users using multipath routes are using it incorrectly. The current model guarantees the feature works only when these gateways are reachable through different devices. The most of the setups I saw are using multipath route NAT-ing to two ADSL lines reachable through one external device and they expect from such route to work. From my patches you can see that this is possible only when the gateway IP address is used as a route lookup key. > something really important, I would suppose that those 80% would have > to go through the same interface, and that HTTP sessions will break > each time a session is opened on a different destination IP for the > same DNS name. I also can't imagine that it would help to store in the Yes, when the route cache entry is flushed (5mins by default) some strange things can happen. This is the reason the NAT connections should keep a pointer to a destination cache entry and to monitor its state and to compare it to the route attached to each packet with the main goal to not send the packet to wrong gateway that does not relay packets with such source IP address. As result, you have to see that packets with specific src and dst IP should go to the right gateway(s). > destination cache the network addresses rather than destination > addresses, as the IPs corresponding to those sites are not continuous, > and there are others in the middle which exist and would go to > different services and/or sites. OTOH, even if not `load balancing', > at least it would avoid breaking complex HTTP sessions. For my > understanding, the solution would be an extension to the RELATED > attribute of a packet, similar as for FTP where a secondary connection > is built, although it must be much more difficult. You are asking some Squid and HTTP questions but the stuff I'm talking about is Layer 7 independent. May be when I reread your email I'll answer again but you first have to understand from my docs what is possible and what is not, what is extended with these patches. To summarize the features the patches give you: - you should be able to define by using routes what IP range (source addresses) through which gateway(s) can reach the universe. In most of the cases it is one gateway per range even if you have many uplink lines (with distinct ranges in such case). - the NAT routers can use multipath feature for the packets from private networks (NAT-ed) to universe by using distinct gateways. By this way the masqueraded traffic can utilize all uplink lines. The traffic already bound to specific IP ranges can go only through its gateway(s). The traffic to your servers (published in DNS server) can reach your through many lines but you need routes to return it back through the allowed gateways. Anything else is a job for your DNS server (failover, etc). As result, the multipath route is only for the NAT-ed networks, i.e. ip rule add prio 100 from 192.168.0.0/16 table 100 ip route add default table 100 src ANYSRC \ nexthop via GW1 dev DEV1 \ nexthop via GW2 dev DEV2 Then alternative routes are used to the same gateways: ip rule add prio 200 from 0/0 table 200 ip route append default table 200 via GW1 dev DEV1 src SRC1 ip route append default table 200 via GW2 dev DEV2 src SRC2 The traffic with src IP from public networks is forwarded without SNAT-ing to the allowed gateway - no multipath routing unless allowed (I already mentioned that most of the setups use only one gateway per public range). Following such rules you can build only with proper routes a good setup that will allow balancing your uplink lines for the outgoing traffic. You can see example for such routes in dgd-usage.txt. Of course, the NAT should be helpful in this direction to always following the actual routes' state. > Christoph Simon > datageo@xxxxxxxxxxxx Regards -- Julian Anastasov <ja@xxxxxx>
