Re: Load balancing multiple ADSL connections

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi there,
I was thinking, if I rent some rack space in a server room with a 10 MBit connection, could I have my Linux gateway set up with 6 NIC cards, 1 LAN and 5xADSL with VPN to my server? The server could feed data across all 5 lines evenly and possibly meter the data based on the ping times. That way I would have a lot more bandwidth for the dollar. 7 MBit down and 3.5MBit up for $275 a month, assuming the rack space costs $100/mo

However I have been searching around and have not found any answer that the standard Linux kernel / OpenVPN can do something like this. Particularly with the ADSL you don't get a fixed bandwidth so you need to dynamically adjust your throttling based on what you get.
If I were to set this up, I'd first create 5 (OpenVPN) tunnels between the local linux gateway/server, and the remote linux server. Ensuring each tunnel is forced over a different ADSL link could be fun. You could set up 5 local routing tables, each with only one default route, one table for each ADSL link, then use the port number to select the appropriate routing table, and therefore ADSL link. My personal preference of doing this is marking the packets in the iptables mangle table, and use the "ip rule" command to select the routing table based on firewall mark. For example (with 2 uplinks):

ip route add default via 192.168.1.1 table 101
ip route add default via 192.168.2.1 table 102
iptables -t mangle -A OUTPUT -d ${remoteServerIP} -p udp --dport ${OpenVPNTunnel1Port} -j MARK --set-mark 201 iptables -t mangle -A OUTPUT -d ${remoteServerIP} -p udp --dport ${OpenVPNTunnel2Port} -j MARK --set-mark 202
ip rule add fwmark 201 pref 11101 table 101
ip rule add fwmark 202 pref 11102 table 102

(I assume there's an ADSL router per link and the public IPs are not on the local box, otherwise I think it'd get messy handling multiple ppp client sessions on the same box, especially if you don't have static IPs on all the links. Well, there's always running each ppp session in a dedicated virtual machine and set up your virtual networks appropriately to tie everything together, but let's not complicate things more than they already are.)

Now that the tunnels are in place, you need to somehow distribute traffic over them. I'd install quagga on both the local and remote servers, and run OSPF between them over all 5 tunnels, with the remote server pushing the default route back to the local one. Hopefully this should install 5 default routes on the local server each pointing over a different tunnel, and the kernel should load balance between them now (you may need to tweak path costs and such in quagga). Also, the same works the other way - you advertise your local subnet to the remote server over all 5 tunnels with quagga, and the remote server should load balance the traffic back to you.

I'm no quagga expert though (only using it to run RIP over OpenVPN at the moment since I don't need the added benefits OSPF provides), so would need to read the manual to work out the config to set this up.

But by running a routing protocol, especially OSPF, in this scenario would mean that if some ADSL links go down and up temporarily, or you add/remove links permanently, OSPF will take care of modifying the available routes on either server transparently for you.

I see load balancing routers but they are connection based - i.e. any one file transfer would still operate at 1.5 MBit, you can just have multiple at the same time. This causes problems too, in my office we have load balancing proxy servers and since your IP address changes all the time many secure websites do not work.
I think even with this set-up, each connection flow would only go over one tunnel, I don't think linux routes on a per packet basis? Nor would I want it to in most cases, you'd get more issues with packets arriving in the wrong order when sending data if you do this, which will hurt tcp throughput.

However, since you are routing everything via your remote server in this case, which will almost certainly need to be NATting traffic as it finally goes out via it's single uplink, you won't have the issue which you see with your office proxy servers which are basically load balancing straight onto the Internet via multiple uplinks hence the multiple IP issue you see.

As such, this may partially negate the fact that you're still limited to 1 ADSL-link worth of bandwidth per flow, since you'll probably have several flows going on simultaneously, especially for web browsing, so I'd still expect an overall increase in performance for such types of tasks.

I have a home office so I need to make sure that the ping times stay low. This is pretty easy with the T1 because of the fixed bandwidth.
Since you're planning on sending all data via a remote server first, you're already building in latency to your solution and so increasing ping times. Short of testing and knowing the exact setup, it's hard to say exactly how much added latency you'll see.

But this is a fairly complex design you're trying to come up with here, and there are a few details missing to fully configure it, such as how the ADSL links are presented to the local server, so it would take quite some fiddling to get it all working. Good luck if you try to go through with it though, I don't think it's impossible to configure. 5 links does feel a bit excessive to me, but you know your own requirements.


-- Muntasir

--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux