RE: Load balancing using connmark

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Let me explain why the marking is done in POSTROUTING.

The first packet of any connection get routed by the multipath routing
entry. This happens AFTER PREROUTING, as you know. And this is what we
want, letting the kernel decide based on the weights. (some people do
think that we shouldn't let multipath decide routing, but thatz a
different story).

So where can this packet be marked? Obviously in POSTROUTING (so that
local pkts also can be caught). We mark it and save it.(connmark).The
mark is decoded by the chosen interface. (eg:-o WAN1 --set mark 1,-o
WAN2 --set-mark 2)

In PREROUTING, there is a restore-mark. You see

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark.

If this packet belong to a connection that has already sent a
packet,this will restore the mark set in POSTROUTING. Then it will be
routed by the corresponding routing table.(wan1 table lookup mark1 and
wan2 table lookup mark2)
If it is a new pkt, it will be routed by multipath routing
statement,since no mark exists.

-----Original Message-----
From: lartc-bounces@xxxxxxxxxxxxxxx
[mailto:lartc-bounces@xxxxxxxxxxxxxxx] On Behalf Of Peter Rabbitson
Sent: Thursday, May 10, 2007 6:51 PM
Cc: lartc@xxxxxxxxxxxxxxx
Subject: Re:  Load balancing using connmark

Salim S I wrote:
> Francis Brosnan Blazquez wrote:
> 
>> Hi,
> 
>> 
> 
>> I've been implementing a load balancing solution using CONNMARK,
based
> 
>> on solution described by Luciano Ruete at [1]. Gracias por el post y
por
> 
>> apuntar en la dirección correcta Luciano!
> 
>> 
> 
>> Once implemented, I've found that due to some reason packets aren't
> 
>> properly marked (or improperly remarked) and sent out using the wrong
> 
>> interface. 
> 
>> 
> 
>> <snip>
> 
>> 
> 
>> iptables -t mangle -A POSTROUTING -m mark  --mark ! 0 -j ACCEPT 
> 
>> iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 0x1
> 
>> iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 0x2
> 
>> iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
> 
>  
> 
> This is wrong. POSTROUTING is exactly what is is _POST_ routing. By
the
> 
> time you do your marks and stuff the kernel has _already_ assigned a
> 
> packet to an interface, and you can not alter this anymore.
> 
>  
> 
>> After a bit of testing with the second solution, it seems to behave
> 
>> better, doing all marking job at the PREROUTING and OUTPUT.
> 
>  
> 
> This is flawed too. OUTPUT suffers from the very same problem as
> 
> POSTROUTING - by the time the packets hit the NF stack the process has
> 
> already bound itself to an interface, which you can not change
anymore.
> 
>  
> 
> Peter
> 
>  
> 
> Disagree with Peter. The marking in postrouting table is CONNMARK.
This
> is for marking the connection, which has already had a route decided
for
> it, so that all packets of the connection passes through this
interface.
> This marking is done for packets with NEW state, see the check for
> mark==0 in the prev. line. The restore mark in PREROUTING will restore
> the connmark and route the subsequent packets.
> 
> This approach will work, but you need some sort of stateful-ness in
> netfilter.
> 

Connmark is exactly the statefullness you are talking about. The problem
is that the marks by themselves do not mean anything. You mark packets
and expect iproute to classify the packet in the correct routing table
etc. CONNMARK is invisible to iproute - this is why you have only
--save-mark and --restore-mark, and the rest of the rules deal with real
MARKs.

Further you (and the OP) seem to be confused by a mix of routing tasks.
In the case of _forwarded_ traffic, you need to make sure that all
packets within a connection leave to WAN over the same interface, and
are SNATed to the same ip, so that they will come bak the same
interface. The SNATting is trivial (as it can be done in POSTROUTING
only), but you need to set all marks before the routing takes place
(which is anywhere _but_ POSTROUTING). You might mark the connection
with the proper CONNMARK. and subsequent packets might get routed
correctly, but the _first_ packet (the one that you use to set the mark)
is already assigned to an interface, and there is nothing you can do
about it.

In the case of _local_ traffic - it becomes even trickier. The problem
is that when sockets are created they already have a source IP (the
kernel determines that by looking at the default routing table, your
marks do not exist yet). So since you can not alter the socket binding,
the only way to make it leave on a different interface is by treating it
as a forwarded connection and performing NAT on it. It is arguable if
NATting locally originating connections is a good idea, but it can be
done in OUTPUT just like it is done for forwarder connections in
PREROUTING.

I hope this clarifies things a bit, feel free to point out any
inconsistencies you may find.

Peter

_______________________________________________
LARTC mailing list
LARTC@xxxxxxxxxxxxxxx
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


_______________________________________________
LARTC mailing list
LARTC@xxxxxxxxxxxxxxx
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux