Thank you Andy, really I have figure out this issue.....go ahead I forgotten include my CONNMARK RESTORE rule at the last message.... so follow my whole script. there are two PREROUTING rule for new connections that they should do the load-balance but I wonder when the socond rule CONNMARK2 coming use it ?? I think never because data flows always going to into at the first rule ( CONNMARK1 ). so the second rule never is using! I think the statistc module could do that rotate at the use rules...but doesn't work! eth1: LAN Interface eth0: WAN1 eth2: WAN2 #!/bin/bash # flush all iptables entries iptables -t filter -F iptables -t filter -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X iptables -t filter -P INPUT ACCEPT iptables -t filter -P OUTPUT ACCEPT iptables -t filter -P FORWARD ACCEPT iptables -t mangle -N CONNMARK1 iptables -t mangle -A CONNMARK1 -j MARK --set-mark 1 iptables -t mangle -A CONNMARK1 -j CONNMARK --save-mark iptables -t mangle -N CONNMARK2 iptables -t mangle -A CONNMARK2 -j MARK --set-mark 2 iptables -t mangle -A CONNMARK2 -j CONNMARK --save-mark iptables -t mangle -N RESTOREMARK iptables -t mangle -A RESTOREMARK -j CONNMARK --restore-mark iptables -t nat -N SNAT1 iptables -t nat -A SNAT1 -j SNAT --to-source 192.168.217.254 iptables -t nat -N SNAT2 iptables -t nat -A SNAT2 -j SNAT --to-source 192.168.216.254 iptables -t mangle -A PREROUTING -i eth1 -s 0/0 -d 0/0 -m state --state ESTABLISHED,RELATED -j RESTOREMARK iptables -t mangle -A PREROUTING -p tcp -m state --state NEW -m statistic --mode nth --every 2 --packet 0 -j CONNMARK1 iptables -t mangle -A PREROUTING -p tcp -m state --state NEW -m statistic --mode nth --every 2 --packet 1 -j CONNMARK2 iptables -t nat -A POSTROUTING -o eth2 -j SNAT1 iptables -t nat -A POSTROUTING -o eth0 -j SNAT2 ip route add 192.168.217.0 via 192.168.217.1 table oitelecom ip route add 192.168.216.0 via 192.168.216.1 table gvttelecom ip route add default via 192.168.217.1 table oitelecom ip route add default via 192.168.216.1 table gvttelecom ip rule del from 192.168.217.254 table oitelecom ip rule add from 192.168.217.254 table oitelecom ip rule del fwmark 1 table oitelecom ip rule del fwmark 2 table gvttelecom ip rule add fwmark 1 table oitelecom ip rule add fwmark 2 table gvttelecom ip route flush cache thanks....any tips is welcome. Em 25 de fevereiro de 2012 08:53, Andrew Beverley <andy@xxxxxxxxxxx> escreveu: >> >> If it's not working, there must be another problem. Please show the >> >> output of "ip rule show", "ip ro" and "ip ro show table <table>" for >> >> each of your tables where <table> is the name of the tables. > >> >> [root@mtjve sbin]# ip ro >> 192.168.215.0/24 dev eth1 proto kernel scope link src 192.168.215.1 >> 192.168.217.0/24 dev eth2 proto kernel scope link src 192.168.217.254 >> 192.168.216.0/24 dev eth0 proto kernel scope link src 192.168.216.254 >> default via 192.168.216.1 dev eth0 >> >> >> [root@mtjve sbin]# ip rule show >> 0: from all lookup 255 >> 32763: from all fwmark 0x2 lookup gvttelecom >> 32764: from all fwmark 0x1 lookup oitelecom >> 32765: from 192.168.217.254 lookup oitelecom >> 32766: from all lookup main >> 32767: from all lookup default >> >> [root@mtjve sbin]# ip ro show table oitelecom >> 192.168.217.0 via 192.168.217.1 dev eth2 >> 192.168.217.254 via 192.168.217.1 dev eth2 >> default via 192.168.217.1 dev eth2 >> >> [root@mtjve sbin]# ip ro show table gvttelecom >> 192.168.216.0 via 192.168.216.1 dev eth0 >> default via 192.168.216.1 dev eth0 >> >> >> I think here is the issue! there are two PREROUTING rule that they >> should do the load-balance but I wonder when the socond rule ( >> CONNMARK2 ) coming use it ?? I think never because data flows always >> going to into at the first rule ( CONNMARK1 ). so...the second rule >> never is using! today I have done a test download from three diferente >> sites and the load-balance it wasn't work. >> >> >> eth1: is my lan interface >> >> iptables -t mangle -A PREROUTING -i eth1 -s 0/0 -d 0/0 -m state >> --state NEW -m statistic --mode nth --every 2 --packet 0 -j CONNMARK1 >> iptables -t mangle -A PREROUTING -i eth1 -s 0/0 -d 0/0 -m state >> --state NEW -m statistic --mode nth --every 2 --packet 1 -j CONNMARK2 > > I can't see any restoration of the marks. You are marking the connection > when it is NEW, but "ip rule fwmark" will match on the packet's mark, so > you need to copy the connection mark to the packet mark with "-j > CONNMARK --restore-mark". > > Andy > > -- To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html