Hello,
Admin a écrit :
We have a load balanced set of web servers (behind an alteon AD3) which all
work fine - except when the real servers (the webserver instances on the load
balanced machines) try to access the virtual ip (vip) on the alteon.
[...]
So the alteon substitutes the virtual server ip with the real server one and
back again for the load balancing to work.
This looks like good old DNAT.
Now the problem, we need the real servers to be able to access the service
provided on the VIP also. The ports on the alteon are configured properly
(client and server enable and so on) but the problem seems to be a routing
one.
Here's the flow of traffic when the real server tries to access the vip
# client request
(client) -> (vip) - sip=CC, dip=BB
(vip) -> (real_server) - sip=CC, dip=CC
# return traffic (real server on CC responds to client on CC - oops that's me)
(real_server) -- sip=CC, dip=CC --> straight back to the real server, no dice.
Now this is a common problem
Yes, rather common when the client and the DNAT target are inside the
same LAN.
If you can tolerate that each server send the request to itself, it's
easy. Something like this will locally redirect connections to the VIP
to the server's own address :
iptables -t nat -A OUTPUT -d <vip> -j DNAT --to <own_ip>
Else, if the alteon can do SNAT (source NAT), this is the usual
workaround to this problem. You can masquerade the servers subnet with
VIP. Written in iptables style :
iptables -t nat -j POSTROUTING -s <servers_subnet> -d <servers_subnet> \
-j SNAT --to <vip>
But this method hides the original source server address on the
destination server. If you need this information, you can do a static
SNAT 1:1 mapping between each server address and a unique address chosen
into an unused subnet which is distinct from the servers subnet. You
won't have the real address in the logs but at least an address that
uniquely identifies the source server. Again in iptables style :
iptables -t nat -j POSTROUTING -s <server1> -d <servers_subnet> \
-j SNAT --to <alias1>
iptables -t nat -j POSTROUTING -s <server2> -d <servers_subnet> \
-j SNAT --to <alias2>
and so on. Or, more concisely :
iptables -t nat -j POSTROUTING -s <servers_subnet> -d <servers_subnet> \
-j NETMAP --to <alias_subnet>
I have been trying to find a way to mangle the packets with iptables (on the
real servers) to make this work.
The obvious (well...kinda obvious) way is to change the sip (to something
outside the local network) on the incoming packet somewhere in the
PREROUTING or INPUT phase, then modify the dip in the response packet in the
POSTROUTING phase back to that of the real server.
Unfortunately this is pretty much the reverse of what DNAT and SNAT are used
for.
We want SNAT on the PREROUTING or INPUT phase and DNAT in the POSTROUTING (or
possibly OUTPUT, but I don't think so) one.
The reason why this is not possible is that routing would probably be
disrupted.
Or perhaps - is there any way to force traffic from one IP to the same IP to
go through a remote router?
That kind of traffic is for a local delivery, and the 'local' routing
table has precedence over all other routing tables. Anyway why would
there be traffic from one IP to the same IP ?
Perhaps mark traffic from the real server to the
vip then act on that mark when the alteon directs it back?
A mark is a local information, and is lost as soon as the packet leaves
the box. But you can alter return traffic this way (what you actually
need). Remember, the wrong part is that replies go back directly to the
client instead of going through the alteon. So iptables will first mark
the packets we're interested in, assuming your servers listen only on
TCP port 80 :
iptables -t mangle -A OUTPUT -d <servers_subnet> -p tcp --sport 80 \
-m state --state ESTABLISHED -j MARK --set-mark <mark_value>
A routing rule will direct the marked packets to an alternate table :
ip rule add fwmark <mark_value> lookup <table_id>
ip route add default via <vip> table <table_id>
One alternative, although I personnaly find it very ugly : by-pass
normal routing with the (experimental) ROUTE target, and force routing
of the servers subnet via the alteon as a gateway.
