RE: Non-traditional Failover Query

Linux Advanced Routing and Traffic Control

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This seems like a band-aid front end solution to a back-end Session
tracking problem unless I am miss-reading your request. 

> 1) Any given process is running on TWO machines at the
> same time. If a process or machine fails, then a new
> backup is started on the third machine. There is thus
> a rotation around the nodes.

Linux Heartbeat

> 2) Packets destined for any of the machines should be
> received by ALL of the machines.

IPT#1
I am not sure if there's already an iptables module that does this, but
you can expect something like that to go into there, since that's where
most really out there networking stuff goes (connection tracking is a
different problem)

> 3) If the primary process is on A, then replies from B
> and C should still be generated, but be transparently
> dropped. Likewise, if the primary process is on B or
> C.

IPT #2
This again would have to be a very elaborate iptables module that also
interacts with a userspace app which keeps track of which backend
process is really alive, and which ones are just spinning.

> Uhhh... because this approach is seriously abusing the
> entire networking stack. We essentially have all three
> machines running with an identical MAC and IP address
> visible to the outside. The "distinct" address is
> purely internal.

Iptables -j SNAT --help

> What this requires is a way of tricking all three
> machines into believing that they are the sole
> recipients. This keeps the stacks in a uniform state,
> which means we can fail-over the connections without
> having to either checkpoint or copy stateful
> information, both of which get ugly when you start
> talking about lots of information.

The iptables module described at IPT #1

> This leads to the second network-related problem. If
> you have two identical machines starting from
> identical states, and processing identical streams,
> then they should end up in identical states - ie:
> crashed.

This can't be guaranteed, but you would hope that two identical machines
doing exactly the same thing would be more or less the same, but it also
depends on external hosts servicing the two machines identically and
within a reasonable time differential.

> This is easily fixed. If A is the machine you are
> starting all the processes on, and B is your "mirror",
> then C needs to take up the excess load.
> 
> In other words, A+C is a cluster, and B+C is a second
> cluster. Processes migrated to C from A or B aren't
> mirrored. (This is akin to RAID 5's partial backup.)

You kind of lost me here. 

> B) From the HA perspective, there is one primary
> machine, one mirror machine and one spare

You are trying to mix two HA paradigms together, Hot and Cold Standby. I
doubt this is possible with most current HA software, but prove me
wrong. I'd like to know!

> C) From the load-balance perspective, there are two
> overlapping clusters.

This isn't quite true since the load-balancer does not split the load,
it just discriminates as to who's response is relevant.

(PS: When the primary goes belly up, how do you plan on doing a takeover
AFTER the secondary has sent out packets that are effectively dropped?
It seems that TCP retries would all be sent and dropped and the session
would timeout)

Having two simultaneously servicing events seems a waste. I would say
that statefull hot-standby is you're best bet. Make it a 3 way hot
standby if the HA software supports it. This of course means that the
server software you're using to has to be HA aware.

> Using Linux' advanced networking, BPF and netfilter
> layers, connection mirroring for HA/load-balancing
> purposes should be straight-forward. Much more so than
> using the "wedge" concept proposed by MIT. Because you
> need send nothing more than flags between the
> machines, it should also be less expensive on the LAN
> and processor.

Trust me, THIS IS NOT EASY!
_______________________________________________
LARTC mailing list / LARTC@xxxxxxxxxxxxxxx
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/


[Index of Archives]     [LARTC Home Page]     [Netfilter]     [Netfilter Development]     [Network Development]     [Bugtraq]     [GCC Help]     [Yosemite News]     [Linux Kernel]     [Fedora Users]
  Powered by Linux