On 07/26/2012 02:50 PM, Steve Campbell wrote:
>
> On 7/26/2012 1:52 PM, Digimer wrote:
>> On 07/26/2012 01:38 PM, Steve Campbell wrote:
>>>
>>> On 7/26/2012 12:01 PM, Digimer wrote:
>>>> On 07/26/2012 08:05 AM, Steve Campbell wrote:
>>>>> I'm creating a firewall HA cluster. The proof of concept for the basic
>>>>> firewall cluster is OK. I can bring up the cluster, start the iptables
>>>>> firewall, and move all of this with no problem. I'm using Conga to do
>>>>> all of this configuration on Centos 6.3 servers.
>>>>>
>>>>> To extend the "HA" part of this, I'd like to use bonded channels
>>>>> instead
>>>>> of plain old NICs. The firewall uses the "IP address" service for the
>>>>> outside firewall IP addresses. Each server behind the firewall is
>>>>> NATted
>>>>> to one of these external IPs on the firewall's external interface.
>>>>>
>>>>> I'm not seeing how I can use bonded channels anywhere for these "IP
>>>>> address" services. Part of the problem is that Conga will "guess" at
>>>>> which interface to place the ip address service upon. In the case of
>>>>> bonded channels, I don't think Conga is even aware of the "bondx"
>>>>> interface, and Conga only uses interfaces like eth0, eth1, etc.
>>>>>
>>>>> I realize that the sysconfig network scripts will come into play
>>>>> here as
>>>>> well, but that's another problem for me to tackle.
>>>>>
>>>>> Does anyone have any experience with bonded channels and Conga? I
>>>>> could
>>>>> sure use some help with this.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> steve campbell
>>>>
>>>> I use bonding extensively, but I always edit cluster.conf directly. If
>>>> conga doesn't support "bond*" device names, please file a bug in red
>>>> hat's bugzilla.
>>>>
>>>> Once the bondX device is up, it will have the IP and the "ethX"
>>>> devices can be totally ignored from the cluster's perspective. Use the
>>>> bondX device just as you would have used simple ethX devices.
>>>>
>>>> In case it helps, here is how I setup bonded interfaces on red hat
>>>> clusters for complete HA;
>>>>
>>>> https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Network
>>> Digimer,
>>>
>>> Thanks very much for the reply. I believe you had pointed out the link
>>> to me before on a more basic query. It was very helpful in giving me a
>>> real nice introduction to all the new stuff in Centos 6 for clustering.
>>>
>>> After reading this page once again, I think my question is not being
>>> understood. It seems to be a problem of mine to not state those
>>> questions plainly.
>>>
>>> In your example, you use a VM to move the entire server from one VM host
>>> to another (or how ever you have that configured). That VM is a
>>> "service" defined under the cluster and it carries the IPs along with
>>> the VM.
>>>
>>> In my situation, my cluster consists of non-VM servers. The servers are
>>> real, with an inside and outside interface and IPs. They become
>>> firewalls by moving the external IPs and iptables rules as services. So
>>> in my situation, I use "ip address" and "script" to only move the IP
>>> addresses and start and stop iptables. The IP addresses would be bonded
>>> channels, much like you do in your VMs.
>>>
>>> If I'm not mistaken, the parameters for "ip address" do not offer
>>> anything like device or interface, so I'm failing to see how I can move
>>> the IPs between nodes as bonded channels. Individual IP addresses are
>>> not a problem. It works as expected.
>>>
>>> My network experience is not strong enough to know why I'd need a bridge
>>> in my situation as well.
>>>
>>> Perhaps I should back up and consider VMs. The main problem I see there
>>> is the time it might take to shutdown one VM and start another VM as
>>> opposed to just moving IPs and starting iptables.
>>>
>>> I've still not attacked conntrack yet either, so there's plenty more for
>>> me to do.
>>>
>>> Thanks again for your very helpful reply.
>>>
>>> steve
>>
>> Ah, ok, I think I get it.
>>
>> The ip resource agent looks for the interface that matches the managed
>> IP's subnet, and uses it. So if your bondX interface has an IP on the
>> same subnet as your virtual IP, it will be used.
>>
>> Think of a bonded network device like you would a traditional mdadm
>> based RAID array. Say you have /dev/sda5 + /dev/sdb5 and they create
>> /dev/md0. Once created, you only look at/use /dev/md0 and you can
>> effectively pretend that the two backing devices no longer exist. The
>> software raid stack handles and hides failure management.
>>
>> In your case, you would, for example, take eth0 + eth1 and create
>> bond0. Once done, eth{0,1} no longer have an IP address, only the
>> bondX device does. The failure of a slaved interface is totally
>> handled behind the scenes by the bond driver. So your application
>> (cluster, iptables) will not know or care that the link changed behind
>> the scenes.
>>
>> As for the VMs;
>>
>> In the tutorial, the VMs are indeed the HA service, but you can
>> imagine your firewall in place of the VM, so far as the cluster is
>> concerned. It's just another resource. Also, if you do decide to go to
>> a VM, you can live-migrate a VM between nodes, so there is no
>> interruption. Of course, if the node backing the VM dies dramatically,
>> the VM will need to reboot on the remaining good node, causing an
>> outage of (in my experience) roughly 30 seconds. Again though, the VM
>> approach is just one of many... Making a firewall the HA service
>> directly is just fine.
>>
>> Of course, one benefit of VMs (and the reason I prefer them) is that
>> the configuration of the software in the VM is trivial... No special
>> consideration is needed on an app by app bases. Once you have your
>> first VM cluster running, you can make anything (on any supported OS) HA.
>>
>> digimer
>
> I'm not sure the gratuitous arp thing would work as effectively when
> moving a VM as it does when moving an ip address. In the firewall
> scenario, with conntrack running and gratuitous arp, there should be
> little if any delay and little to no loss of connections to be transparent.
>
> I'll try the bonded channel once I get some real servers running. For
> now, I've just used VMs to ensure the IP and iptables move as expected,
> which they appear to do. It'll also give be a chance to try some real
> fencing, which I also don't use on the VMs.
>
> Again, thanks for your documentation on how you did this all. You don't
> realize how helpful it was in understanding the newer clustering
> software. For the most part, all the examples I could find used the
> older heartbeat.
>
> steve
(Almost) All hypervisors have a fence agent now, so you can in fact use
"real" fencing with VM'ed nodes.
Given HA is your priority, be sure to use mode=1 bonding (aka
Active/Passive). It has the fastest/smoothest failover. You don't get
any aggregation of the bandwidth, but I suspect that's not a concern for
you.
As for migrating VMs, it works smoothly with node network interruption.
There is a very brief interruption when the processing actually kicks
over to the new host, but it's <1s. Again though, the real question is
tolerable down time. If you lose the node hosting a VM, you're down
until the VM reboots (say one minute, to guess high). If you make the
iptables firewall the service, then even a total node failure recovers
in seconds. The trade-off being complexity in the configuration.
I learned a lot writing those docs, and it was a great way to convince
people to help me learn the inners of clustering. So it was as much a
selfish endeavour or collecting what others know more than anything. I
love hearing that it has helped others though, so thanks! :)
digimer
PS - #linux-cluster on freenode is a good place to ask questions and
learn about clustering, too. Of course, questions here get archived.
--
Digimer
Papers and Projects: https://alteeve.com
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
