On 03/12/15 08:39 PM, Kelvin Edmison wrote: > On 12/03/2015 06:14 PM, Digimer wrote: >> On 03/12/15 02:19 PM, Kelvin Edmison wrote: >>> I am hoping that someone can help me understand the problems I'm having >>> with linux clustering for VMs. >>> >>> I am clustering 2 VMs on two separate VM hosts, trying to ensure that a >>> service is always available. The hosts and guests are both RHEL 6.7. >>> The goal is to have only one of the two VMs running at a time. >>> >>> The configuration works when we test/simulate VM deaths and graceful VM >>> host shutdowns, and administrative switchovers (i.e. clusvcadm -r ). >>> >>> However, when we simulate the sudden isolation of host A (e.g. ifdown >>> eth0), two things happen >>> 1) the VM on host B does not start, and repeated fence_xvm errors appear >>> in the logs on host B >>> 2) when the 'failed' node is returned to service, the cman service on >>> host B dies. >> If the node's host is dead, then there is no way for the survivor to >> determine the state of the lost VM node. The cluster is not allowed to >> take "no answer" as confirmation of fence success. >> >> If your hosts have IPMI, then you could add fence_ipmilan as a backup >> method where, if fence_xvm fails, it moves on and reboots the host >> itself. > > Thank you for the suggestion. The hosts do have ipmi. I'll explore it > but I'm a little concerned about what it means for the other > non-clustered VM workloads that exist on these two servers. > > Do you have any thoughts as to why host B's cman process is dying when > 'host A' returns? > > Thanks, > Kelvin It's not dieing, it's blocking. When a node is lost, dlm blocks until fenced tells it that the fence was successful. If fenced can't contact the lost node's fence method(s), then it doesn't succeed and dlm stays blocked. To anything that uses DLM, like rgmanager, it appears like the host is hung but it is by design. The logic is that, as bad as it is to hang, it's better than risking a split-brain. As for what will happen to non-cluster services, well, if I can be blunt, you shouldn't mix the two. If something is important enough to make HA, then it is important enough for dedicated hardware in my opinion. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? -- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
