Here's a patch you might like to try
Xx
Chrissie
On 01/03/13 15:26, Steven Dake wrote:
On 03/01/2013 05:26 AM, Jan Friesse wrote:
Steven Dake napsal(a):
Hi,
Speaking with Angus who spoke with Andrew about the problem of getting
Corosync to run inside cloud guests, specially OpenStack, spawned this
conversation...
Currently the problem is that with UDPU mode and a bindnetaddr set, the
interface binding code first searches for the ip address, then once
found, determines the interface which that ip address is connected to.
This would work great if cloud guests had static ip addresses which they
are assigned, but in most cases this is not the case. IP addresses are
assigned dynamically from an address pool. This means on restart,
without detailed knowledge of the cloud deployment, it is not possible
to set a bindnetaddr up that works consistently. Further complicating
matters is that the target node list IP addresses change in the
non-floating-ip model.
One simple solution is to use the following config model:
1. obtain a floating ip address from OpenStack
2. configure the guest vms with the floating IPs
3. hold floating IP for lifetime of application
The way floating IPs work (atleast in OpenStack) is that they create
iptables on the host to represent the ip address. As a result, using
floating IPs with the current totemip behavior doesn't solve the problem
because totemip will still desire to bind to a specific address.
Unfortunately the floating IPs don't present a physical interface inside
the guest VM, but are handled externally by iptables. This makes
binding impossible.
One solution to this problem is to add a new mode "inaddr_any" for
bindnetaddr. Then when corosync starts, it will bind to INADDR_ANY
interface and the udpu traffic will be routed properly by iptables.
I've found following problems with your solution:
- Corosync is inserting bind IP address (system_from filed) of all rings
to every packet. So it's needed to know WHAT IP address is in use. How
we will find it with inaddr_any? (Ignoring fact that this field may be
ignored because it should be used in multiring which is not implemented)
- Because we don't know IP addresses, how we will fill udpu members
list? How will corosync know what are other nodes to contact them?
Totem trusts all "system_from" fields in each message header, meaning it
doesn't actually look at the receipt address from the kernel IP system.
One solution to this problem is to put the host's floating IP address in t
This all starts out at instance->my_id which is set in:
https://github.com/corosync/corosync/blob/needle-2.2/exec/totemsrp.c#L4549
Which comes from totemip.
Regards
-steve
Any thoughts on a simpler solution or on this solution?
Actually I believe only way to achieve correct functionality will be
much harder solution of proper OpenStack integration... Hopefully I'm
wrong in there.
Regards
-steve
Regards,
Honza
_______________________________________________
discuss mailing list
discuss@xxxxxxxxxxxx
http://lists.corosync.org/mailman/listinfo/discuss
diff --git a/exec/totemconfig.c b/exec/totemconfig.c
index 0e1e7c6..f6c0887 100644
--- a/exec/totemconfig.c
+++ b/exec/totemconfig.c
@@ -487,6 +487,12 @@ extern int totem_config_read (
icmap_get_string("totem.cluster_name", &cluster_name);
+ icmap_get_string("totem.inaddr_any", &str);
+ if (strcmp(str, "yes") == 0)
+ totem_config->inaddr_any = 1;
+ else
+ totem_config->inaddr_any = 0;
+
totem_config->ip_version = AF_INET;
if (icmap_get_string("totem.ip_version", &str) == CS_OK) {
if (strcmp(str, "ipv4") == 0) {
diff --git a/exec/totemip.c b/exec/totemip.c
index 7ba746e..29a85a5 100644
--- a/exec/totemip.c
+++ b/exec/totemip.c
@@ -67,6 +67,40 @@ void totemip_nosigpipe(int s)
}
#endif
+/* Return an INADDR_ANY that matches the family of the passed-in totem_ip_address */
+int totemip_get_inaddr_any(const struct totem_ip_address *addr, uint16_t port, struct sockaddr_storage *saddr, int *addrlen)
+{
+ *addrlen = 0;
+
+ if (addr->family == AF_INET) {
+ struct sockaddr_in *sin = (struct sockaddr_in *)saddr;
+
+ memset(sin, 0, sizeof(struct sockaddr_in));
+ sin->sin_addr.s_addr = INADDR_ANY;
+ sin->sin_port = ntohs(port);
+ *addrlen = sizeof(struct sockaddr_in);
+#ifdef HAVE_SOCK_SIN_LEN
+ sin->sin_len = sizeof(struct sockaddr_in);
+#endif
+ }
+ if (addr->family == AF_INET6) {
+ struct sockaddr_in6 *sin = (struct sockaddr_in6 *)saddr;
+
+ memcpy(&sin->sin6_addr, &in6addr_any, *addrlen);
+ memset(sin, 0, sizeof(struct sockaddr_in6));
+ sin->sin6_port = ntohs(port);
+ *addrlen = sizeof(struct sockaddr_in6);
+#ifdef HAVE_SOCK_SIN6_LEN
+ sin->sin6_len = sizeof(struct sockaddr_in6);
+#endif
+ }
+ if (*addrlen == 0)
+ return 1;
+
+ saddr->ss_family = addr->family;
+ return 0;
+}
+
/* Compare two addresses */
int totemip_equal(const struct totem_ip_address *addr1,
const struct totem_ip_address *addr2)
diff --git a/exec/totemudpu.c b/exec/totemudpu.c
index 12ec63c..6f4e076 100644
--- a/exec/totemudpu.c
+++ b/exec/totemudpu.c
@@ -659,8 +659,14 @@ static int totemudpu_build_sockets_ip (
/*
* Bind to unicast socket used for token send/receives
* This has the side effect of binding to the correct interface
+ * unless inaddr_any was requested
*/
- totemip_totemip_to_sockaddr_convert(bound_to, instance->totem_interface->ip_port, &sockaddr, &addrlen);
+ if (instance->totem_config->inaddr_any) {
+ totemip_get_inaddr_any(bound_to, instance->totem_interface->ip_port, &sockaddr, &addrlen);
+ }
+ else {
+ totemip_totemip_to_sockaddr_convert(bound_to, instance->totem_interface->ip_port, &sockaddr, &addrlen);
+ }
res = bind (instance->token_socket, (struct sockaddr *)&sockaddr, addrlen);
if (res == -1) {
LOGSYS_PERROR (errno, instance->totemudpu_log_level_warning,
@@ -1052,7 +1058,13 @@ static int totemudpu_create_sending_socket(
/*
* Bind to sending interface
*/
- totemip_totemip_to_sockaddr_convert(&instance->my_id, 0, &sockaddr, &addrlen);
+ if (instance->totem_config->inaddr_any) {
+ fprintf(stderr, "CC: binding 2 to INADDR_ANY\n");
+ totemip_get_inaddr_any(&instance->my_id, 0, &sockaddr, &addrlen);
+ }
+ else {
+ totemip_totemip_to_sockaddr_convert(&instance->my_id, 0, &sockaddr, &addrlen);
+ }
res = bind (fd, (struct sockaddr *)&sockaddr, addrlen);
if (res == -1) {
LOGSYS_PERROR (errno, instance->totemudpu_log_level_warning,
diff --git a/include/corosync/totem/totem.h b/include/corosync/totem/totem.h
index 72a3e1a..ef687c1 100644
--- a/include/corosync/totem/totem.h
+++ b/include/corosync/totem/totem.h
@@ -181,6 +181,8 @@ struct totem_config {
unsigned int miss_count_const;
int ip_version;
+
+ int inaddr_any;
};
#define TOTEM_CONFIGURATION_TYPE
diff --git a/include/corosync/totem/totemip.h b/include/corosync/totem/totemip.h
index 533735a..9908425 100644
--- a/include/corosync/totem/totemip.h
+++ b/include/corosync/totem/totemip.h
@@ -77,6 +77,7 @@ struct totem_ip_if_address
struct list_head list;
};
+extern int totemip_get_inaddr_any(const struct totem_ip_address *addr, uint16_t port, struct sockaddr_storage *saddr, int *addrlen);
extern int totemip_equal(const struct totem_ip_address *addr1,
const struct totem_ip_address *addr2);
extern int totemip_compare(const void *a, const void *b);
diff --git a/man/corosync.conf.5 b/man/corosync.conf.5
index 9da9dbb..0c9f9d8 100644
--- a/man/corosync.conf.5
+++ b/man/corosync.conf.5
@@ -1,6 +1,6 @@
.\"/*
.\" * Copyright (c) 2005 MontaVista Software, Inc.
-.\" * Copyright (c) 2006-2012 Red Hat, Inc.
+.\" * Copyright (c) 2006-2013 Red Hat, Inc.
.\" *
.\" * All rights reserved.
.\" *
@@ -266,6 +266,13 @@ This specifies the name of cluster and it's used for automatic generating
of multicast address.
.TP
+inaddr_any
+When in updu mode, setting this to "yes" tells corosync to bind the network
+sockets to INADDR_ANY rather than the local IP address. This might be
+necessary for when running corosync in some cloud configurations that use
+dynamic IP addresses
+
+.TP
config_version
This specifies version of config file. This is converted to unsigned 64-bit int.
By default it's 0. Option is used to prevent joining old nodes with not
_______________________________________________
discuss mailing list
discuss@xxxxxxxxxxxx
http://lists.corosync.org/mailman/listinfo/discuss
